Dan Q

Even More 1999!

Spencer’s filter

Last month I implemented an alternative mode to view this website “like it’s 1999”, complete with with cursor trails, 88×31 buttons, tables for layout¹, tiled backgrounds, and even a (fake) hit counter.

One thing I’d have liked to do for 1999 Mode but didn’t get around to would have been to make the images look like it was the 90s, too.

Back then, many Web users only had graphics hardware capable of displaying 256 distinct colours. Across different platforms and operating systems, they weren’t even necessarily the same 256 colours²! But the early Web agreed on a 216-colour palette that all those 8-bit systems could at least approximate pretty well.

I had an idea that I could make my images look “216-colour”-ish by using CSS to apply an SVG filter, but didn’t implement it.

But Spencer, a long-running source of excellent blog comments, stepped up and wrote an SVG filter for me! I’ve tweaked 1999 Mode already to use it… and I’ve just got to say it’s excellent: huge thanks, Spencer!

The filter coerces colours to their nearest colour in the “Web safe” palette, resulting in things like this:

Plenty of pictures genuinely looked like that on the Web of the 1990s, especially if you happened to be using a computer only capable of 8-bit colour to view a page built by somebody who hadn’t realised that not everybody would experience 24-bit colour like they did³.

Dithering

But not all images in the “Web safe” palette looked like this, because savvy web developers knew to dither their images when converting them to a limited palette. Let’s have another go:

Dithering introduces random noise to media⁴ in order to reduce the likelihood that a “block” will all be rounded to the same value. Instead; in our picture, a block of what would otherwise be the same colour ends up being rounded to maybe half a dozen different colours, clustered together such that the ratio in a given part of the picture is, on average, a better approximation of the correct colour.

The result is analogous to how halftone printing – the aesthetic of old comics and newspapers, with different-sized dots made from few colours of ink – produces the illusion of a continuous gradient of colour so long as you look at it from far-enough away.

The other year I read a spectacular article by Surma that explained in a very-approachable way how and why different dithering algorithms produce the results they do. If you’ve any interest whatsoever in a deep dive or just want to know what blue noise is and why you should care, I’d highly recommend it.

You used to see digital dithering everywhere, but nowadays it’s so rare that it leaps out as a revolutionary aesthetic when, for example, it gets used in a video game.

All of which is to say that: I really appreciate Spencer’s work to make my “1999 Mode” impose a 216-colour palette on images. But while it’s closer to the truth, it still doesn’t quite reflect what my website would’ve looked like in the 1990s because I made extensive use of dithering when I saved my images in Web safe palettes⁵.

Why did I take the time to dither my images, back in the day? Because doing the hard work once, as a creator of graphical Web pages, saves time and computation (and can look better!), compared to making every single Web visitor’s browser do it every single time.

Which, now I think about it, is a lesson that’s still true today (I’m talking to you, developers who send a tonne of JavaScript and ask my browser to generate the HTML for you rather than just sending me the HTML in the first place!).

Footnotes

¹ Actually, my “1999 mode” doesn’t use tables for layout; it pretty much only applies a CSS overlay, but it’s deliberately designed to look a lot like my blog did in 1999, which did use tables for layout. For those too young to remember: back before CSS gave us the ability to lay out content in diverse ways, it was commonplace to use a table – often with the borders and cell-padding reduced to zero – to achieve things that today would be simple, like putting a menu down the edge of a page or an image alongside some text content. Using tables for non-tabular data causes problems, though: not only is it hard to make a usable responsive website with them, it also reduces the control you have over the order of the content, which upsets some kinds of accessibility technologies. Oh, and it’s semantically-invalid, of course, to describe something as a table if it’s not.

² Perhaps as few as 22 colours were defined the same across all widespread colour-capable Web systems. At first that sounds bad. Then you remember that 4-bit (16 colour) palettes used to look look perfectly fine in 90s videogames. But then you realise that the specific 22 “very safe” colours are pretty shit and useless for rendering anything that isn’t composed of black, white, bright red, and maybe one of a few greeny-yellows. Ugh. For your amusement, here’s a copy of the image rendered using only the “very safe” 22 colours.

³ Spencer’s SVG filter does pretty-much the same thing as a computer might if asked to render a 24-bit colour image using only 8-bit colour. Simply “rounding” each pixel’s colour to the nearest available colour is a fast operation, even on older hardware and with larger images.

⁴ Note that I didn’t say “images”: dithering is also used to produce the same “more natural” feel for audio, too, when reducing its bitrate (i.e. reducing the number of finite states into which the waveform can be quantised for digitisation), for example.

⁵ I’m aware that my footnotes are capable of nerdsniping Spencer, so by writing this there’s a risk that he’ll, y’know, find a way to express a dithering algorithm as an SVG filter too. Which I suspect isn’t possible, but who knows! 😅

Easy Socialising

This weekend I invited over a bunch of our old university buddies, and it was great.

We still didn’t feel up to a repeat of the bigger summer party we held the year before last, but we love our Abnib buddies, so put the call out to say: hey, come on over, bring a tent (or be willing to crash on a sofa bed) if you want to stay over; we’ll let the kids run themselves ragged with a water fight and cricket and football and other garden games, then put them in front of a film or two while we hang out and drink and play board games or something.

14 adults, 8 children, and a dog stand on/in front of a garden climbing frame: Dan is in the centre. — Every one of these people is awesome. Or else a dog.

The entire plan was deliberately low-effort. Drinks? We had a local brewery drop us off a couple of kegs, and encouraged people to BYOB. Food? We threw a stack of pre-assembled snacks onto a table, and later in the day I rotated a dozen or so chilled pizzas through the oven. Entertainments? Give the kids a pile of toys and the adults one another’s company.

We didn’t even do more than the bare minimum of tidying up the place before people arrived. Washing-up done? No major trip hazards on the floor? That’s plenty good enough!

I found myself recalling our university days, when low-effort ad-hoc socialising seemed… easy. We lived close together and we had uncomplicated schedules, which combined to make it socially-acceptable to “just turn up” into one another’s lives and spaces. Many were the times that people would descend upon Claire and I’s house in anticipation that there’d probably be a film night later, for example¹.

I remember one occasion a couple of decades ago, chilling with friends². Somebody – possibly Liz – commented that it’d be great if in the years to come our kids would be able to be friends with one another. I was reminded of it when our eldest asked me, of our weekend guests, “why are all of your friends’ children are so great?”

A group of adults stand around on a patio, socialising. — It’s not *the same* as those days long ago, but I’m not sure I’d want it to be. It is, however, fantastic.

What pleased me in particular was how relatively-effortless it was for us all to slip back into casually spending time together. With a group of folks who have, for the most part, all known each other for over two decades, even not seeing one another in-person for a couple of years didn’t make a significant dent on our ability to find joy in each other’s company.

Plus, being composed of such laid-back folks, it didn’t feel awkward that we had, let’s face it, half-arsed the party. Minimal effort was the order of the day, but the flipside of that was that the value-for-effort coefficient was pretty-well optimised³.

A delightful weekend that I was glad to be part of.

Footnotes

¹ That Claire and I hosted so many social events, both regular and unplanned, eventually lead us to the point that it was the kind of thing we considered whenever we moved house!

² Perhaps at the Ship & Castle, where we spent a reasonable amount of our education.

³ I’m pretty sure that if I’d have used the term “value-for-effort coefficient” at the party, though, then it’d have immediately sucked 100% of the fun out of the room.

Golf

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

…

Runners will talk about how much they enjoy the feeling of wind in their hair. Boxers won’t shut up about the grace and art of their profession. Even soccer players can be moved to wax poetical about how enjoyable it is to be part of a truly great game.

But all golfers ever talk about is how little golf they hope to play. A typical pre-match interview will go something like this:

Some guy in a blazer: Great to have you here with us, what are your goals for the first round this morning.

Golfer: Well today I hope to play as little golf as possible. Mathematically speaking the course could be done in 18 shots but that is probably physically impossible. But ideally as close to 18 as I can get. Any additional golf is bad.

Blazer: What is your strategy for avoiding the golf.

Golfer: I have a guy who follows me around to help share the burden of all this damn golf. He is going to help me out by suggesting ways to avoid playing any more golf than we have to. Of course, I pay him but his real motivation is to bring this sorry excuse for a pastime to the speediest conclusion.

Blazer: Better you than me, but good luck out there.

…

Andrew Stephens

Draw Me a Comment!

Why must a blog comment be text? Why could it not be… a drawing?¹

Red and black might be more traditional ladybird colours, but sometimes all you’ve got is blue.

I started hacking about and playing with a few ideas and now, on selected posts including this one, you can draw me a comment instead of typing one.

Just don’t tell the soup company what I’ve been working on, okay?

I opened the feature, experimentally (in a post available only to RSS subscribers²) the other week, but now you get a go! Also, I’ve open-sourced the whole thing, in case you want to pick it apart.

What are you waiting for: scroll down, and draw me a comment!

Footnotes

¹ I totally know the reasons that a blog comment shouldn’t be a drawing; I’m not completely oblivious. Firstly, it’s less-expressive: words are versatile and you can do a lot with them. Secondly, it’s higher-bandwidth: images take up more space, take longer to transmit, and that effect compounds when – like me – you’re tracking animation data too. But the single biggest reason, and I can’t stress this enough, is… the penises. If you invite people to draw pictures on your blog, you’re gonna see a lot of penises. Short penises, long penises, fat penises, thin penises. Penises of every shape and size. Some erect and some flacid. Some intact and some circumcised. Some with hairy balls and some shaved. Many of them urinating or ejaculating. Maybe even a few with smiley faces. And short of some kind of image-categorisation AI thing, you can’t realistically run an anti-spam tool to detect hand-drawn penises.

² I’ve copied a few of my favourites of their drawings below. Don’t forget to subscribe if you want early access to any weird shit I make.

How To: Sunday

Permanent Record

To:

****@fulwoodacademy.co.uk

From:

“Dan Q” <***@danq.me>

Subject:

Subject Access Request – Dan Q, pupil Sep 1992 – Jun 1997

Date:

Tue, 23 Jul 2024 15:18:07 +0100

To Whom It May Concern,

Please supply the personal data you hold about me, per data protection law. Specifically, I’m looking for: a list of all offences for which I was assigned detention at school.

Please find attached a variety of documentation which I feel proves my identity and the legitimacy of this request. If there’s anything else you need or you have further questions, please feel free to email me.

Thanks in advance;

Dan Q

To:

“Dan Q” <***@danq.me>

From:

“Jodie Clayton” <*.*******@fulwoodacademy.co.uk>

Subject:

Re: Subject Access Request – Dan Q, pupil Sep 1992 – Jun 1997

Date:

Fri, 26 Jul 2024 10:48:33 +0100

Dear Dan Q,

We do not retain records of detentions of former pupils, and we certainly have no academic records of pupils going back thirty years ago.

Jodie Clayton | Office Manager with Cover and Admissions
Black Bull Lane, Fulwood, Preston, PR2 9YR
+44 (0) 1772 719060

To:

“Jodie Clayton” <*********@fulwoodacademy.co.uk>

From:

“Dan Q” <***@danq.me>

Subject:

Re: Subject Access Request – Dan Q, pupil Sep 1992 – Jun 1997

Date:

Fri, 26 Jul 2024 17:00:49 +0100

But, but… I was always told that this would go on my permanent record. Are you telling me that teachers lied to me? What else is fake!?

Maybe I will always have a calculator with me and I won’t actually need to know how to derive a square root using a pen and paper. Maybe nobody will ever care what my GCSE results are for every job I apply for. Maybe my tongue isn’t divided into different taste areas capable of picking out sweet, salty, bitter etc. flavours. Maybe practicing my handwriting won’t be an essential skill I use every day.

And maybe I will amount to something despite never turning in any History homework, Mr. Needham!

Dan Q

Tidying WordPress’s HTML

Terence Eden, who’s apparently inspiring several posts this week, recently shared a way to attach a hook to WordPress’s get_the_post_thumbnail() function in order to remove the extraneous “closing mark” from the (self-closing in HTML) <img> element.

By default, WordPress outputs e.g. <img src="..." />, where <img src="..."> would suffice.

It’s an inconsequential difference for most purposes, but apparently it bugs him, so he fixed it… although he went on to observe that he hadn’t managed to successfully tackle all the instances in which WordPress was outputting redundant closing marks.

This is a problem that I’ve already solved here on my blog. My solution’s slightly hacky… but it works!

My Solution: Runing HTMLTidy over WordPress

Tidy is an excellent tool for tiding up HTML! I used to use its predecessor back in the day for all kind of things, but it languished for a few years and struggled with support for modern HTML features. But in 2015 it made a comeback and it’s gone from strength to strength ever since.

I run it on virtually all pages produced by DanQ.me (go on, click “View Source” and see for yourself!), to:

Standardise the style of the HTML code and make it easier for humans to read¹.
Bring old-style emphasis tags like <i>, in my older posts, into a more-modern interpretation, like <em>.
Hoist any inline <style> blocks to the <head>, and detect any repeated inline style="..."s to convert to classes.
Repair any invalid HTML (browsers do this for you, of course, but doing it server-side makes parsing easier for the browser, which might matter on more-lightweight hardware).

WordPress isn’t really designed to have Tidy bolted onto it, so anything it likely to be a bit of a hack, but here’s my approach:

Install libtidy-dev and build the PHP bindings to it.
Note that if you don’t do this the code might appear to work, but it won’t actually tidy anything².
Add a new output buffer to my theme’s header.php³, with a callback function: ob_start('tidy_entire_page').
Without an corresponding ob_flush or similar, this buffer will close and the function will be called when PHP finishes generating the page.
Define the function tidy_entire_page($buffer)
Have it instantiate Tidy ($tidy = new tidy) and use $tidy->parseString (with your buffer and Tidy preferences) to tidy the code, then return $tidy.
Ensure that you’re caching the results!
You don’t want to run this every page load for anonymous users! WP Super Cache on “Expert” mode (with the requisite webserver configuration) might help.

I’ve open-sourced a demonstration that implements a child theme to TwentyTwentyOne to do this: there’s a richer set of instructions in the repo’s readme. If you want, you can run my example in Docker and see for yourself how it works before you commit to trying to integrate it into your own WordPress installation!

Footnotes

¹ I miss the days when most websites were handwritten and View Source typically looked nice. It was great to learn from, too, especially in an age before we had DOM debuggers. Today: I can’t justify dropping my use of a CMS, but I can make my code readable.

² For a few of its extensions, some PHP developer made the interesting choice to fail silently if the required extension is missing. For example: if you don’t have the zip extension enabled you can still use PHP to make ZIP files, but they won’t be compressed. This can cause a great deal of confusion for developers! A similar issue exists with tidy: if it isn’t installed, you can still call all of the methods on it… they just don’t do anything. I can see why this decision might have been made – to make the language as portable as possible in production – but I’d prefer if this were an optional feature, e.g. you had to set try_to_make_do_if_you_are_missing_an_extension=yes in your php.ini to enable it, or if it at least logged that it had done so.

³ My approach probably isn’t suitable for FSE (“block”) themes, sorry.

Stolen from Costa

Shared Email Addresses

Email Antipatterns

There are two particular varieties of email address that I don’t often see, but I’ve been known to ridicule when I have:

Geographically-based personal email addresses, e.g. OurHouseName@example.com. These always seemed to me to undermine one of the single-best things about an email address compared to postal mail – that they don’t change when you move house!¹
Shared/couple email addresses, e.g. MrAndMrsSmith@example.net. These make me want to scream “You know email addresses are basically free, right? You don’t have to share one!” Even back when most people got their email address directly from their dial-up provider, most ISPs offered some number of addresses (e.g. five).

If you’ve come across either of the above before, there’s… perhaps a reasonable chance that it was in the possession of somebody born before 1960 (and the older, the more-likely)².

You’ll never catch me doing that!

I found myself thinking about this as I clicked the “No” button on a poll by Terence Eden that asked whether I used a “shared” email address when in a stable long-term relationship.

Terence Eden (@Edent@mastodon.social) on Mastodon asks: "If you're currently in a stable, long term relationship with someone - do you have a joint email address with them?" — Of course I don’t! Why would I? Oh… wait…

It wasn’t until after I clicked “No” that I realised that, in actual fact, I have had multiple email addresses that I’ve share with significant other(s). And more than that, sometimes they’ve been geographically-based! What’s going on?

I’ve routinely had domains or subdomains that I’ve used to represent a place that I live. They’re convenient for when you want to give somebody a short web address which’ll take them to a page with directions to you and links to your location in a variety of different services and formats.

And by that point, you might as well have an email alias, e.g. all@myhouse.example.org, that forwards on email to, well, all the adults at the house. What I’ve described there is, after a fashion, a shared email address tied to a geographical location. But we don’t ever send anything from it. Nor do we use it for any kind of personal communication with anybody outside the house.

We don’t give out these all@ addresses (or their aliases: every company gets their own) to people willy-nilly. But they’re useful for shared services that send automated emails to us all. For example:

Giving a forwarding alias to the supermarket means that receipts (listing any unavailable products) g0 to all of us, and whoever’s meal plan’s been scuppered by an awkward substitution will know what’s up.
Using a forwarding alias with the household Netflix account means anybody can use the “send me a sign-in link” feature to connect a new device.
When confirming that you’ve sent money to a service provider, CC’ing one of these nice, short aliases provides a quick way to let the others know that a bill’s been paid (this one’s especially useful where, like me, you live in a 3+ adult household and otherwise you’d be having to add multiple people to the CC field).

Sure, the need for most of these solutions would evaporate instantly if more services supported multi-user or delegated access³. But outside of that fantasy world, shared aliases seem to be pretty useful!

Footnotes

¹ The most ill-conceived example of geographically-based email addresses I’ve ever seen came from a a 2003 proposal by then-MP Derek Wyatt, who proposed that the domain name part of every single email address should contain not only the country of the owner (e.g. .uk) but also their complete postcode. He was under the delusion that this would somehow prevent spam. Even ignoring the immense technical challenges of his proposal and the impossibility of policing it across the borders of every country that uses email… it probably wouldn’t even be effective at his stated goal. I’ll let The Register take it from here.

² No ageism intended: I suspect that the phenomenon actually stems from the fact that as email took off in the noughties this demographic who were significantly more-likely than younger folks to have (a) a very long-term home that they didn’t anticipate moving out of any time soon, and (b) an existing anticipation that people and companies wrote to them as a couple, not individually.

³ I’d love it if the grocery delivery sites would let multiple “accounts”, by mutual consent, share a delivery slot, destination, and payment method. It’d be cool to know that we could e.g. have a houseguest and give them temporary access to a specific order that was scheduled for during their stay. But that’s probably a lot of work for very little payoff if you’re busy running a supermarket.

Blogging Like It’s 1999

In anticipation of WWW Day on 1 August, some work colleagues and I were sharing pictures of the first (or early) websites we worked on. I was pleased to be able to pull out a screenshot of how my blog looked back in 1999!

Because I’m such a digital preservationist, many of those ancient posts are still available on my blog, so I also shared a photo of me browsing the same content on my blog as it is today, side-by-side with that 25+-year-old screenshot.¹

Dan poses in front of circa 1999 and present-day copies of his blog, both showing posts from January 1999. — The posts are in reverse-chronological order now, rather than chronological order, but the content’s all the same (even though the design is now very different and, of course, responsive!).

Update: This photo eventually appeared on a LinkedIn post on Automattic’s profile.

This inspired me to make a toggleable “alternate theme” for my blog: 1999 Mode.

Switch to it, and you’ll see a modern reinterpretation of my late-1990s blog theme, featuring:

A “table-like” layout.²
White text on a black marbled “seamless texture” background, just like you’d expect on any GeoCities page.
Pre-rendered fire text³, including – of course – animated GIFs.⁴
A (fake) hit counter.
A stack of 88×31 micro-banners, as was all the rage at the time. (And seem to be making a comeback in IndieWeb circles…)
Cursor trails (with thanks to Tim Holman)!
I’ve even applied img { image-rendering: crisp-edges; } to try to compensate for modern browsers’ capability for subpixel rendering when rescaling images: let them eat pixels!⁵

I’ve added 1999 Mode to my April Fools gags so, like this year, if you happen to visit my site on or around 1 April, there’s a change you’ll see it in 1999 mode anyway. What fun!

I think there’s a possible future blog post about Web design challenges of the 1990s. Things like: what it the user agent doesn’t support images? What if it supports GIFs, but not animated ones (some browsers would just show the first frame, so you’d want to choose your first frame appropriately)? How do I ensure that people see the right content if they skip my frameset? Which browser-specific features can I safely use, and where do I need a fallback⁶? Will this work well on all resolutions down to 640×480 (minus browser chrome)? And so on.

Any interest in that particular rabbit hole of digital history?

Footnotes

¹ Some of the addresses have changed, but from Summer 2003 onwards I’ve had a solid chain of redirects in place to try to keep content available via whatever address it was at. Because Cool URIs Don’t Change. This occasionally turns out to be useful!

² Actually, the entire theme is just a CSS change, so no tables are added. But I’ve tried to make it look like I’m using tables for layout, because that (and spacer GIFs) were all we had back in the day.

³ Obviously the title saying “Dan Q” is modern, because that wasn’t even my name back then, but this is more a reimagining of how my site would have looked if I were transported back to 1999 and made to do it all again.

⁴ I was slightly obsessed for a couple of years in the late 90s with flaming text on black marble backgrounds. The hit counter in my screenshot above – with numbers on fire – was one I made, not a third-party one; and because mine was the only one of my friends’ hosts that would let me run CGIs, my Perl script powered the hit counters for most of my friends’ sites too.

⁵ I considered, but couldn’t be bothered, implementing an SVG CSS filter: to posterize my images down to 8-bit colour, for that real “I’m on an old graphics card” feel! If anybody’s already implemented such a thing under a license that I can use, let me know and I’ll integrate it!

⁶ And what about those times where using a fallback might make things worse, like how Netscape 7 made the <blink><marquee> combination unbearable!

Where Should Visual Programming Go?

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

…

Level 3: Diagrams are code

This is what the endgame should be IMO. Some things are better represented as text. Some are best understood visually. We should mix and match what works best on a case-by-case basis. Don’t try to visualize simple code. Don’t try to write code where a diagram is better.

One of the attempts was Luna. They tried dual representation: everything is code and diagram at the same time, and you can switch between the two:

But this way, you are not only getting benefits of both ways, you are also constrained by both text and visual media at the same time. You can’t do stuff that’s hard to visualize (loops, recursions, abstractions) AND you can’t do stuff that’s hard to code.

…

Niki

Interesting thoughts from Niki (and from Sebastian Bensusan) on how diagrams and code might someday be intertwined as first class citizens (but not in the gross ways you might have come across in the past when people have tried to sell you on “visual programming”).

As Niki wrote about what he calls levels 2 and 3 of the concept – in which diagrams and code are intrinsically linked I found myself thinking about Twine, a programming language (or framework? or tool?… not sure how best to describe or define it!) intended for making interactive “choose your own adventure”-style hypertext fiction.

Twine’s sort-of a level 2 implementation of visual programming: the code (scene descriptions) is mostly what’s responsible for feeding the diagram. But that’s not entirely true: it’s possible to create new nodes in your story graph in a completely visual way, and then dip into them to edit their contents and imply how they link to others.

It’s possible that the IF engine community – who are working to lower the barriers to programming in order to improve accessibility to people who are fiction authors first, developers second – are ahead of the curve in the area of visual programming. Consider for example how Inform’s automated test framework graphs the permutations you (or your human testers) try, and allow you to “bless” (turn into assertions) the results so that regression testing becomes visually automated affair:

The Elegance of the ASCII Table

Duration 15:19

Podcast Version

This post is also available as a podcast. Listen here, download for later, or subscribe wherever you consume podcasts.

If you’ve been a programmer or programming-adjacent nerd¹ for a while, you’ll have doubtless come across an ASCII table.

An ASCII table is useful. But did you know it’s also beautiful and elegant.

ASCII‘s still very-much around; even if you’re transmitting modern Unicode³ the most-popular encoding format UTF-8 is specifically-designed to be backwards-compatible with ASCII! If you decoded this page as ASCII you’d get the gist of it… so long as you ignored the garbage characters at the end of this sentence! 😁

History

ASCII was initially standardised in X3.4-1963 (which just rolls off the tongue, doesn’t it?) which assigned meanings to 100 of the potential 128 codepoints presented by a 7-bit⁴ binary representation: that is, binary values 0000000 through 1111111:

If you’ve already guessed where I’m going with this, you might be interested to look at the X3.4-1963 table and see that yes, many of the same elegant design choices I’ll be talking about later already existed back in 1963. That’s really cool!

Table

In case you’re not yet intimately familiar with it, let’s take a look at an ASCII table. I’ve colour-coded some of the bits I think are most-beautiful:

That table only shows decimal and hexadecimal values for each character, but we’re going to need some binary too, to really appreciate some of the things that make ASCII sublime and clever.

Control codes

The first 32 “characters” (and, arguably, the final one) aren’t things that you can see, but commands sent between machines to provide additional instructions. You might be familiar with carriage return (0D) and line feed (0A) which mean “go back to the beginning of this line” and “advance to the next line”, respectively⁵. Many of the others don’t see widespread use any more – they were designed for very different kinds of computer systems than we routinely use today – but they’re all still there.

32 is a power of two, which means that you’d rightly expect these control codes to mathematically share a particular “pattern” in their binary representation with one another, distinct from the rest of the table. And they do! All of the control codes follow the pattern 00_____: that is, they begin with two zeroes. So when you’re reading 7-bit ASCII⁶, if it starts with 00, it’s a non-printing character. Otherwise it’s a printing character.

Not only does this pattern make it easy for humans to read (and, with it, makes the code less-arbitrary and more-beautiful); it also helps if you’re an ancient slow computer system comparing one bit of information at a time. In this case, you can use a decision tree to make shortcuts.

Two rolls of punched paper tape. — That there’s one exception in the control codes: *DEL* is the *last* character in the table, represented by the binary number `1111111`. This is a historical throwback to paper tape, where the keyboard would punch some permutation of seven holes to represent the ones and zeros of each character. You can’t *delete* holes once they’ve been punched, so the only way to mark a character as invalid was to rewind the tape and punch out *all* the holes in that position: i.e. all `1`s.

Space

The first printing character is space; it’s an invisible character, but it’s still one that has meaning to humans, so it’s not a control character (this sounds obvious today, but it was actually the source of some semantic argument when the ASCII standard was first being discussed).

Putting it numerically before any other printing character was a very carefully-considered and deliberate choice. The reason: sorting. For a computer to sort a list (of files, strings, or whatever) it’s easiest if it can do so numerically, using the same character conversion table as it uses for all other purposes⁷. The space character must naturally come before other characters, or else John Smith won’t appear before Johnny Five in a computer-sorted list as you’d expect him to.

Being the first printing character, space also enjoys a beautiful and memorable binary representation that a human can easily recognise: 0100000.

Numbers

The position of the Arabic numbers 0-9 is no coincidence, either. Their position means that they start with zero at the nice round binary value 0110000 (and similarly round hex value 30) and continue sequentially, giving:

Binary	Hex	Decimal digit (character)
`011 0000`	`30`	0
`011 0001`	`31`	1
`011 0010`	`32`	2
`011 0011`	`33`	3
`011 0100`	`34`	4
`011 0101`	`35`	5
`011 0110`	`36`	6
`011 0111`	`37`	7
`011 1000`	`38`	8
`011 1001`	`39`	9

The last four digits of the binary are a representation of the value of the decimal digit depicted. And the last digit of the hexadecimal representation is the decimal digit. That’s just brilliant!

If you’re using this post as a way to teach yourself to “read” binary-formatted ASCII in your head, the rule to take away here is: if it begins 011, treat the remainder as a binary representation of an actual number. You’ll probably be right: if the number you get is above 9, it’s probably some kind of punctuation instead.

Shifted Numbers

Subtract 0010000 from each of the numbers and you get the shifted numbers. The first one’s occupied by the space character already, which is a shame, but for the rest of them, the characters are what you get if you press the shift key and that number key at the same time.

“No it’s not!” I hear you cry. Okay, you’re probably right. I’m using a 105-key ISO/UK QWERTY keyboard and… only four of the nine digits 1-9 have their shifted variants properly represented in ASCII.

That, I’m afraid, is because ASCII was based not on modern computer keyboards but on the shifted positions of a Remington No. 2 mechanical typewriter – whose shifted layout was the closest compromise we could find as a standard at the time, I imagine. But hey, you got to learn something about typewriters today, if that’s any consolation.

A Remington Portable No. 3 typewriter. — Bonus fun fact: early mechanical typewriters omitted a number 1: it was expected that you’d use the letter I. That’s fine for printed work, but not much help for computer-readable data.

Letters

Like the numbers, the letters get a pattern. After the @-symbol at 1000000, the uppercase letters all begin 10, followed by the binary representation of their position in the alphabet. 1 = A = 1000001, 2 = B = 1000010, and so on up to 26 = Z = 1011010. If you can learn the numbers of the positions of the letters in the alphabet, and you can count in binary, you now know enough to be able to read any ASCII uppercase letter that’s been encoded as binary⁸.

And once you know the uppercase letters, the lowercase ones are easy too. Their position in the table means that they’re all exactly 0100000 higher than the uppercase variants; i.e. all the lowercase letters begin 11! 1 = a = 1100001, 2 = b = 1100010, and 26 = z = 1111010.

If you’re wondering why the uppercase letters come first, the answer again is sorting: also the fact that the first implementation of ASCII, which we saw above, was put together before it was certain that computer systems would need separate character codes for upper and lowercase letters (you could conceive of an alternative implementation that instead sent control codes to instruct the recipient to switch case, for example). Given the ways in which the technology is now used, I’m glad they eventually made the decision they did.

Beauty

There’s a strange and subtle charm to ASCII. Given that we all use it (or things derived from it) literally all the time in our modern lives and our everyday devices, it’s easy to think of it as just some arbitrary encoding.

But the choices made in deciding what streams of ones and zeroes would represent which characters expose a refined logic. It’s aesthetically pleasing, and littered with historical artefacts that teach us a hidden history of computing. And it’s built atop patterns that are sufficiently sophisticated to facilitate powerful processing while being coherent enough for a human to memorise, learn, and understand.

Footnotes

¹ Programming-adjacent? Yeah. For example, geocachers who’ve ever had to decode a puzzle-geocache where the coordinates were presented in binary (by which I mean: a binary representation of ASCII) are “programming-adjacent nerds” for the purposes of this discussion.

² In both the book and the film, Mark Watney divides a circle around the recovered Pathfinder lander into segments corresponding to hexadecimal digits 0 through F to allow the rotation of its camera (by operators on Earth) to transmit pairs of 4-bit words. Two 4-bit words makes an 8-bit byte that he can decode as ASCII, thereby effecting a means to re-establish communication with Earth.

³ Y’know, so that you can type all those emoji you love so much.

⁴ ASCII is often thought of as an 8-bit code, but it’s not: it’s 7-bit. That’s why virtually every ASCII message you see starts every octet with a zero. 8-bits is a convenient number for transmission purposes (thanks mostly to being a power of two), but early 8-bit systems would be far more-likely to use the 8th bit as a parity check, to help detect transmission errors. Of course, there’s also nothing to say you can’t just transmit a stream of 7-bit characters back to back!

⁵ Back when data was sent to teletype printers these two characters had a distinct different meaning, and sometimes they were so slow at returning their heads to the left-hand-side of the paper that you’d also need to send a few null bytes e.g. 0D 0A 00 00 00 00 to make sure that the print head had gotten settled into the right place before you sent more data: printers didn’t have memory buffers at this point! For compatibility with teletypes, early minicomputers followed the same carriage return plus line feed convention, even when outputting text to screens. Then to maintain backwards compatibility with those systems, the next generation of computers would also use both a carriage return and a line feed character to mean “next line”. And so, in the modern day, many computer systems (including Windows most of the time, and many Internet protocols) still continue to use the combination of a carriage return and a line feed character every time they want to say “next line”; a redundancy build for a chain of backwards-compatibility that ceased to be relevant decades ago but which remains with us forever as part of our digital heritage.

⁶ Got 8 binary digits in front of you? The first digit is probably zero. Drop it. Now you’ve got 7-bit ASCII. Sorted.

⁷ I’m hugely grateful to section 13.8 of Coded Character Sets, History and Development by Charles E. Mackenzie (1980), the entire text of which is available freely online, for helping me to understand the importance of the position of the space character within the ASCII character set. While most of what I’ve written in this blog post were things I already knew, I’d never fully grasped its significance of the space character’s location until today!

⁸ I’m sure you know this already, but in case you’re one of today’s lucky 10,000 to discover that the reason we call the majuscule and minuscule letters “uppercase” and “lowercase”, respectively, dates to 19th century printing, when moveable type would be stored in a box (a “type case”) corresponding to its character type. The “upper” case was where the capital letters would typically be stored.

Good Food, Bad Authorisation

I was browsing (BBC) Good Food today when I noticed something I’d not seen before: a “premium” recipe, available on their “app only”:

I clicked on the “premium” recipe and… it looked just like any other recipe. I guess it’s not actually restricted after all?

Just out of curiosity, I fired up a more-vanilla web browser and tried to visit the same page. Now I saw an overlay and modal attempting¹ to restrict access to the content:

It turns out their entire effort to restrict access to their premium content… is implemented in client-side JavaScript. Even when I did see the overlay and not get access to the recipe, all I needed to do was open my browser’s debugger and run document.body.classList.remove('tp-modal-open'); for(el of document.querySelectorAll('.tp-modal, .tp-backdrop')) el.remove(); and all the restrictions were lifted.

What a complete joke.

Why didn’t I even have to write my JavaScript two-liner to get past the restriction in my primary browser? Because I’m running privacy-protector Ghostery, and one of the services Ghostery blocks by-default is one called Piano. Good Food uses Piano to segment their audience in your browser, but they haven’t backed that by any, y’know, actual security so all of their content, “premium” or not, is available to anybody.

I’m guessing that Immediate Media (who bought the BBC Good Food brand a while back and have only just gotten around to stripping “BBC” out of the name) have decided that an ad-supported model isn’t working and have decided to monetise the site a little differently². Unfortunately, their attempt to differentiate premium from regular content was sufficiently half-hearted that I barely noticed that, too, gliding through the paywall without even noticing were it not for the fact that I wondered why there was a “premium” badge on some of their recipes.

Recipes probably aren’t considered a high-value target, of course. But I can tell you from experience that sometimes companies make basically this same mistake with much more-sensitive systems. The other year, for example, I discovered (and ethically disclosed) a fault in the implementation of the login forms of a major UK mobile network that meant that two-factor authentication could be bypassed entirely from the client-side.

These kinds of security mistakes are increasingly common on the Web as we train developers to think about the front-end first (and sometimes, exclusively). We need to do better.

Footnotes

¹ The fact that I could literally see the original content behind the modal was a bit of a giveaway that they’d only hidden it, not actually protected it in any way.

² I can see why they’d think that: personally, I didn’t even know there were ads on the site until I did the experiment above: turns out I was already blocking them, too, along with any anti-ad-blocking scripts that might have been running alongside.

Derpy McBlepsleep

Derpy McBlepsleep, diligently guarding the front door just in case somebody comes by with treats.

Write Websites

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Enbies and gentlefolk of the class of ‘24:

Write websites.

If I could offer you only one tip for the future, coding would be it. The long term benefits of coding websites remains unproved by scientists, however the rest of my advice has a basis in the joy of the indie web community’s experiences. I will dispense this advice now:

Enjoy the power and beauty of PHP; or never mind. You will not understand the power and beauty of PHP until your stack is completely jammed. But trust me, in 20 years you’ll look back at your old sites and recall in a way you can’t grasp now, how much possibility lay before you and how simple and fast they were. JS is not as blazingly fast as you imagine.

Don’t worry about the scaling; or worry, but know that premature scalability is as useful as chewing bubble gum if your project starts cosy and small. The real troubles on the web are apt to be things that never crossed your worried mind; if your project grows, scale it up on some idle Tuesday.

Code one thing every day that amuses you.

…

Sara Joy

Well that’s made my day.

I can’t say I loved Baz Luhrmann’s Everybody’s Free To Wear Sunscreen. I’m not sure it’s possible for anybody who lived through it being played to death in the late 1990s; a period of history when a popular song was basically inescapable. Also, it got parodied a lot. I must’ve seen a couple of dozen different parodies of varying quality in the early 2000s.

But it’s been long enough that I was, I guess, ready for one. And I couldn’t conceive of a better topic.

Y’see: the very message of the value of personal websites is, like Sunscreen, a nostalgic one. When I try to sell people on the benefits of a personal digital garden or blog, I tend to begin by pointing out that the best time to set up your own website is… like 20+ years ago.

But… the second-best time to start a personal website is right now. With cheap and free static hosting all over the place (and more-dynamic options not much-more expensive) and domain names still as variably-priced as they ever were, the biggest impediment is the learning curve… which is also the fun part! Siloed social media is either eating its own tail or else fighting to adapt to once again be part of a more-open Web, and there’s nothing that says “I’m part of the open Web” like owning your own online identity, carving out your own space, and expressing yourself there however you damn well like.

As always, this is a drum I’ll probably beat until I die, so feel free to get in touch if you want some help getting set up on the Web.