Why does SSH send 100 packets per keystroke?

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Further analysis on a smaller pcap pointed to these mysterious packets arriving ~20ms apart.

This was baffling to me (and to Claude Code). We kicked around several ideas like:

  • SSH flow control messages
  • PTY size polling or other status checks
  • Some quirk of bubbletea or wish

One thing stood out – these exchanges were initiated by my ssh client (stock ssh installed on MacOS) – not by my server.

In 2023, ssh added keystroke timing obfuscation. The idea is that the speed at which you type different letters betrays some information about which letters you’re typing. So ssh sends lots of “chaff” packets along with your keystrokes to make it hard for an attacker to determine when you’re actually entering keys.

That makes a lot of sense for regular ssh sessions, where privacy is critical. But it’s a lot of overhead for an open-to-the-whole-internet game where latency is critical.

Keystroke timing obfuscation: I could’ve told you that! Although I wouldn’t necessarily have leapt to the possibility of mitigating it server-side by patching-out support for (or at least: the telegraphing of support for!) it; that’s pretty clever.

Altogether this is a wonderful piece demonstrating the whole “engineer mindset”. Detecting a problem, identifying it, understanding it, fixing it, all tied-up in an engaging narrative.

And after playing with his earlier work, ssh tiny.christmas – which itself inspired me to learn a little Bubble Tea/Wish (I’ve got Some Ideas™️) – I’m quite excited to see where this new ssh-based project of Royalty’s is headed!

Nostalgia, Music, and Computers

Duration

Podcast Version

This post is also available as a podcast. Listen here, download for later, or subscribe wherever you consume podcasts.

This is a blog post about things that make me nostalgic for other things that, objectively, aren’t very similar…

When I hear Dawnbreaker, I feel like I’m nine years old…

…and I’ve been allowed to play OutRun on the arcade cabinet at West View Leisure Centre. My swimming lesson has finished, and normally I should go directly home.

On those rare occasions I could get away1 with a quick pause in the lobby for a game, I’d gravitate towards the Wonderboy machine. But there was something about the tactile controls of OutRun‘s steering wheel and pedals that gave it a physicality that the “joystick and two buttons” systems couldn’t replicate.

The other thing about OutRun was that it always felt… fast. Like, eye-wateringly fast. This was part of what gave it such appeal2.

OutRun‘s main theme, Magical Sound Shower, doesn’t actually sound much like Dawnbreaker. But both tracks somehow feel like… “driving music”?

(It should, I suppose: Metrik wrote Dawnbreaker explicitly for that purpose in the first place, for use in a videogame I haven’t played3.)

But somehow when I’m driving or cycling and it this song comes on, I’m instantly transported back to those occasionally-permitted childhood games of OutRun4.

When I start a new Ruby project, I feel like I’m eleven years old…

…and I’m writing Locomotive BASIC on the family’s Amstrad CPC. Like many self-taught coders in the 1980s, my journey as a programmer begin with BASIC. When I transitioned from that to more “grown-up” languages5 I missed the feeling of programming in an environment where every line brought me joy.

Animation of an Amstrad CPC 6128 on which a program is typed and then executed. The program clears the screen and then prints the message 'Thanks for visiting DANQ.ME".
It’s not quite a HELLO WORLD, but it’s pretty-similar.

At first I assumed that the tedious bits and the administrative overhead (linking, compiling, syntactical surprises, arcane naming conventions…) was just what “real”, “grown-up” programming was supposed to feel like. But Ruby helped remind me that programming can be fun for its own sake. Not just because of the problems you’re solving or the product you’re creating, but just for the love of programming.

The experience of starting a new Ruby project feels just like booting up my Amstrad CPC and being able to joyfully write code that will just work.

I still learn new programming languages because, well, I love doing so. But I’m yet to find one that makes me want to write poetry in it in the way that Ruby does.

When I hear In Yer Face, I feel like I’m thirteen years old…

…and I’m painting Advanced HeroQuest miniatures6 in the attic at my dad’s house.

I’ve cobbled together a stereo system of my very own, mostly from other people’s castoffs, and set it up in “The Den”, our recently-converted attic7, and my friends and I would make and trade mixtapes with one another. One tape began with 808 State’s In Yer Face8, and it was often the tape that I would put on when I’d sit down to paint.

Several jigsaw-edged board game pieces lay out a dungeon map, with painted plastic minatures representing doors and characters. A party of four adventurers have just opened a door into a chamber containing five skaven (ratmen), guarding a treasure chest.
Advanced HeroQuest came with some fabulously ornate secondary components, like the doors that were hinged so their their open/closed state could be toggled, and I spent way too long painting almost the entirety of my base set.

In a world before CD audio took off, “shuffle” wasn’t a thing, and we’d often listen to all of the tracks on a medium in sequence9.

That was doubly true for tapes, where rewinding and fast-forwarding took time and seeking for a particular track was challenging compared to e.g. vinyl. Any given song would loop around a lot if I couldn’t be bothered to change tapes, instead just flipping again and again10. But somehow it’s whenever I hear In Yer Face11 that I’m transported right back to that time, in a reverie so corporeal that I can almost smell the paint thinner.

When I see a personal Web page, I (still) feel like I’m fifteen years old…

…and the Web is on the cusp of becoming the hot “killer application” for the Internet. I’ve been lucky enough to be “online” for a few years by now12, and basic ISP-provided hosting would very soon be competing with cheap, free, and ad-supported services like Geocities to be “the place” to keep your homepage.

Since its early days, the Web has always been an expressive medium. Open a Web browser, and you’re seeing a blank canvas of potential. And with modern browser debug tools, you don’t even have to reach for your text editor to begin to create in that medium.

Fresh web browser, semitransparent, on an artistic 'airy' background, with a caption to say 'The entire potential of the Web, and by proxy, the World, exists within this newly-opened window.
I don’t often see a browser with no tabs open13. But a fresh tab still gives me a tingle when I remember that it might take me anywhere!

The limitations of that medium in the pre-CSS era were a cause for inspiration, not confinement: web pages of the mid-1990s would use all kinds of imaginative tricks to lay out and style their content!

Nowadays, even with a hugely-expanded toolbox, virtually every corporate homepage fundamentally looks the same:

  • Logo in the top left
  • Search and login in the top right, if applicable
  • A cookie/privacy notice covering everything until you work out the right incantation to make it go away without surrendering your firstborn child
  • A “hero banner
  • Some “below the fold” content that most people skip over
  • A fat footer with several columns of links, to ensure that all the keywords are there so that people never have to see this page and the search engine will drop them off at relevant child page and not one of their competitors
  • Finally, a line of icons representing various centralised social networks: at least one is out-of-date, either because (a) it’s been renamed, (b) it’s changed its branding, or (c) nobody with any moral fortitude uses that network any more14

But before the corporate Web became the default, personal home pages brought a level of personality that for a while I worried was forever dead.

But… personal home pages didn’t die: everybody is free to write websites, so they’re still out there15, and they’re amazing. Look at this magic:

A handful of the personal home pages I visited while writing this article16. Don’t they just make you want to give the Web an enormous hug?

Last year, I wrote:

Writing HTML is punk rock. A “platform” is the tool of the establishment.

That still feels right to me. 🤘


So… it turns out that I get nostalgic about technology in the same way as I get nostalgic about music.

Footnotes

1 My dad in particular considered arcade games financially wasteful when we, y’know, had a microcomputer at home that could load a text-based adventure from an audiotape and be ready to play in “only” about 3-5 minutes.

2 Have you played Sonic Racing: CrossWorlds? The first time I played it I was overwhelmed by the speed and colours of the game: it’s such a high-octane visual feast. Well that’s what OutRun felt like to those of us who, in the 1980s, were used to much-simpler and slower arcade games.

3 Also, how cool is it that Metrik has a blog, in this day and age? Max props.

4 Did you hear, by the way, that there’s talk of a movie adaptation of OutRun, which could turn out to be the worst videogame-to-movie concept that I’ll ever definitely-watch.

5 In very-approximate order: C, Assembly, Pascal, HTML, Perl, Visual Basic (does that even count as a “grown-up” language?), Java, Delphi, JavaScript, PHP, SQL, ASP (classic, pre-.NET), CSS, Lisp, C#, Ruby, Python (though I didn’t get on with it so well), Go, Elixir… plus many others I’m sure!

6 Or possibly they were Warhammer Quest miniatures by this point; probably this memory spans one, and also the other, blended together.

7 Eventually my dad and I gave up on using the partially-boarded loft to intermittently build a model railway layout, mostly using second-hand/trade-in parts from “Trains & Transport”, which was exactly the nerdy kind of model shop you’re imagining right now: underlit and occupied by a parade of shuffling neckbeards, between whom young-me would squeeze to see if the mix-and-match bin had any good condition HO-gauge flexitrack. We converted the attic and it became “The Den”, a secondary space principally for my use. This was, in the most part, a concession for my vacating of a large bedroom and instead switching to the smallest-imaginable bedroom in the house (barely big enough to hold a single bed!), which in turn enabled my baby sister to have a bedroom of her own.

8 My copy of In Yer Face was possibly recorded from the radio by my friend ScGary, who always had a tape deck set up with his finger primed close to the record key when the singles chart came on.

9 I soon learned to recognise “my” copy of tracks by their particular cut-in and -out points, static and noise – some of which, amazingly, survived into the MP3 era – and of course the tracks that came before or after them, and there are still pieces of music where, when I hear them, I “expect” them to be followed by something that they used to some mixtape I listened to a lot 30+ years ago!

10 How amazing a user interface affordance was it that playing one side of an audio cassette was mechanically-equivalent to (slowly) rewinding the other side? Contrast other tape formats, like VHS, which were one-sided and so while rewinding there was literally nothing else your player could be doing. A “full” audio cassette was a marvellous thing, and I especially loved the serendipity where a recognisable “gap” on one side of the tape might approximately line-up with one on the other side, meaning that you could, say, flip the tape after the opening intro to one song and know that you’d be pretty-much at the start of a different one, on the other side. Does any other medium have anything quite analogous to that?

11 Which is pretty rare, unless I choose to put it on… although I did overhear it “organically” last summer: it was coming out of a Bluetooth speaker in a narrowboat moored in the Oxford Canal near Cropredy, where I was using the towpath to return from a long walk to nearby Northamptonshire where I’d been searching for a geocache. This was a particularly surprising place to overhear such a song, given that many of the boats moored here probably belonged to attendees of Fairport’s Cropredy Convention, at which – being a folk music festival – one might not expect to see significant overlap of musical taste with “Madchester”-era acid house music!

12 My first online experiences were on BBS systems, of which my very first was on a mid-80s PC1512 using a 2800-baud acoustic coupler! I got onto the Internet at a point in the early 90s at which the Web existed… but hadn’t yet demonstrated that it would eventually come to usurp the services that existed before it: so I got to use Usenet, Gopher, Telnet and IRC before I saw my first Web browser (it was Cello, but I switched to Netscape Navigator soon after it was released).

13 On the rare occasion I close my browser, these days, it re-opens with whatever hundred or so tabs I was last using right back where I left them. Gosh, I’m a slob for tabs.

14 Or, if it’s a Twitter icon: all three of these.

15 Of course, they’re harder to find. SEO-manipulating behemoths dominate the search results while social networks push their “apps” and walled gardens to try to keep us off the bigger, wider Web… and the more you cut both our of your online life, the calmer and happier you’ll be.

16 The sites featured in the video are: praze, elle’s homepage, sctech, Konfetti Explorations (Marisabel), Frills (check her character sheet “about” page!), mrkod, Raven Winters, Cobb, ajazz, Yusuf Ertan, Alvin Bryan, Armando Cordova, Ens/DepartedGlories, and Jamie Tanna.

× × ×

DOCTYPE

This weekend, I received my copy of DOCTYPE, and man: it feels like a step back to yesteryear to type in a computer program from a magazine: I can’t have done that in at least thirty years.

Dan sits at a cluttered desk reading a copy of DOCTYPE, a magazine with an aggressively 'cyberspace circa 1990' graphic design cover.
I mentioned that I’ve been on a bit of a nostalgic Web Revivalist kick lately, right?

So yeah, DOCTYPE is a dead-tree (only) medium magazine containing the source code to 10 Web pages which, when typed-in to your computer, each provide you with some kind of fun and interactive plaything. Each of the programs is contributed by a different author, including several I follow and one or two whom I’m corresponded with at some point or another, and each brings their own personality and imagination to their contribution.

I opted to start with Stuart Langridge‘s The Nine Pyramids, a puzzle game about trying to connect all nodes in a 3×3 grid in a continuous line bridging adjacent (orthogonal or diagonal) nodes without visiting the same node twice nor moving in the same direction twice in a row (that last provision is described as “not visiting three in a straight line”, but I think my interpretation would have resulted in simpler code: I might demonstrate this, down the line!).

Open magazine showing program code in front of a screen showing a text editor and the running program.
The puzzle actually made me stop to think about it for a bit, which was unexpected and pleasing!

Per tradition with this kind of programming, I made a couple of typos, the worst of which was missing an entire parameter in a CSS conic-gradient() which resulted in the majority of the user interface being invisible: whoops! I found myself reminded of typing-in the code for Werewolves and Wanderer from The Amazing Amstrad Omnibus, whose data section – the part most-liable to be affected by a typographic bug without introducing a syntax error – had a helpful “checksum” to identify if a problem had occurred, and wishing that such a thing had been possible here!

But thankfully a tiny bit of poking in my browser’s inspector revealed the troublesome CSS and I was able to complete the code, and then the puzzle.

I’ve really been enjoying DOCTYPE, and you can still buy a copy if you’d like one of your own. It manages to simultaneously feel both fresh and nostalgic, and that’s really cool.

Are you closer to Ireland or Scotland?

I nerdsniped myself today when, during a discussion on the potential location of a taekwondo tournament organised by our local martial arts school, somebody claimed that Scotland would be “nearer” than Ireland.

WhatsApp conversation. Somebody says "Scotland preferred just because nearer/cheaper!", to which Dan replies, with a map showing rulers connecting Witney to the nearest relevant borders, "Certainly cheaper, but (/puts pedantry hat on/) Witney's actually about 40km nearer to Ireland than Scotland. (/takes pedantry hat off and apologises/) (Sorry, sorry, sorry... geography nerd... sorry.)"
I don’t dispute that somebody living near me can get to Scotland faster than Ireland, unless they can drive at motorway speeds across Wales… and the Irish Sea. But the word they used was nearer, and I can be a pedantic arse.

But the question got me thinking:

Could I plot a line across Great Britain, showing which parts are closer to Scotland and which parts are closer to Ireland?

If the England-facing Irish and Scottish borders were completely straight, one could simply extend the borders until they meet, bisect the angle, and we’d be done.

Extremely simplified hand-drawn map of the British Isles, with the (artificially-straight) borders of Scotland and Ireland extended to meet and then their angle bisected to produce a line that splits England by proximity to each country.
Of course, the borders aren’t straight. They also don’t look much like this. I should not draw maps.

In reality, the border between England and Scotland is a winding mess, shaped by 700 years of wars and treaties1. Treating the borders as straight lines is hopelessly naive.

What I’m really looking for… is a Voronoi partition

Animation showing colours gradually expanding from points on a plane, carving out borders along their edges.
Voronoi diagrams are pretty, and cool, and occasionally even useful! This one expands from points, but there’s no reason you can’t expand from a line (line a border!) instead.

My Python skills are pretty shit, but it’s the best tool for the job for geohacking2. And so, through a combination of hacking, tweaking, and crying, I was able to throw together a script that produces a wonderful slightly-wiggly line up the country.

A line cuts from North-West to South-East through England and Wales.
The entire island of Ireland is used here to determine boundaries (you can tell because otherwise parts of County Antrim, in Northern Ireland, would be marked as closer to Scotland than the Republic of Ireland: which they are, of course, but the question was about England!).

Once you’ve bisected England in this way – into parts that are “closer to Ireland” versus parts that are “closer to Scotland”, you start to spot all kinds of interesting things3.

Like: did you know that the entire subterranean part of the Channel Tunnel is closer to Scotland than it is to Ireland… except for the ~2km closest to the UK exit.

Map showing the Le Shuttle terminal and the entrance to the Channel Tunnel, marked up to show how the first 2km of the underground part (only) are closer to Ireland than to Scotland.

A little further North: London’s six international airports are split evenly across the line, with Luton, Stansted and Southend closer to Scotland… and City, Heathrow and Gatwick closer to Ireland.

Map showing the locations of London's six major airports: three North-East of the line, three South-West of it.

The line then pretty-much bisects Milton Keynes, leaving half its population closer to Scotland and half closer to Ireland, before doing the same to Daventry, before – near Sutton Coldfield – it drives right through the middle of the ninth hole of the golf course at the Lea Marston Hotel.

Players tee off closer to Ireland and – unless they really slice it – their ball lands closer to Scotland:

Aerial photograph of a golf course with a red line superimposed across it. Near the green, a speech bubble has a golfer saying "Next shot's going to Scotland!"

In Cannock, it bisects the cemetery, dividing the graves into those on the Scottish half and those in the Irish half:

Aerial photography of Cannock Cemetery, bisected with a thick red line.

The line crosses the Welsh border at the River Dee, East of Wrexham, leaving a narrow sliver of Wales that’s technically closer to Scotland than it is to Ireland, running up the coastline from Connah’s Quay to Prestatyn and going as far inland as Mold before – as is the case in most of Wales – you’re once again closer to Ireland:

Highlighted fragment of Wales along the West bank of the River Dee and stretching about 10-12km inland.
If you live in Flint or Mold, ask your local friends whether they live closer to Ireland or Scotland. The answer’s Scotland, and I’m confident that’ll surprise them.

I’d never have guessed that there were any parts of Wales that were closer to Scotland than they were to Ireland, but the map doesn’t lie4

Anyway: that’s how I got distracted, today. And along the way I learned a lot about geodata encoding, a little about Python, and a couple of surprising things about geography5.

Footnotes

1 Not to mention the crazy history of places like Berwick-upon-Tweed, which has jumped the border several times, and Ba Green, ownership of which has traditionally been decided by game of football.

2 Or, at least: it’s the one that’s most-widely used and so I could find lots of helpful StackOverflow answers when I got stuck!

3 Interesting… if you’re specifically looking for some geographical trivia, that is!

4 Okay, the map lies a little. My program was only simple so it plotted everything on a flat plane, failing to accommodate for Earth’s curvature. The difference is probably marginal, but if you happen to live on or very close to the red line, you might need to do your own research!

5 Like: Chester and Rugby are closer to Scotland than they are to Ireland, and Harpenden and Towcester are closer to Ireland than they are to Scotland! Who knew?

× × × × × × × × ×

The Scroll Art Museum

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Scroll art is a form of ASCII art where a program generates text output in a command line terminal. After the terminal window fills, it begins to scroll the text upwards and create an animated effect. These programs are simple, beautiful, and accessible as programming projects for beginners. The SAM is a online collection of several scroll art examples.

Here are some select pieces:

  • Zig-zag, a simple periodic pattern in a dozen lines of code.
  • Orbital Travels, sine waves intertwining.
  • Toggler, a woven triangular pattern restricted to two characters.
  • Proton Stream, a rapid, chaotic lightning pattern.

There are two limitations to most scroll art:

  • Program output is limited to text (though this could include emoji and color.)
  • Once printed, text cannot be erased. It can only scroll up.

But these restrictions compel creativity. The benefit of scroll art is that beginner programmers can create scroll art apps with a minimal amount of experience. Scroll art requires knowing only the programming concepts of print, looping, and random numbers. Every programming langauge has these features, so scroll art can be created in any programming language without additional steps. You don’t have to learn heavy abstract coding concepts or configure elaborate software libraries.

Okay, so: scroll art is ASCII art, except the magic comes from the fact that it’s very long and as your screen scrolls to show it, an animation effect becomes apparent. Does that make sense?

Here, let me hack up a basic example in… well, QBASIC, why not:

Anyway, The Scroll Art Museum has lots of them, and they’re much better than mine. I especially love the faux-parallax effect in Skulls and Hearts, created by a “background” repeating pattern being scrolled by a number of lines slightly off from its repeat frequency while a foreground pattern with a different repeat frequency flies by. Give it a look!

Lowriders & websites

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

I think of ElonStan420 standing in that exhibit hall, eyeing those cars with disdain because all that time, energy, care, and expression “doesn’t really matter”. Those hand-painted pinstripes don’t make the car faster or cheaper. Chrome-plated everything doesn’t make it more efficient. No one is going to look under the hood anyway.

Don’t read the comments on HackerNews, Adam! (I say this, but I’ve yet to learn not to do so myself, when occasionally my writing escapes from my site and finds its way over there.)

But anyway, this is a fantastic piece about functionalism. Does it matter whether your website has redundant classes defined in the HTML? It renders the same anyway, and odds are good that nobody will ever notice! I’m with Adam: yes, of course it can matter. It doesn’t have to, but coding is both a science and an art, and art matters.

Should every website be the subject of maximal craft? No, of course not. But in a industry rife with KPI-obsessed, cookie-cutter, vibe-coded, careless slop, we could use more lowriders.

Well said, Adam.

Four perspectives on AI

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

A two-axis graph labeled 'Beliefs about AI' with the x-axis of 'Transformative' and the y-axis of 'Positive'. The field is divided into four quadrants: 'Skeptical' in the bottom-left, 'Wary' in the bottom-right, 'Pragmatist' in the top-left, and 'Optimist' in the top-right.

I’ve grouped these four perspectives, but everything here is a spectrum. Depending on the context or day, you might find yourself at any point on the graph. And I’ve attempted to describe each perspectively [sic] generously, because I don’t believe that any are inherently good or bad. I find myself switching between perspectives throughout the day as I implement features, use tools, and read articles. A good team is probably made of members from all perspectives.

Which perspective resonates with you today? Do you also find yourself moving around the graph?

An interesting question from Sean McPherson. He sounds like he’s focussed on LLMs for software development, for which I’ve drifted around a little within the left-hand-side of the graph. But perhaps right now, this morning, you could simplify my feelings like this:

The same graph with a hand-drawn arrow moving from the Skeptical to the Pragmatic quadrants.My stance is that AI-assisted coding can be helpful (though the question remains open about whether it’s “worth it”), so long as you’re not trying to do anything that you couldn’t do yourself, and you know how you’d go about doing it yourself. That is: it’s only useful to accelerate tasks that are in your “known knowns” space.

As I’ve mentioned: the other week I had a coding AI help me with some code that interacted with the Google Sheets API. I know exactly how I’d go about it, but that journey would have to start with re-learning the Google Sheets API, getting an API key and giving it the appropriate permissions, and so on. That’s the kind of task that I’d be happy to outsource to a less-experienced programmer who I knew would bring a somewhat critical eye for browsing StackOverflow, and then give them some pointers on what came back, so it’s a fine candidate for an AI to step in and give it a go. Plus: I’d be treating the output as “legacy code” from the get-go, and (because the resulting tool was only for my personal use) I wasn’t too concerned with the kinds of security and accessibility considerations that GenAI can often make a pig’s ear of. So I was able to palm off the task onto Claude Sonnet and get on with something else in the meantime.

If I wanted to do something completely outside of my wheelhouse: say – “write a program in Fortran to control a robot arm” – an AI wouldn’t be a great choice. Sure, I could “vibe code” something like that, but I’d have no idea whether what it produced was any good! It wouldn’t even be useful as a springboard to learning how to do that, because I don’t have the underlying fundamentals in robotics nor Fortran. I’d be producing AI slop in software form: the kind of thing that comes out when non-programmers assume that AI can completely bridge the gap between their great business idea and a fully working app!

The latest episode of South Park kinda nailed parodying the unrealistic expectations that some folks seem to put on generative AI: treating it as intelligent or as a friend is unhealthy and dangerous!

They’ll get a prototype that seems to do what you want, if you squint just right, but the hard part of software engineering isn’t making a barebones proof-of-concept! That’s the easy bit! (That’s why AI can do it pretty well!) The hard bit is making it work all the time, every time; making it scale; making it safe to use; making it maintainable; making it production-ready… etc.

But I do benefit from coding AI sometimes. GenAI’s good at summarisation, which in turn can make it good at relatively-quickly finding things in a sprawling codebase where your explanation of those things is too-woolly to use a conventional regular expression search. It’s good at generating boilerplate that’s broadly-like examples its seen before, which means it can usually be trusted to put together skeleton applications. It’s good at “guessing what comes next” – being, as it is, “fancy autocomplete” – which means it can be helpful for prompting you for the right parameters for that rarely-used function or for speculating what you might be about to do with the well-named variable you just created.

Anyway: Sean’s article was pretty good, and it’s a quick and easy read. Once you’ve read it, perhaps you’ll share where you think you sit, on his diagram?

× ×

Just a Little More Context Bro, I Promise, and It’ll Fix Everything

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Solving problems with LLMs is like solving front-end problems with NPM: the “solution” comes through installing more and more things — adding more and more context, i.e. more and more packages.

  • LLM: Problem? Add more context.
  • NPM: Problem? There’s a package for that.

As I’m typing this, I’m thinking of that image of the evolution of the Raptor engine, where it evolved in simplicity:

Photograph of three versions of the raptor engine, each one getting progressively simplified in mechanical parts.

This stands in contrast to my working with LLMs, which often wants more and more context from me to get to a generative solution:

Photograph of three versions of the raptor engine, but the image is reversed showing the engine get progressively complicated in mechanical parts over time. Each engine represents an LLM prompt.

Jim Nielsen speaks to my experience, here. Because a programming LLM is simply taking inputs (all of your code, plus your prompt), transforming it through statistical analysis, and then producing an output (replacement code), it struggles with refactoring for simplicity unless very-carefully controlled. “Vibe coding” is very much an exercise in adding hacks upon hacks… like the increasingly-ludicrous epicycles introduced by proponents of geocentrism in its final centuries before the heliocentric model became fully accepted.

Geocentric representation of the apparent motion of the Sun, Mercury, and Venus from the Earth, based on 15th century diagrams. It consists of many looping spirals approaching and then withdrawing from the Earth as they orbit around it.
This mess used to be how many perfectly smart people imagined the movements of the planets. When observations proved it couldn’t be right, they’d just add more complexity to catch the edge cases.

I don’t think that AIs are useless as a coding tool, and I’ve successfully used them to good effect on several occasions. I’ve even tried “vibe coding”, about which I fully agree with Steve Krouse‘s observation that “vibe code is legacy code”. Being able to knock out something temporary, throwaway, experimental, or for personal use only… while I work on something else… is pretty liberating.

For example: I couldn’t remember my Google Sheets API and didn’t want to re-learn it from the sprawling documentation site, but wanted a quick personal tool to manipulate such a sheet from a remote system. I was able to have an AI knock up what I needed while I cooked dinner for the kids, paying only enough attention to check-in on its work. Is it accessible? Is it secure? Is it performant? Is it maintainable? I can’t answer any of those questions, and so as a professional software engineer I have to reasonably assume the answer to all of them is “no”. But its only user is me, it does what I needed it to do, and I didn’t have to shift my focus from supervising children and a pan in order to throw it together!

Anyway: Jim hits the nail on the head here, as he so often does.

× × ×

Note #26999

In my first few weeks at my new employer, my code contributions have added 218 lines of code, deleted 2,663. Only one of my PRs has resulted in a net increase in the size of their codebases (by two lines).

GitHub change summary showing +218 lines added, -2,663 lines removed.

I need to pick up the pace if I’m going to reach the ultimate goal of deleting ALL of the code within my lifetime. (That’s the ultimate aim, right?)

×

ArtificialCast

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Type-safe transformation powered by inference.

ArtificialCast is a lightweight, type-safe casting and transformation utility powered by large language models. It allows seamless conversion between strongly typed objects using only type metadata, JSON schema inference, and prompt-driven reasoning.

Imagine a world where Convert.ChangeType() could transform entire object graphs, infer missing values, and adapt between unrelated types – without manual mapping or boilerplate.

ArtificialCast makes that possible.

Features

  • Zero config – Just define your types.
  • Bidirectional casting – Cast any type to any other.
  • Schema-aware inference – Auto-generates JSON Schema for the target type.
  • LLM-powered transformation – Uses AI to “fill in the blanks” between input and output.
  • Testable & deterministic-ish – Works beautifully until it doesn’t.

As beautiful as it is disgusting, this C# is fully-functional and works exactly as described… and yet you really, really should never use it (which its author will tell you, too).

Casting is the process of transforming a variable of one type into one of another. So for example you might cast the number 3 into a string and get "3" (though of course this isn’t the only possible result: "00000011" might also be a valid representation, depending on the circumstances1).

Casting between complex types defined by developers is harder and requires some work. Suppose you have a User model with attributes like “username”, “full name”, “hashed password”, “email address” etc., and you want to convert your users into instances of a new model called Customer. Some of the attributes will be the same, some will be absent, and some will be… different (e.g. perhaps a Customer has a “first name” and “last name” instead of a “full name”, and it’s probably implemented wrong to boot).

The correct approach is to implement a way to cast one as the other.

The very-definitely incorrect approach is to have an LLM convert the data for you. And that’s what this library provides.

ArtificialCast is a demonstration of what happens when overhyped AI ideas are implemented exactly as proposed – with no shortcuts, no mocking, and no jokes.

It is fully functional. It passes tests. It integrates into modern .NET workflows. And it is fundamentally unsafe.

This project exists because:

  • AI-generated “logic” is rapidly being treated as production-ready.
  • Investors are funding AI frameworks that operate entirely on structure and prompts.
  • Developers deserve to see what happens when you follow that philosophy to its logical conclusion.

ArtificialCast is the result.

It works. Until it doesn’t. And when it doesn’t, it fails in ways that look like success. That’s the danger.

I’ve played with AI in code a few times. There are some tasks it’s very good at, like summarising and explaining (when the developer before you didn’t leave a sufficiency of quality comments). There are some tasks it can be okay at, with appropriate framing and support: like knowing its way around unfamiliar-to-you but well-documented APIs2.

But if you ask an AI to implement an entire product or even just a significant feature from scratch, unsupervised, you’re at risk of rapidly hitting the realm of Heisenbugs, security vulnerabilities, and enormous redundancies.

This facetious example – of using AI as a universal typecasting engine – helps hammer that point home, and I love it.

Footnotes

1 How to cast basic types isn’t entirely standardised: PHP infamously casts the string "0" as false when it’s coerced into a boolean, which virtually no other programming language does, for example.

2 The other week, I had a GenAI help me write some code that writes to a Google Sheets document, because I was fuzzy on the API and knew the AI would pick it up faster than me while I wrote the code “around” it.

Adding a feature because ChatGPT incorrectly thinks it exists

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Our scanning system wasn’t intended to support this style of notation. Why, then, were we being bombarded with so many ASCII tab ChatGPT screenshots? I was mystified for weeks — until I messed around with ChatGPT myself and got this:

Screenshot of ChatGPT telling users to enter this ASCII tab into soundslice.com

Turns out ChatGPT is telling people to go to Soundslice, create an account and import ASCII tab in order to hear the audio playback. So that explains it!

With ChatGPT’s inclination to lie about the features of a piece of technology, it was only a matter of time before a frustrated developer actually added a feature that ChatGPT had imagined, just to stop users from becoming dissatisfied when they tried to use nonexistent tools that ChatGPT told them existed.

And this might be it! This could be the very first time that somebody’s added functionality based on an LLM telling people the feature existed already.

Adrian Holovaty runs a tool that can “read” scanned sheet music and provide a digital representation to help you learn how to play it. But after ChatGPT started telling people that his tool could also read ASCII-formatted guitar tablature, he went and implemented it!

His blog post’s got more details, and it’s worth a read. This could be a historic moment that we’ll look back on!

×

Smug Interview Moment

I’ve been in a lot of interviews over the last two or three weeks. But there’s a moment that stands out and that I’ll remember forever as the most-smug I’ve ever felt during an interview.

Close-up of part of a letter, the visible part of which reads: Dear Dan, We are pleased to offer you a position as Senior Softwa... / and reporting to the Company's Manager, Software E... / (the "Commencement Date"). You will receive an... / By accepting this offer you warrant and agree...
There’ll soon be news to share about what I’m going to be doing with the second half of this year…

This particular interview included a mixture of technical and non-technical questions, but a particular technical question stood out for reasons that will rapidly become apparent. It went kind-of like this:

Interviewer: How would you go about designing a backend cache that retains in memory some number of most-recently-accessed items?

Dan: It sounds like you’re talking about an LRU cache. Coincidentally, I implemented exactly that just the other week, for fun, in two of this role’s preferred programming languages (and four other languages). I wrote a blog post about my design choices: specifically, why I opted for a hashmap for quick reads and a doubly-linked-list for constant-time writes. I’m sending you the links to it now: may I talk you through the diagrams?

Interviewer:

'Excuse me' GIF reaction. A white man blinks and looks surprised.

That’s probably the most-overconfident thing I’ve said at an interview since before I started at the Bodleian, 13 years ago. In the interview for that position I spent some time explaining that for the role they were recruiting for they were asking the wrong questions! I provided some better questions that I felt they should ask to maximise their chance of getting the best candidate… and then answered them, effectively helping to write my own interview.

Anyway: even ignoring my cockiness, my interview the other week was informative and enjoyable throughout, and I’m pleased that I’ll soon be working alongside some of the people that I met: they seem smart, and driven, and focussed, and it looks like the kind of environment in which I could do well.

But more on that later.

× ×

I Wrote the Same Code Six Times!

I was updating my CV earlier this week in anticipation of applying for a handful of interesting-looking roles1 and I was considering quite how many different tech stacks I claim significant experience in, nowadays.

There are languages I’ve been writing in every single week for the last 15+ years, of course, like PHP, Ruby, and JavaScript. And my underlying fundamentals are solid.

But is it really fair for me to be able to claim that I can code in Java, Go, or Python: languages that I’ve not used commercially within the last 5-10 years?

Animated GIF showing Sublime Text editor flicking through six different languages, with equivalent code showing in each.
What kind of developer writes the same program six times… for a tech test they haven’t even been asked to do? If you guessed “Dan”, you’d be correct!

Obviously, I couldn’t just let that question lie2. Let’s find out!

Contents

The Test

I fished around on Glassdoor for a bit to find a medium-sized single-sitting tech test, and found a couple of different briefs that I mashed together to create this:

In an object-oriented manner, implement an LRU (Least-Recently Used) cache:

  • The size of the cache is specified at instantiation.
  • Arbitrary objects can be put into the cache, along with a retrieval key in the form of a string. Using the same string, you can get the objects back.
  • If a put operation would increase the number of objects in the cache beyond the size limit, the cached object that was least-recently accessed (by either a put or get operation) is removed to make room for it.
  • putting a duplicate key into the cache should update the associated object (and make this item most-recently accessed).
  • Both the get and put operations should resolve within constant (O(1)) time.
  • Add automated tests to support the functionality.

My plan was to implement a solution to this challenge, in as many of the languages mentioned on my CV as possible in a single sitting.

But first, a little Data Structures & Algorithms theory:

The Theory

Simple case with O(n) complexity

The simplest way to implement such a cache might be as follows:

  • Use a linear data structure like an array or linked list to store cached items.
  • On get, iterate through the list to try to find the matching item.
    • If found: move it to the head of the list, then return it.
  • On put, first check if it already exists in the list as with get:
    • If it already exists, update it and move it to the head of the list.
    • Otherwise, insert it as a new item at the head of the list.
      • If this would increase the size of the list beyond the permitted limit, pop and discard the item at the tail of the list.
Illustration of a simple Cache class with a linked list data structure containing three items, referenced from the class by its head item. It has 'get' and 'put' methods.
It’s simple, elegant and totally the kind of thing I’d accept if I were recruiting for a junior or graduate developer. But we can do better.

The problem with this approach is that it fails the requirement that the methods “should resolve within constant (O(1)) time”3.

Of particular concern is the fact that any operation which might need to re-sort the list to put the just-accessed item at the top 4. Let’s try another design:

Achieving O(1) time complexity

Here’s another way to implement the cache:

  • Retain cache items in a doubly-linked list, with a pointer to both the head and tail
  • Add a hash map (or similar language-specific structure) for fast lookups by cache key
  • On get, check the hash map to see if the item exists.
    • If so, return it and promote it to the head (as described below).
  • On put, check the hash map to see if the item exists.
    • If so, promote it to the head (as described below).
    • If not, insert it at the head by:
      • Updating the prev of the current head item and then pointing the head to the new item (which will have the old head item as its next), and
      • Adding it to the hash map.
      • If the number of items in the hash map would exceed the limit, remove the tail item from the hash map, point the tail at the tail item’s prev, and unlink the expired tail item from the new tail item’s next.
  • To promote an item to the head of the list:
    1. Follow the item’s prev and next to find its siblings and link them to one another (removes the item from the list).
    2. Point the promoted item’s next to the current head, and the current head‘s prev to the promoted item.
    3. Point the head of the list at the promoted item.
Illustration of a more-sophisticated Cache class with a doubly-linked list data structure containing three items, referenced from the class by its head and tail item. It has 'get' and 'put' methods. It also has a separate hash map indexing each cached item by its key.
Looking at a plate of pointer-spaghetti makes me strangely hungry.

It’s important to realise that this alternative implementation isn’t better. It’s just different: the “right” solution depends on the use-case5.

The Implementation

That’s enough analysis and design. Time to write some code.

GitHub repo 'Languages' panel, showing a project that includes Java (17.2%), Go (16.8%), Python (16.4%), Ruby (15.7%), TypeScript (13.7%), PHP (11.5%), and 'Other' (8.7%).
Turns out that if you use enough different languages in your project, GitHub begins to look like itwants to draw a rainbow.

Picking a handful of the more-useful languages on my CV6, I opted to implement in:

  • Ruby (with RSpec for testing and Rubocop for linting)
  • PHP (with PHPUnit for testing)
  • TypeScript (running on Node, with Jest for testing)
  • Java (with JUnit for testing)
  • Go (which isn’t really an object-oriented language but acts a bit like one, amirite?)
  • Python (probably my weakest language in this set, but which actually ended up with quite a tidy solution)

Naturally, I open-sourced everything if you’d like to see for yourself. It all works, although if you’re actually in need of such a cache for your project you’ll probably find an alternative that’s at least as good (and more-likely to be maintained!) in a third-party library somewhere!

What did I learn?

This was actually pretty fun! I might continue to expand my repo by doing the same challenge with a few of the other languages I’ve used professionally at some point or another7.

And there’s a few takeaways I got from this experience –

Lesson #1: programming more languages can make you better at all of them

As I went along, one language at a time, I ended up realising improvements that I could make to earlier iterations.

For example, when I came to the TypeScript implementation, I decided to use generics so that the developer can specify what kind of objects they want to store in the cache, rather than just a generic Object, and better benefit type-safety. That’s when I remembered that Java supports generics, too, so I went back and used them there as well.

In the same way as speaking multiple (human) languages or studying linguistics can help unlock new ways of thinking about your communication, being able to think in terms of multiple different programming languages helps you spot new opportunities. When in 2020 PHP 8 added nullsafe operators, union types, and named arguments, I remember feeling confident using them from day one because those features were already familiar to me from Ruby8, TypeScript9, and Python10, respectively.

Lesson #2: even when I’m rusty, I can rely on my fundamentals

I’ve applied for a handful of jobs now, but if one of them had invited me to a pairing session on a language I’m rusty on (like Java!) I might’ve felt intimidated.

But it turns out I shouldn’t need to be! With my solid fundamentals and a handful of other languages under my belt, I understand when I need to step away from the code editor and hit the API documentation. Turns out, I’m in a good position to demo any of my language skills.

I remember when I was first learning Go, I wanted to make use of a particular language feature that I didn’t know whether it had. But because I’d used that feature in Ruby, I knew what to search for in Go’s documentation to see if it was supported (it wasn’t) and if so, what the syntax was11.

Lesson #3: structural rules are harder to gearshift than syntactic ones

Switching between six different languages while writing the same application was occasionally challenging, but not in the ways I expected.

I’ve had plenty of experience switching programming languages mid-train-of-thought before. Sometimes you just have to flit between the frontend and backend of your application!

But this time around I discovered: changes in structure are apparently harder for my brain than changes in syntax. E.g.:

  • Switching in and out of Python’s indentation caught me out at least once (might’ve been better if I took the time to install the language’s tools into my text editor first!).
  • Switching from a language without enforced semicolon line ends (e.g. Ruby, Go) to one with them (e.g. Java, PHP) had me make the compiler sad several times.
  • This gets even tougher when not writing the language but writing about the language: my first pass at the documentation for the Go version somehow ended up with Ruby/Python-style #-comments instead of Go/Java/TypeScript-style //-comments; whoops!

I’m guessing that the part of my memory that looks after a language’s keywords, how a method header is structured, and which equals sign to use for assignment versus comparison… are stored in a different part of my brain than the bit that keeps track of how a language is laid-out?12

Okay, time for a new job

I reckon it’s time I got back into work, so I’m going to have a look around and see if there’s any roles out there that look exciting to me.

If you know anybody who’s looking for a UK-based, remote-first, senior+, full-stack web developer with 25+ years experience and more languages than you can shake a stick at… point them at my CV, would you?

Footnotes

1 I suspect that when most software engineers look for a new job, they filter to the languages, frameworks, they feel they’re strongest at. I do a little of that, I suppose, but I’m far more-motivated by culture, sector, product and environment than I am by the shape of your stack, and I’m versatile enough that technology specifics can almost come second. So long as you’re not asking me to write VB.NET.

2 It’s sort-of a parallel to how I decided to check the other week that my Gutenberg experience was sufficiently strong that I could write standard ReactJS, too.

3 I was pleased to find a tech test that actually called for an understanding of algorithm growth/scaling rates, so I could steal this requirement for my own experiment! I fear that sometimes, in their drive to be pragmatic and representative of “real work”, the value of a comprehension of computer science fundamentals is overlooked by recruiters.

4 Even if an algorithm takes the approach of creating a new list with the inserted/modified item at the top, that’s still just a very-specific case of insertion sort when you think about it, right?

5 The second design will be slower at writing but faster at reading, and will scale better as the cache gets larger. That sounds great for a read-often/write-rarely cache, but your situation may differ.

6 Okay, my language selection was pretty arbitrary. But if I’d have also come up with implementations in Perl, and C#, and Elixir, and whatever else… I’d have been writing code all day!

7 So long as I’m willing to be flexible about the “object-oriented” requirement, there are even more options available to me. Probably the language that I last wrote longest ago would be Pascal: I wonder how much of that I remember?

8 Ruby’s safe navigation/”lonely” operator did the same thing as PHP’s nullsafe operator since 2015.

9 TypeScript got union types back in 2015, and apart from them being more-strictly-enforced they’re basically identical to PHP’s.

10 Did you know that Python had keyword arguments since its very first public release way back in 1994! How did it take so many other interpreted languages so long to catch up?

11 The feature was the three-way comparison or “spaceship operator”, in case you were wondering.

12 I wonder if anybody’s ever laid a programmer in an MRI machine while they code? I’d be really interested to see if different bits of the brain light up when coding in functional programming languages than in procedural ones, for example!

× × × ×

Dynamic Filters in Pure CSS

While working on something else entirely1, I had a random thought:

Could the :checked and and :has pseudo-classes and the subsequent-sibling (~) selector be combined to perform interactive filtering without JavaScript?

Turns out, yes. Have a play with the filters on the side of this. You can either use:

  • “OR” mode, so you can show e.g. “all mammals and carnivores”, or
  • “AND” mode, so you can show e.g. “all mammals that are carnivores”.

Filter the animals!

(if it doesn’t work right where you are, e.g. in a feed reader, you can view it “standalone”)

  • Alpaca
  • Anteater
  • Bat
  • Beetle
  • Butterfly
  • Camel
  • Cat
  • Chameleon
  • Cobra
  • Cow
  • Crab
  • Crocodile
  • Dog
  • Duck
  • Elephant
  • Elk
  • Fish
  • Frog
  • Giraffe
  • Hippo
  • Husky
  • Kangaroo
  • Lion
  • Macaw
  • Manatee
  • Monkey
  • Mouse
  • Octopus
  • Ostrich
  • Owl
  • Panda
  • Pelican
  • Penguin
  • Pig
  • Rabbit
  • Raccoon
  • Ray
  • Rhino
  • Rooster
  • Shark
  • Sheep
  • Sloth
  • Snake
  • Spider
  • Squirrel
  • Swan
  • Tiger
  • Toucan
  • Turtle
  • Whale

The source code is available to download under the Unlicense, but the animal images are CC-BY licensed (with thanks to Aslan Almukhambetov).

How does it work?

There’s nothing particularly complicated here, although a few of the selectors are a little verbose.

First, we set the initial state of each animal. In “OR” mode, they’re hidden, because each selected checkbox is additive. In “AND” mode, they’re shown, because checking a checkbox can only ever remove an animal from the result set:

#filters:has(#filter-or:checked) ~ #animals .animal {
  display: none;
}

#filters:has(#filter-and:checked) ~ #animals .animal {
  display: flex;
}

The magic of the :has pseudo-class is that it doesn’t change the scope, which means that after checking whether “AND” or “OR” is checked within the #filters, the #animals container is still an adjacent element.

Touchscreen interactive restaurant menu with filter categories on the left and dishes for selection in the centre.
Next time you’re implementing a filter interface, like this restaurant menu, perhaps ask whether you actually need JavaScript.

Then all we need to do is to use daisy-chain :has to show animals with a particular class if that class is checked in “OR” mode, or to hide animals that don’t have a particular class in “AND” mode. Here’s what that looks like:

#filters:has(#filter-or:checked):has(#aquatic:checked)  ~ #animals .aquatic,
#filters:has(#filter-or:checked):has(#bird:checked)     ~ #animals .bird,
...
#filters:has(#filter-or:checked):has(#reptile:checked)  ~ #animals .reptile {
  display: flex;
}

#filters:has(#filter-and:checked):has(#aquatic:checked) ~ #animals .animal:not(.aquatic),
#filters:has(#filter-and:checked):has(#bird:checked)    ~ #animals .animal:not(.bird),
...
#filters:has(#filter-and:checked):has(#reptile:checked) ~ #animals .animal:not(.reptile) {
  display: none;
}

It could probably enjoy an animation effect to make it clearer when items are added and removed2, but that’s a consideration for another day.

Many developers would be tempted to use JavaScript to implement the client-side version of a filter like this. And in some cases, that might be the right option.

But it’s always worth remembering that:

  • A CSS solution is almost-always more-performant than a JS one.
  • A JS solution is usually less-resilient than a CSS one: a CDN failure, unsupported API, troublesome content-blocker or syntax error will typically have a much larger impact on JavaScript.
  • For the absolutely maximum compatibility, consider what you can do in plain HTML, or on the server-side, and treat anything on the client-side as progressive enhancement.

Footnotes

1 The thing I was actually working on when I got distracted was an OAuth provider implementation for Three Rings, connected with work that took place at this weekend’s hackathon to (eventually) bring single-sign-on “across” Three Rings CIC’s products. Eventually being the operative word.

2 Such an animation should, of course, be wrapped in a @media (prefers-reduced-motion: no-preference) media query!

×