Articles – Dan Q

Wikipedia @ 25: Yo-Yo

Duration 8:31

Podcast Version

This post is also available as a podcast. Listen here, download for later, or subscribe wherever you consume podcasts.

To celebrate the site’s 25th birthday this year, Wikipedia is encouraging/challenging people to read one Wikipedia article a day for 25 consecutive days. I felt that I could do one better than that: not only reading an article but – where I found one that was particularly interesting – to write a blog post or record a podcast episode for each of those days, sharing what I learned. For each entry, I’ll hit “random article” a few times until something catches my interest, start reading, and then start writing! Everything I’ve written below came from Wikipedia… so you should check other sources before you use it to do your homework. Happy birthday, Wikipedia!

Today’s random article: Marcus Koh
Today’s topic: Yo-yo

One of the things I’ve discovered over my past few days of hitting “Random Article” on Wikipedia is that sometimes you get something that’s worth writing about. But more often you get something worth reading but not writing about. But more often still you get something that doesn’t interest you at all, and you just need to click “Random Article” again.

And that latter category is the one I thought I was in when I discovered Marcus Koh, who’s a Singaporean yo-yo enthusiast who came first in the 1A division at the World Yo-Yo Contest in 2011. The page almost felt like a stub… but then I started clicking and found myself learning much more about yo-yos than I ever thought possible.

Like… I knew that the yo-yo was an old toy, but I had no idea how old.

Drawing showing a woman playing with an early form of the yo-yo. — This 1791 image allegedly from a French fashion journal. The French usually called the toy a *emigrette* at the time, but the 1888 republication of this image in *Le Costume Historique* called it *Joujou de Normandie*, so who knows.

Obviously there’s a lot of pictures from around the end of the eighteenth century, which is when they became popular in Europe. In the English-speaking world at that point they were known as “bandalores”, which I think is a nicer name than “yo-yo”, frankly.

But their influence was clearly felt much further away and much longer ago than this.

I mean, here’s a 1770 watercolor from Northern India that clearly depicts something that, despite being held in two hands, is definitely something-like-a-yo-yo:

But we can go further.

If you lived in Greece in around the 5th century BCE and were serving wine to your guests, the popular drinking vessel to use was a kylix. Kylikes were pottery cups basically the shape of modern wine glasses but much more squat, having a wide bowl atop a pedestal that tapered outwards. Unlike modern wine glasses, though, they had handles, and these handles were used to play a game called kottabos: once you’d finished your wine, you’d use a handle to “flick” the sediment from your wine (I guess fining/clarification agents weren’t a thing yet?) at a target in order to win a cake or something.

Sounds pretty gross for whoever had to clean up afterwards, if you ask me.

Anyway: oftentimes the inner bowl of a kylix would be decorated. Depending on the kind of party you were throwing you might have a nautical theme where everybody finds a different kind of boat at the bottom of their cup when they drain it… or for a more raucous party perhaps you’d get out the cups where the faces at the bottom all had genitals hidden in them. That way, somebody gets surprised to find that at the end of a drinking session they have a penis in their face (I’ve certainly had parties like that before, if you know what I mean):

Interior of the Bomford Cup, a kylix with a face at the bottom whose nose is distinctly shaped like male genitalia. — I guess that these were the Ancient Greek equivalent of shot glasses with swear words etched into them?

What I’m saying is… the Ancient Greeks liked to play drinking games, and they liked drinking vessels with pictures on. Which makes you look at the “Greek culture” of fraternity houses in a whole new light.

But the pictures weren’t always either (a) boats or (b) crude, of course. They could be anything. Here’s an example of the bottom of a kylix that was probably used as a drinking vessel in or near Athens around 2,500 years ago:

Boy playing yo-yo. Tondo of an Attic red-figure kylix, ca. 440 BC. — What the actual fuck? That boy’s *clearly* playing with a yo-yo in a picture painted before the Parthenon was built!

It’s not just novelty earthenware that tells us that the Ancient Greeks had the yo-yo, by the way. We’ve found actual examples of them made from bronze or terracotta, although archaeologists suspect that there were many more wooden variants that have been lost to time.

I guess it’s true that it’s a toy that just keeps making a comeback. Every few centuries it gets reinvented and improved, I guess! “Modern” yo-yos got their relaunch in the 1920s, when Pedro Flores (a Filipino businessman whose time in his birth country spanned a previous story) brought to the USA a toy that had been popular in his homeland but seemed to be mostly-unknown in the States. The name apparently derives from a Tagalog word that means “come-come” or “come-go” or something similar. He produced both traditional “tied-on” yo-yos and “slip-string” varieties that allowed the toy to “sleep” – to spin-freely at the end of its string – which unlocked a diversity of new tricks.

From here on, the yo-yo saw surges in popularity every 20 to 40 years. The full article’s worth a read because unless you’re a complete yo-yo nut I can guarantee there are things in there that you didn’t know.

I was also very interested in the article about the “Eskimo yo-yo”, which I’d love to see somebody operate! It’s basically a bola of two weights attached to a stick using strings of two different lengths, and the trick is to get them spinning in opposite directions but using only one hand. That sounds amazing!

I also got briefly distracted by clackers, a hyperlink-adjacent childrens’ toy that lends its name to the excellent lawsuit title United States v. Article Consisting of 50,000 Cardboard Boxes More or Less, Each Containing One Pair of Clacker Balls, which is going right up there in my list of favourite Wikipedia page titles alongside Salmon chaos, List of lists of lists, Thinking about the immortality of the crab, 2022 United Kingdom government crisis (disambiguation), Pope John numbering, and Pentagon pizza theory.

Built In Obsolescence

This post contains and links to (clearly-identified) AI-generated content. As remains the case, none of my writing on this blog was generated by AI.

Imagine my excitement to learn that Pagan Wander Lu just dropped a new EP, Built In Obsolescence. And then imagine my horror to discover that it’s actually produced by P-AI-gan Wanderer Lu; an AI that’s been given PWL lyrics and some artistic direction.

Wot.

AI-generated EP cover of Built In Obsolescence by PAIgan Wanderer Lu, showing a neon digital outline of Andy. — The album art’s clearly also AI-generated, and that’s… well… you know. At least this robot hand has got the correct number of fingers.

Nothingness is what silicon dreams

My younger child’s been getting into PWL in a big way lately. As a result of this, I ended up making time for a careful re-listen to a lot of the back catalogue. This in turn inspired a blog post last year in which I mentioned that Checker Charlie‘s observations about humans replacing their work with machine effort feels increasingly prophetic in the age of generative AI. That’s something I didn’t see in it when I first reviewed it 13 years prior.

I’ve played with AI-generated music a couple of times myself, of course, mostly as an academic exercise. And it’s becoming more and more apparent that it’s hard to avoid bumping into it in the “real world”.

Early efforts at AI music were pretty unconvincing, always sounding a bit auto-tuney, frequently struggling to stress lines in the right places, and tripping over themselves when they try to do anything even remotely more-interesting than a simple repeating melody atop a predictable chord sequence. But they’re getting… shall we say… “better”, and there have been times nowadays when I’ve gotten some way through a track before realising that I’m listening to AI.

At least PWL’s being honest about it and declaring at the outset that this is AI-generated art. There’s plenty of folks using AI to generate content online and not declaring it, which is pretty awful¹. Anyway: in this EP the AI’s moderately well-concealed and listening casually to most of the tracks I wouldn’t have noticed it if I hadn’t been told².

Is there life enough in these chords?

So I listened to the EP. Three times.

The cover of Checker Charlie, I’m sad to admit, works. It’s got the feel of early-nineties pop, full of synths and saccharine, but instead of insipid lyrics about love it benefits a lot from Andy’s lyrical prowess. It’s a bouncy bop that would be forgettable if it weren’t for the excellent story told by the words is, I suppose, what I mean to say. And, of course, it’s the song that would have made me think about this. Anyway: I enjoyed it and would absolutely listen to it again, and I don’t know what that says about me, about the song, or anything else.

Uncanny Valley doesn’t work as well. Musically, it feels like a new artist in 2012 drew inspiration from their dad’s new wave albums but wanted to make it sound more like Carly Rae Jepsen was collabing with Daft Punk. And the result is kind-of…flat? Could I even say… soulless? It feels like it might have been the B-side of their cover of Chemicals Like You, which rolls out next in the same vein. Twice was probably enough for these two.

Repetition 4 is among my favourite – let’s say top 15? – Pagan Wanderer Lu songs and the AI’s cover of it starts so strong. It finishes pretty strong too. The voice it’s chosen shows only a hint of uncanny-valley-autotune and it wails plaintively. The most human-made bits – the lyrical themes of fighting for creativity against your own struggles as a vulnerable and flawed human “machine” – remain solid. I really expected to love this one! But by the time we were half way through the song it felt… musically-repetitive. You know when you get a pop cover of a classic song sometimes³ and you feel like the cover artist… missed the point somehow? That’s what this feels like to me.

The repetitions of “we are all machines… for dancing” in the original felt meaningful and real; a human’s cathartic resignation to pleasure in the simple things we all enjoy, despite the challenges of life… but the AI cover adds this kind of doo-woppy backing vocals that subtract, rather than adding to, the meaning. I’m not saying it ruins it – it’s still a fun and bouncy version of a great song… but it’s one of those covers that leaves you longing for the original.

And then there’s the “unaligned version” of Uncanny Valley. I’m not sure if the introduced distortions in this version are AI-generated or not. They don’t feel like the kinds of “creative” choices that any AI I’ve played with would make, so I suspect this represents a closer human intervention in the AI’s process: humans imitating machines imitating humans, perhaps? Anyway: the change doesn’t add anything for me.

Had this been produced entirely by a human, I’d say that EP consists one one track I’d add to my everyday playlist (the cover of Checker Charlie), maybe one or two tracks that I “wouldn’t necessarily skip” if they came up on a random shuffle while I wad driving… and the rest just feels too much like “bad cover” vibes.

And that’s as much of a review as I’m willing to give, for the reasons touched-upon below.

Building the engines of our own defeat

I continue to have several issues with the widespread use of generative AI, and in particular I have problems with it being used in the production of art. Those are partially mitigated by it being used by an artist to remix their own work, and partially mitigated by the transparent declaration of the use of AI by the publisher both of which are true in this case. But many issues (ethical, environmental, etc.) still remain.

Perhaps the biggest of which in this case is my concern that we’re using automation wrong.

As a child, I was optimistic about a future in which machines would take away the boring and repetitive work that humans do, leaving us free to pivot to experimental and experiential roles: the joy of working hard in the quest of discovery and of creativity. But instead, the predominant popular use of generative AI is to replace exactly those things, leaving humans only with an increasing amount of drudgery, review, and fact-checking. Where did we go wrong?

Don’t get me wrong: I love that Pagan Wanderer Lu has created this EP. Taking art that he’s created, whose concept touches on the concepts of AI… and feeding them into an actual AI for reinterpretation is transformative. It’s worthy of discussion as a piece of art in its own right. And the result is… well, some of it’s good, and other bits are okay.

What I don’t like is what it represents: the wider societal issue of the mainstream use of these technologies that have enormous unsolved problems.

So I guess… I appreciate the cognitive dissonance of enjoying a peice of music and disliking what it means?

Footnotes

¹ Whether or not the side-effect of undisclosed AI-generated content “poisoning the well” for future AI training is a good or bad thing remains an open question, in my mind, but it’s certainly a real phenomenon. You know how we salvage the wrecks of ships sunk before the atomic age because they’re untainted by man-made radioactivity, which makes them useful for special purposes? It feels like the Internet before the explosion in generative AI may provide a similar cultural resource for future AI training, if you see what I mean.

² And assuming I wasn’t already familiar with the artist, who doesn’t usually sound like an auto-tuned female singer.

³ I don’t have a specific example so I hope this is a universal experience!

Wikipedia @ 25: Rail transport in Indonesia

Duration 7:22

Podcast Version

This post is also available as a podcast. Listen here, download for later, or subscribe wherever you consume podcasts.

To celebrate the site’s 25th birthday this year, Wikipedia is encouraging/challenging people to read one Wikipedia article a day for 25 consecutive days. I felt that I could do one better than that: not only reading an article but – where I found one that was particularly interesting – to write a blog post or record a podcast episode for each of those days, sharing what I learned. For each entry, I’ll hit “random article” a few times until something catches my interest, start reading, and then start writing! Everything I’ve written below came from Wikipedia… so you should check other sources before you use it to do your homework. Happy birthday, Wikipedia!

Today’s random article: Argo Wilis
Today’s topic: Rail transport in Indonesia

A long train snakes around hillside stepped farms in a tropical and mountainous landscape. — The *Argo Wilis*, near Lebakjero Station. Photograph courtesy of Naufal Farras, used under a Creative Commons license.

With such an unfamiliar-sounding article title as “Argo Wilis” I momentarily thought I was playing Two Of These People Are Lying, but it turns out that it’s just a train. Well, I say just a train, but it’s a train that took me on a journey (ah-hah!) to a rabbithole of Wikipedia pages, and today I’m going to drag you along with me.

The Argo Wilis is a train that goes back and forth along the Southernmost train line connecting Surabaya Gubeng, in the East, to Bandung, in the West, along Java, the vastly most-populous island of the Indonesian archipelago: most of the length of the island. “Argo” means “mountain”: it’s part of a modern collection of “Argo network” trains that are each named after mountains in the region. Mount Wilis itself is a dormant volcano whose magma chamber apparently has the potential for future geothermal power generation possibilities.

Map showing the East-West route of a train line along the island of Java. — Map courtesy Twotwofourtysix, used under a Creative Commons license.

Learning about the Argo Wilis got me to reading about rail travel in Indonesia in general. There are particular challenges to running a train network in a mountainous island nation with a somewhat monsoonal climate, it seems!

Like: one of the stops on the Argo Wilis‘s line is Cipeundeuy, a relatively tiny mountain station that every single passing train stops at in both directions. Why? Because every train is required to have its brakes tested here before proceeding down the mountain slops on either side of it!

Cipeundeuy Railway Station, a small white building with a railway track running alongside, with people on the platform. — All services must stop here, and have since the 1910s (except for a brief period in the 1990s).

That rule’s existed since the railway was first built, under Dutch East Indies rule, over a century ago. It’s been consistently enforced ever since… except for a spell in the early 1990s when the practice was stopped… until a head-on crash in 1995 nearby acted as a reminder of the importance of the checks, at which point they were reinstated.

Workers pose at a railway tunnel under construction in the mountains. — The construction of the Javanese railways up and over or through the many mountains of the island would have been an incredible feat of engineering even today, let alone in the late 19th and very-early 20th centuries.

Anyway, here are some other things I learned about Indonesia’s railways while I was exploring Wikipedia:

Trains drive on the right

Like many island nations (and in common with some non-island nations, particularly those that were part of the British Empire), Indonesian cars drive on the left. But unusually, their railways don’t follow the same pattern: on twin-tracks, Indonesian trains typically travel on the right.

The Dutch colonists were already running their railways on the right and brought this tradition with them, but when the Netherlands switched to right-hand driving for their cars in 1906 (except in Rotterdam, which imposed no fixed rules about which side of the road you should drive on until 1917!), they only dragged some of their colonies along for the ride.

Suriname is another former Dutch colony that still drives on the left. The question of which side their trains travel on is somewhat moot, though, because they don’t currently operate any trains on their railways.

Train classes

Not sufficing to have just first and second class travel like we do here in the UK, Indonesian trains are broken down into at least four classes: luxury, executive, business, and economy. Plus a further two categories for tourist-centric trains, imperial and priority. Plus some sub-classes that seem to be line-specific.

Interior of a busy train carriage. — “Premium economy”-class interior of the train *Sawunggalih Utama*. Photo courtesy Gaudi Renanda, used under a Creative Commons license.

It’s all mostly diesel locomotives…

Jakarta’s got an electrified metro system, but most of the Indonesian rail network’s powered by diesel. However, a handful of industrial narrow-gauge mountain railways might still see the use of steam locomotives for farming or mining purposes, like this one seen hauling sugar cane in 2003:

A small steam locomotive pulls carts full of cut sugar cane along a railway line through cropped fields. — Photo courtesy Joachim Lutz, used under a Creative Commons license.

Jakarta was supposed to be getting an electrified monorail, but the project stalled in 2008 and the already-built infrastructure is in the process of being demolished.

Lebong Tandai is a special case

The remote mountain village of Lebong Tandai is only reliably connected to the rest of the world via a mountain railway line. Much of the narrow-gauge track is connected via a plateway, rather than by sleepers, and residents operate the tiny motorised locomotives independently of the rest of the railway network.

A tiny enclosed passenger rail vehicle crosses an old narrow iron bridge in a jungle. — This “Molek-Motor” on the remote line to Lebong Tandai is constructed out of the remains of a goods vehicle that was written-off after an accident. Photo courtesy Harry Siswoyo, used under a Creative Commons license.

Anyway, that’s what I enjoyed learning about on today’s Wikipedia dive. I wonder what I’ll learn tomorrow! (If it’s as-interesting, I’ll let you know!)

Wikipedia @ 25: Wesley Merritt

Duration 10:38

Podcast Version

This post is also available as a podcast. Listen here, download for later, or subscribe wherever you consume podcasts.

To celebrate the site’s 25th birthday this year, Wikipedia is encouraging/challenging people to read one Wikipedia article a day for 25 consecutive days. I felt that I could do one better than that: not only reading an article but – where I found one that was particularly interesting – to write a blog post or record a podcast episode for each of those days, sharing what I learned. For each entry, I’ll hit “random article” a few times until something catches my interest, start reading, and then start writing! Everything I’ve written below came from Wikipedia… so you should check other sources before you use it to do your homework. Happy birthday, Wikipedia!

Today’s random article: Governor-General of the Philippines
Today’s topic: Wesley Merritt

The Philippines spent a lot of modern history under colonial rule:

First, from 1565, by the Spanish out of their Viceroyalty of New Spain (Mexico).
Then by the British for a few years who captured it after Spain sided with France in the Seven Years War.
Then back to Spain at the signing of the 1763 Treaty of Paris, where, when Britain was arguing which captured territories it should be allowed to keep, everybody forgot about it and so it fell into the default bucket of “back to its previous controller”: it seems that Spain hadn’t even noticed that Manilla had been captured!
Then, after the Mexican War of Independence… still under Spain, but now directly under the Spanish crown and managed from Madrid.
And finally, courtesy of the 1898 Treaty of Paris, under the United States (with the exception of the period during which it was occupied by Japan).
The Philippines finally gained independence in 1946.

The US flag is lowered and the Philippine flag is raised. — The Flag of the United States of America is lowered while the Flag of the Philippines is raised during the Independence Day ceremonies on July 4, 1946.

As you might expect if you know anything about colonialism, there are absolutely horrible stories that could be told about any of those periods of history. So when I landed on the page Governor-General of the Philippines, I decided that it might be cheerier to pick out a person from it.

And so I picked what I believe to be the person whose term as Governor-General of the Philippines was shortest: in post for just 16 days in August 1898: Wesley Merritt.

Portrait photograph of a young white man in military uniform. — Gen. Wesley Merritt, circa 1865.

Wesley was a cavalryman in the American Civil War during which, in 1863, he managed to leapfrog three ranks by getting promoted from Captain right up to Brigadier General. After the Civil War he was posted to the Texan frontier where he commanded a cavalry regiment in the American Indian Wars. His success in… umm… “freeing up land” for American settlers (it turns out this post can’t escape from the ugliness of imperialism)… lead him to a new role in using his troops to police the civilians rushing to “claim” land formerly occupied by native Americans.

But it’s right at the end of the 19th century that his story intersects with today’s random article.

Spanish propaganda cartoon showing Uncle Sam standing atop the United States and reaching out his long arms and boney fingers across the Caribbean towards Cuba. — “Uncle Sam’s Craving: Saving the island so it won’t get lost.” says this Spanish propaganda cartoon.

As the 19th century wore on, the world-spanning Spanish Empire came under serious threat. The Napoleonic Wars had cut Spain off from its colonies, and one by one they lost control of Mexico, Peru, Colombia, Chile, Argentina, and others (often with thanks to quiet support from Britain). But Spain had managed to keep hold of Cuba and the Philippines, despite growing unrest and uprisings, which were often brutally suppressed.

Cuba in particular was a major trade partner to the United States, and so the US tried to insert itself as a negotiator in the war between the Cuban independence movement and the Spanish crown.

At the time, the US was working to establish itself as a modern naval power, building new steel warships to compete with European powers and Brazil, and making plans for what would eventually become the Panama Canal, and so this was a perfect opportunity to show off their armoured cruiser the USS Maine.

Cruiser with two funnels, seen from starboard side. — Starboard bow view of USS *Maine*, shortly before her deployment to Cuba. Fun fact: the last surviving officer who was aboard on the day it sank, Wat Tyler Cluverius Jr., would go on to serve as an engineering officer on the *new* USS *Maine*, a pre-dreadnaught battleship that would still be in service at the time of the First World War (although she was only used as a training ship because her coal efficiency was so terrible that it was no-longer sensible to have her cross an ocean).

The Maine got sent to Havana as a show of force and to protect American interests in Cuba, where, a couple of weeks later, she… blew up.

Probably what happened was that the bituminous coal stored in her bunkers was leaking methane out, which spontaneously ignited, starting a fire that ignited the ship’s powder store. But some, including Theodore Roosevelt (who was then assistant navy secretary and on his way to becoming vice-president) and much of the popular press, claimed that the ship must have been struck by a Spanish mine or torpedo.

Newspaper headline and image (The Evening Times, Washington, D.C., U.S., February 16, 1898) reporting sinking of the USS Maine, leading to the Spanish-American War. — Neither the Spanish nor American official reports had been published before the newspapers were claiming that the *Maine* had been sunk deliberately. Fun fact: the inscription on the monument to the victims that stands in Havana claims it *was* deliberate… but *by the Americans* as a false-flag operation to justify a declaration of war against Spain! This interpretation was added by the communist government in 1961.

The next month, after Congress had had a chance to discuss the matter (do you remember when the US Congress used to have to be involved in the US declaring war on another country?), the US declared war on Spain and began actively attacking her fleets and colonies in the Caribbean and the Pacific.

The US fleet steamed into Manilla Bay for what might be the most one-sided naval battle ever. The Spanish fleet at Manilla would have been severely outmatched even were it not for the fact that the second-lead ship was unpowered, the shore batteries’ range was insufficient to be involved, and the mines had been placed suboptimally. Only a single American sailor lost his life in the battle, and it was apparently as a result of a heart attack.

Fanciful patriotic painting showing flaming wooden Spanish warships being bombarded by the guns of a line of steel American battleships. — Battle of Manila Bay by James Gale Tyler (1898).

Okay, we’re at last up to Wesley Merritt‘s bit. Merritt was placed in command of the ground forces that were tasked with capturing Manilla. They sailed out of San Francisco, landed in the Philippines, and prepared to attack the city.

Merritt and Admiral Dewey made a point not to coordinate with Emilio Aguinaldo y Famy, the leader of the Filipino resistance against the Spanish, who by this point had already taken control of most of the Philippines and besieged Manilla, cutting off its water supply and beginning negotiations with the local Spanish leaders. It seems that Americans feared that if the revolutionaries captured the city it would result in significant bloodshed as a result of violent looting and the murder of those who were seen to have collaborated with the Spanish, and so they came up with an alternative plan: the American expeditionary force would attack and capture the city first!

Working through the Belgian consul to Manilla Édouard André, Merritt negotiated with the Spanish Governor-General Fermín Jáudenes to arrange a “mock” battle. The ships in the bay would fire upon a fort that they knew was only used for storage and against defensive walls that they knew they were not capable of breaching, and Spanish troops would be ordered to retreat as Merritt’s soldiers advanced. Then, Merritt would demand that the Spanish surrender the city, and they would comply, turning it over to the American forces.

This would minimise casualties while allowing the Spanish Governor-General to avoid the shame of being seen to have lost the city to the revolutionaries (it being far more politically-acceptable to lose to the might of the American invaders). Meanwhile, Aguinaldo’s troops initially saw the battle as genuine, which led to some casualties as Filipino fighters advanced under fire; they joined the victims of other misunderstandings during the mock battle.

Drawing showing American troops standing at attention on a fort with cannons, as the American flag is raised and the Spanish flag is lowered. — A drawing from *Harper’s Pictorial History of the War with Spain*. There’s a whole lot of pictures of flags getting rotated in this blog post!

Needless to say, the Filipinos deeply resented being told to stay out of the capital city that, given time, they might well have taken for themselves by force, had their efforts not been leapfrogged by the USA. Ultimately this lead to a guerilla warfare campaign against the USA by Philippine nationalists, which in turn contributed to growing concern in US political circles that America was becoming exactly the kind of imperialist power that it had opposed, at least on paper, since its founding.

Anyway: on 13 August 1898 Wesley Merritt became the de facto Governor-General of the Philippines and the first American to hold that position. Two weeks later Major General Elwell Stephen Otis turned up and relieved him of the position, making Merritt the shortest ever Governor-General of the Philippines.

An older whtie man in military uniform with medals and a sash. — Major General Wesley Merritt from *Illustrated Roster of California Volunteer Soliders in the War with Spain* (1898).

Merritt retired the next year and lived ten more years.

Anyway: that’s enough of today’s history lesson courtesy of a random Wikipedia page. I wonder what I’ll learn tomorrow! (If it’s as-interesting, I’ll let you know!)

Dynamically-Deployed Static Site Subdomains on Caddy

I’ve recently been experimenting with where I host my small and open-source static sites. In my latest experiment, I wanted to try a low-maintenance selfhosting solution¹. Here’s what I wanted:

Pushing to the main branch of my GitHub/Codeberg/wherever repo would send a webook to my server.
Upon receiving the webhook, my server would pull the latest changes².
Using a wildcard certificate, my webserver automatically mounts each project at a subdomain matching its project name³.

Here’s what I came up with:

Step 1: webhook handler

I’m using Caddy as my webserver, because despite its considerable power and versatility it’s a breeze to set up. To sort wildcard DNS later I’ll want to swap in a custom build, but to get started I just ran apt install caddy. Then I used apt install webhook to install Adnan Hajdarević’s webhook endpoint, and tied the two together in my Caddyfile:

webhook.duckling.danq.me {
  reverse_proxy localhost:9000
}

My static server’s called duckling.danq.me, so you’ll see that turn up a lot in these configs.

Then I created a webhook in a GitHub repository:

GitHub webhook pointing to https://webhook.duckling.danq.me/hooks/github-push, with a secret set and SSL verification enabled, triggered by a push event. — I generated a long random string to use as the secret, and kept a copy for later.

When you create a webhook in GitHub it immediately sends a test event, but it doesn’t quite look like a real push event so I pushed an inconsequential change to the repo to trigger another. Once you’ve got a “real” one sent, you can re-send it via the “Recent Deliveries” tab as many times as you like, to help with testing.

Then, on the server, I checked-out a copy of the code (anonymously: this is a public repository so I don’t need keys to read from it anyway) and set up my /etc/webhook.conf to expect these calls:

[
    {
      "id": "github-push",
      "execute-command": "/var/www/github-push/webhook.sh",
      "command-working-directory": "/var/www/github-push/",
      "pass-arguments-to-command": [
        {
          "source": "payload",
          "name": "repository.name"
        }
      ],
      "trigger-rule": {
        "and": [
          {
            "match": {
              "type": "payload-hash-sha256",
              "secret": "[MY SECRET KEY HERE]",
              "parameter": {
                "source": "header",
                "name": "X-Hub-Signature-256"
              }
            }
          },
          {
            "match": {
              "type": "value",
              "value": "refs/heads/main",
              "parameter": {
                "source": "payload",
                "name": "ref"
              }
            }
          }
        ]
      }
    }
  ]

The trigger-rule directives ensure that (a) the secret key is correct (it uses a HMAC hash across the entire JSON request, so it prevents payload tampering too) and (b) the event only triggers on pushes to the main branch. The execute-command specifies the Bash script I want to run when the webhook is triggered. The pass-arguments-to-command configuration says to send the repo name on to that script.

Now all I needed to do was write the /var/www/github-push/webhook.sh Bash script so that it pulled the latest copy of the code when triggered:

#!/bin/bash
cd /var/www/github-push/$1 && git pull

I was able to test this by pushing inconsequential changes to my codebase and watching them get replicated down to my webserver. Neat!

Step 2: low-maintenance webserver

After pointing the DNS for *.static.duckling.danq.me at my static server, I set about configuring Caddy to be able to use DNS-01 challenges to get itself wildcard SSL certificates⁴. Caddy can’t do DNS-01 challenges out of the box, so you either need to write your own renewal script or compile Caddy with plugins corresponding to your DNS provider. My domains’ DNS are managed by a mixture of AWS Route 53, Gandi, and Namecheap, so my xcaddy build step looked like this:

xcaddy build \
  --with github.com/caddy-dns/route53 \
  --with github.com/caddy-dns/gandi \
  --with github.com/caddy-dns/namecheap

Of course, if I’d have preferred somebody else build it for me, CaddyServer’s download configurator would have done it for me on-demand.

For Gandi and Namecheap I just need a personal access token or API key, respectively, but Route 53’s configuration is slightly more-involved: I needed to create a new user via IAM and give it permission to write DNS TXT records for the appropriate hosted zone. Fortunately the guide for the caddy-dns/route53 repo had an almost copy-pastable example.

I added the AWS access key and secret key as environment variables (like this!) into my /etc/systemd/system/multi-user.target.wants/caddy.service service definition, and then told my Caddyfile to make use of them when renewing the wildcard certificate:

*.static.duckling.danq.me {
    tls {
        dns route53 {
          access_key_id {env.AWS_ACCESS_KEY_ID}
          secret_access_key {env.AWS_SECRET_ACCESS_KEY}
        }
      }
      root * /var/www/github-push/{http.request.host.labels.4}
      file_server
    }
}

The {http.request.host.labels.4} refers to the fourth part of the domain name, when separated at the dots and counted from the right, so 0 = me,

1 =
                danq

, 2 = duckling, 3 = static, and 4 = the part that we’re interested in. So long as I don’t store any other directories in the /var/www/github-push/ directory then this will simply map each subdomain onto its git repository name and return a 404 for any other request.

DNS-01 challenges are necessarily slower than HTTP-01/ALPN challenges, because they’re limited by DNS propogation, so it took a while before the certificate was issued. I ran Caddy in the foreground to watch the logs while it did so:

You can see the whole thing working (for now at least; I don’t know if I’m keeping this approach!) by going to e.g. embed-html.static.duckling.danq.me, which dynamically tracks the main branch of the embed-html repo on GitHub.

I don’t yet know if this is going to be the future forever-home of my many static site side projects, but it’s certainly been the most-satisfying experiment to run so-far.

Footnotes

¹ I’ve drifted away from selfhosting simple static sites lately because I’ve accidentally broken them with configuration changes too many times! But I figured I’d be open to in-housing them again if I had a single simple architecture for them all, so I spun up a VPS and gave it a go

² Running a build script or some other static site generation tool is out of scope for now, but I want to be able to confirm that it would be possible in the future.

³ It also needs to be possible for me to map other domain names to it, but that’s a triviality.

⁴ It’s absolutely possible to use tls { on_demand } to do this, but it’s better to use a wildcard certificate which can be pre-generated and doesn’t let people trick your server into making ludicrous numbers of certificate requests by hammering random subdomain names.

A Postcard from Norway!

I got another postcard, and another new country for the senders list.

Dan, a white pan with a blue ponytail, holds a postcard in front of his face. — It’s always an exciting surprise to receive a Postcard From The Internet(!).¹

I’ve added it to my gallery. This one’s from grubbyfox, whom I’ve crossed paths with on a Web forum².

It remains a huge pleasure to receive a postcard from an Internet stranger (or even somebody I already know). It’s so much more tactile, and thoughtful, and human, and real than most of the other feedback I get.³

I guess there’s sort-of a scale of effort that it takes to react to content online. At the lowest end of the scale, barely above “doing nothing”, is “clicking a reaction button” (hey, did you see that you can click one below this?). It takes more effort to fill in a contact form. More still to send an individual email or ping me on Mastodon. But yet more still to write a postcard, find a stamp, go to the postbox… And I really appreciate it when somebody makes the effort.

Footnotes

¹ The surprise is dampened only slightly by the fact that my PO Box provider emails me in advance to tell me I’ve received something.

² Remember Web forums? They’re still very much around, and there are some cool communities if you can find the right one for you.

³ If you want to send my a postcard, my PO Box address is over here…

Moving a static site from GitHub to Codeberg Pages

Late to the party,¹ I finally got around to experimentally moving a GitHub Pages-hosted static site to Codeberg. I wanted a low-risk site to try first, so I moved Beige Buttons, the site hosting my “90s PC turbo button simulator” web component.

Ê

Mostly for my own benefit later, here’s the steps I took and the things I learned along the way:

You can migrate a repository across in about two clicks. Easy!
Codeberg Pages is deployed from the pages branch. If there’s no build step to the static site, all you need to do is rename the main branch to pages (and probably make it the default branch).²
The default URL is https://username.codeberg.page/repository.
You can use a custom domain by adding a .domains file that lists domains; if migrating from GitHub Pages you can just rename your CNAME file to .domains.
You’ll need to tweak your DNS CNAME, ALIAS (or, worst-case, A/AAAA) record to point at Codeberg Pages.³

Change propogation feels slightly slower than GitHub, but perfectly tolerable.

The one thing that’s causing me trouble is that Codeberg Pages’ CORS headers prevent people from hotlinking the Beige Buttons JS, so there are some projects for which this wouldn’t be a suitable migration (issues are raised). But for most static sites, it’d probably Just Work and seems to be a great alternative.

Footnotes

¹ With thanks to Kev for reminding me I’d had this on my list.

² There are other ways to deploy but they don’t support custom domains yet.

³ Like GitHub Pages, Codeberg Pages uses LetsEncrypt for certificate provision, so you don’t need to change any CAA records.

£4,803.40 a Year for Water?

When we picked up the keys to the rental house we’ll be living in while our home is repaired following the flood that forced us to evacuate, I took an initial meter reading then got in touch with super-reputable water company Thames Water to let them know the situation.

A digital water meter showing the value 381641. — The Chicory House’s water meter, found in a cupboard, is so much easier to read than the one at our regular house, which is found down a frequently-flooded manhole on the busy road outside.

Unfortunately, Thames Water had fucked-up¹ and created an account for us already with the wrong information², so by the time I’d reached out to them they were already getting themselves into a pickle.

It turns out that, presumably because of some shortsightedness on the part of their software engineers, their computer systems wouldn’t let them change the information to correct the problem. Nor could they simply delete the account and create a new one³. Instead, the had to close the account they’d erroneously set up such that the start and end date of the contract was our moving-in date… and then set up a new account starting from the day after.

Sigh. Fine! So long as it’s sorted, I didn’t even care. Until, that is, the bill arrived for the one day of the first (incorrectly-created) contract:

A Thames Water bill dated 18 April, for an account being closed '04/09/2026', covering a billing period of 9 April - 9 April, for £13.16. — This looks pretty low for a metered water bill, until you realise that it covers a period of *literally only eleven hours* from us moving in (and taking a meter reading) until the end of that day. And that during most of that time the water was switched-off because a pair of plumbers were installing a new bathroom!

That bill:

is for £13.16.
covers “9 April 2026 through 9 April 2026”, i.e. one day.
- (which means that our estimated annual bill would be £4,803.40 (£13.16 × 365) – about eight times the national average)
states that our account closure was/will be “04/09/2026” – the only date on the letter that’s in “short” date format and which would appear to be 4 September (in UK date format) even though 9 April would make more sense (but would require interpreting it in US date format, which would make no sense).

Let’s see how that breaks down:

Breakdown of the '1 day' bill covering three cubic metres of water at £2.7346 (usage) and £1.4721 (sewerage), plus fixed charges, totalling £13.16. — The rates are standard, albeit a little on the high end: Thames Water need to raise funds right now to fix all of the leaks in their pipes, apparently. What’s odd is the volume of water they claim has been used.

According to this bill, we used three cubic metres of water between collecting the keys (at around 1pm), moving in, and taking a meter reading… and the end of the day. That’s three thousand litres of water.

Is it possible to achieve that level of water usage in the nine hours of billable time that this bill covers? I guess, if you really tried, you could:

completely fill and then drain our 100-litre bathtub, three times an hour, taking a five-and-a-half minute bath in each before draining it again, for the rest of the day⁴; or
run the kitchen tap – the highest-pressure tap in the house – continuously for six hours and forty minutes; or
repeatedly flush all three toilets, on “full-flush” mode, once every 79 seconds until midnight⁵, for example.

Dan, a white man with a blue ponytail, wearing a green t-shirt, watches a stopwatch timer on his phone while filling a measuring jug from the kitchen sink's tap. — Some science was involved in the writing of this blog post.

Obviously this is all ridiculous. I’m being ridiculous.

But then again, so is this bill, which claims that three adults spent 11 hours in a house and somehow used the amount of water that’s the recommended amount to drink in a day… by 1,500 adults. Despite the water being shut-off to install a shower and toilet for some of that time.

But then again, so is Thames Water’s computer system, which disallows the correction of mistakes even by their own staff and instead requires the creation of one-day contracts. And also can’t decide which country’s date format to use. And, possibly, doesn’t allow them to obey data protection laws.

The whole thing’s ridiculous. Which I’ll be letting Thames Water know. Let’s see if they agree.

Footnotes

¹ This may be no surprise to anybody who’s ever dealt with Thames Water before, or who follows the news about their seemingly endless inability to keep clean water in its pipes and raw sewage out of our rivers, for example, while taking out loans in order to pay bonuses to their self-back-patting executives.

² They used information provided to them by the estate agent and failed to connect it to the information they already had for us… which thanks to quirks of their information systems resulted in bigger problems down the line. Amusingly – and for a change! – none of the problems were related to my unusual name, this time around.

³ Curiously, these initial mistakes on the part of Thames Water left them processing personal information about me – an email address – that I’d never given to them, and allegedly unable to delete or correct it for six months after being asked to. This is the kind of thing that normally gives me an excuse for a field day of DPA2018-related letter writing, but this time around I’ve been too busy dealing with the bigger problems they’ve created to have a chance to stop and think about that: that’s how much of a mess they’ve made.

⁴ It’s only barely possible to repeatedly fill the bath this quickly, you need to use both hot and cold water: the cold inlet alone doesn’t have the pressure to fill it fast enough, but the hot water tank has its own separate inlet which makes all the difference. Also, a cold bath would suck, even if you’re only allowed five minutes in it before it’s time to drain the tub and start filling it again.

⁵ I once had a really rough night after a particularly dodgy curry, but I’ve never needed to be flushing a toilet twice a minute for eleven hours.

Once You’re Asking the Right Question, You Don’t Need To Ask!

Folks at work have been encouraging to make more use of generative AI in my workflow¹; going beyond my current “fancy autocomplete” use and giving my agents more autonomy. My experience of such “vibe coding” so far has been… mixed², but I promised I’d revisit it.

One thing that these models are usually effective at is summarisation³. This is valuable if you’re faced with a large and unfamiliar codebase and you’re looking to trace a particular thing but you’re not certain where it is or what it’ll be called. While they’re not always fast, these tools can at least work in the background, which allows the developer to get on with something else while the agent trawls logs, code, and configuration to find and explain a fuzzily-defined thing.

Recently, I had a moment which I thought might be such an instance… but it didn’t turn out quite the way I expected. Here’s the story⁴:

The broken dev env

I’d been drafted into an established and ongoing project to provide more hands, following a coworker’s departure last week. This project touches parts of our (sprawling, microsevices-based) infrastructure that I hadn’t looked at before, so there was a lot I didn’t yet know.

I picked an issue that had belonged to my former colleague that QA had rejected and set out to retrace their steps: to replicate the problem that the QA engineers had identified and in doing so learn more about the underlying process. I spun up my development environment and tried to follow the steps.

But I couldn’t even get as far as their problem before my frontend barfed out an error message. Sigh! Probably there’s some configuration I’ve missed somewhere in the myriad microservices, or else the data I’m testing with isn’t a fair reflection on what they’re doing as-standard.

Following some staff changes, I have no teammates on this side of the Atlantic who could help me decipher this: a “quick question on Slack” wouldn’t solve this one until hours from now. It was time to start debugging!

But… maybe Claude could help? It’s got access to almost all the same code, logs, tools and browser windows I do. I started typing:

✨ What’s up next, Dan?

In my development environment for https://service.dev/asset/new, when I click “Save”, I see the error “Oops, something went wrong.” Why?

Context is key

It’s quite possible that Claude would have gone away, had a “think”, done some tests, and then come back to me with a believable answer. It might even have been correct, and I’d have been able to short-cut my way back to productivity (and I’d have time to make a mug of coffee and finish reading my emails while it did so). Then, I’d just have to check that it was right, make the change, and get on with things.

But I realised that it’d probably work faster (and cheaper, and using less energy) if it had slightly more context from the get-go, so I elaborated. The first thing I’d want to know if I were debugging this is what was actually happening behind the scenes. I dipped into my browser’s Network debugger and extracted the relevant output, adding it to my prompt:

✨ What’s up next, Dan?

In my development environment for https://service.dev/asset/new, when I click “Save”, I see the error “Oops, something went wrong.” Why?The payload POSTed to the server is { content: 'test1', audience: [ 'one' ], status: 'draft' } and the response is a HTTP 500 with the following stack trace: pasted 94 lines

That’s more like it, now I could let it get on with its work. But wait…

Rubberducking

There’s a concept in computer programming called “rubberducking”. The name comes from an anecdote in The Pragmatic Programmer about a developer who, when stuck on a problem, would explain the code line-by-line to a rubber duck. The thinking is that talking-through a problem, even to someone (or something) who doesn’t understand it, can lead the speaker to insights they were otherwise missing.

I’ve done it myself many, many times: recruiting a convenient colleague or friend and talking them through the technical problem I was faced with, and inviting them to ask me to go into greater detail if I seemed to be skimming over anything, and I can promise that it can work.

A witch is happy and proud of her invention - a rubber duck. She explains to her friend: I just figured that formulating my questions out loud helps me to solve them, and finally that's all I needed. — I discovered *Mini Fantasy Theater* recently and loved this episode from its backlog.

The panel above is part of a series in which a sorceress called Cepper who’s coerced by her university into using Avian Intelligence (“AI”) – a robotic parrot⁵ that her headmaster insists is the future of magic. She experiments with it, finds it occasionally useful but more-often frustrating, attempts to implement her own local version but find that troublesome in different ways, and eventually settles on using an inanimate rubber duck instead. I get it, Cepper!

Let’s put that distraction aside for a moment and get back to the story of my broken development environment.

Clues in the stack trace

The top entry in the stack trace was an unsuccessful call to a different microservice, so I figured I’d pull its logs too, in order to further help direct the AI in the right direction⁶:

✨ What’s up next, Dan?

In my development environment for https://service.dev/asset/new, when I click “Save”, I see the error “Oops, something went wrong.” Why?The payload POSTed to the server is

{ content: 'test1',
                audience: [ 'one' ], status: 'draft' }

and the response is a HTTP 500 with the following stack trace: pasted 94 linesThe stack trace suggests that a call is being made to the dojo backend service, where the following error log looks relevant: pasted 9 lines

I haven’t tried it, but I’m pretty confident that the LLM, after much number-crunching and a little warming-up of some datacentre somewhere, would get to the answer. But again, I found something niggling inside me: the second-from top line in the dojo logs suggested that a connection was being made to a further, deeper microservice.

I should pull its logs too, I figured.

The final puzzle piece

As an aide mémoire – in a way I’ve taken to doing when taking notes or when talking to AI – I first typed what I was going to provide. This is useful if, for example, somebody distracts me at a key moment: it means you’ve got a jumping-off point predefined by my past self:

✨ What’s up next, Dan?

In my development environment for https://service.dev/asset/new, when I click “Save”, I see the error “Oops, something went wrong.” Why?The payload POSTed to the server is

{ content: 'test1',
                audience: [ 'one' ], status: 'draft' }

and the response is a HTTP 500 with the following stack trace: pasted 94 linesThe stack trace suggests that a call is being made to the dojo backend service, where the following error log looks relevant: pasted 9 lines. It’s calling osiris, which says:

I dipped into the directory for

osiris , and before I even got to the logs I spotted a problem: that microservice was on an old feature branch. How odd! I switched to the main branch and… everything started working.

The entire event took only a few minutes. I’d find some information, type it into Claude’s input field, realise that more information could be valuable, and repeat.

By the time I’d finished describing the problem, I’d discovered the solution. That’s the essence of successful rubberducking. I didn’t need the AI at all. All I needed was the illusion of something that might be able to help if I just talked through what I was thinking.

I don’t know what the moral is, here.

I wonder if I’d have been as effective had I just typed into my text editor. I suppose I would have, but I wonder if I’d have been motivated to do so in the first place? I’ve tried rubberducking before by talking to an imaginary person, but I’ve never tried typing to one⁷; maybe I should start?

Footnotes

¹ I’m pretty sure every engineering department nowadays has it’s rabid fanboys, but I’m pleased that for the most part my colleagues take a more-pragmatic and realistic outlook: balancing the potential benefits of LLM-assisted coding with its many shortfalls, downsides, and risks.

² My experience of vibe-coding in a nutshell: LLMs are great at knocking out the easy 80% of any engineering problem, but often in a way that makes the remaining 20% – already the hard part – harder than it would have been if a human had done the first 80% (especially if it’s the same human and they can bring their learnings with them)… and I’m definitely not the only one who’s found that. I also suspect that the unsatisfying and unimproving task of shepherding a flock of agents to write code and then casually reviewing it is not significantly more-productive (which research backs up) and results in a significantly increased regression rate… but I’m ready to be proven wrong when more studies come out. In short: I continue to think that GenAI isn’t useless, but neither is it necessarily always worthwhile.

³ So long as what you’ve got them summarising is something you can later verify!

⁴ I’ve taken huge liberties with the strict factual accuracy to make this more-readable as well as to to not-expose things I probably oughtn’t. So before you swoop in to criticise my prompt-fu (not that I asked you, but I know there’s somebody out there who’s thinking about doing this right now), please note that none of the text in this page are what I actually wrote to the AI; it’s a figurative example.

⁵ A literal stochastic parrot, one might say!

⁶ I’d had an experience just the previous week in which it’d gone off on completely the wrong track, attempting to change code in order to “fix” what was ultimately a configuration or data problem, and so I thought it might be useful to give it some rails to follow, to start with.

⁷ Except insofar as this AI agent is an “imaginary person”, which it possibly already a step-too-far in implying personhood for my liking!

BBC News RSS… with full-size thumbnails!

I love that my tool for making BBC News RSS feeds “better” continues to help people¹. But I also enjoy that as a platform, it’s still got room to grow.

For instance, at the start of the weekend I received an email from somebody called Phil, who asked:

Could you possibly have an alternative ‘HQ’ version of your feeds which replaces standard/240 with standard/1200 in the URL for each article in the XML?

I am obviously pretty desperate for this feature, hence me reaching out.

Phil’s right. The BBC News RSS feeds contain thumbnail images that look like this:

<media:thumbnail width="240"
                height="135"
                   url="https://ichef.bbci.co.uk/ace/standard/240/cpsprodpb/623a/live/5f8c30c0-3d7f-11f1-ac78-2112837ce2aa.jpg"
/>

You see the /240/ in that URL? If you change it to /1200/ then, as Phil observes, you get a much-higher resolution thumbnail. Naturally you ought to correct the width and height attributes accordingly, too.

The difference is pretty significant. See:

So I raised Phil’s request as a GitHub issue, like a good maintainer, before realising that – hang on – this would be a really easy improvement and I should just… do it.

My BBC feeds “improver” leverages one of my very favourite RubyGems, Nokogiri, to perform XML parsing and modification. The code you need to tweak these URLs is super simple:

# Iterate through each <media:thumbnail> element in the RSS feed:
rss.xpath('//media:thumbnail').each do |thumb|
  # Skip any that don't start the way we expect:
  next unless thumb['url'] =~ /^https:\/\/ichef.bbci.co.uk\/ace\/(standard|ws)\/240\//
  # Swap the 240 for 1200 in the url="..." attribute:
  thumb['url'] = thumb['url'].gsub(/\/ace\/(standard|ws)\/240\//, "/ace/\\1/1200/")
  # Set width="1200":
  thumb['width'] = "1200"
  # Set the height="..." proportionally (they're not always the same!):
  thumb['height'] = (thumb['height'].to_f / 240 * 1200).round.to_s
end

In the actual code I wrote the magic numbers 240 and 1200 are constants, of course.

That really is all there is to it, but look at what a difference it makes in an RSS reader:

I got that merged and the GitHub action that makes the magic happen got started on its usual 20-minute schedule soon afterwards. I didn’t even have to finish waiting for my lunchtime ramen to cool down before the change was out there and, hopefully, helping people. Phil emailed me again soon afterwards:

You managed to fix something in your lunch break that has been bugging me for well over a decade. The difference in quality is night and day.

Anyway: it pleased me to discover that my software is out there, helping people.

As with most of my open source work, I put little to no effort into tracking any kind of metrics of usage, which means I only get to find out if I’ve done good in the world when people reach out and tell me. So I was delighted to hear from Phil (as well as to take his suggestion and improve the tool for everybody!).

Footnotes

¹ Specifically, the code I’ve written makes a few improvements to the BBC News RSS feeds: (1) removing duplicate news, (2) removing non-news content such as “nudges” towards the app or to iPlayer content, and (3) optionally removing sports news. If that sounds like a better version of the BBC News RSS feeds, you should take a look!

FlipFlop Solitaire’s Deck-Generation Secret

For the last few years, my go-to mobile game¹ has been FlipFlop Solitaire.

The game

The premise is simple enough:

5-column solitaire game with 1-5 suits.
23 cards dealt out into those columns; only the topmost ones face-up.
2 “reserve” cards retained at the bottom.
Stacks can be formed atop any suit based on value-adjacency (in either order, even mixing the ordering within a stack)
Individual cards can always be moved, but stacks can only be moved if they share a value-adjacency chain and are all the same suit.
Aim is to get each suit stacked in order at the top.

One of the things that stands out to me is that the game comes with over five thousand pre-shuffled decks to play, all of which guarantee that they are “winnable”.

Playing through these is very satisfying because it means that if you get stuck, you know that it’s because of a choice that you made², and not (just) because you get unlucky with the deal.

Every deck is “winnable”?

When I first heard that every one of FlipFlop‘s pregenerated decks were winnable, I misinterpreted it as claiming that every conceivable shuffle for a game of FlipFlop was winnable. But that’s clearly not the case, and it doesn’t take significant effort to come up with a deal that’s clearly not-winnable. It only takes a single example to disprove a statement!

Paper-based sketch showing an 'unwinnable' deck of 'five suit' FlopFlop, formed by stacking the cards in a strict reverse order to make it as hard as possible. — If you think you’ve found a solution to this deal – for example, by (necessarily) dealing out all of the cards, then putting both reserve kings out and stacking everything else on top of them in order to dig down to the useful cards, bear in mind that (a) the maximum stack depth of 20 means you can get to a 6, or a 5, but not both, and (b) you can’t then move any of those stacks in aggregate because – although it’s not clear in my monochrome sketch – the suits cycle in a pattern to disrupt such efforts.

That it’s possible for a fairly-shuffled deck of cards to lead to an “unwinnable” game of FlipFlop Solitaire means the author must have necessarily had some mechanism to differentiate between “winnable” (which are probably the majority) and “unwinnable” ones. And therein lies an interesting problem.

If the only way to conclusively prove that a particular deal is “winnable” is to win it, then the developer must have had an algorithm that they were using to test that a given deal was “winnable”: that is – a brute-force solver.

So I had a go at making one³. The code is pretty hacky (don’t judge me) and, well… it takes a long, long time.

Partially that’s because the underlying state engine I used, BfsBruteForce, is a breadth-first optimising algorithm. It aims to find the absolute fewest-moves solution, which isn’t necessarily the fastest one to find because it means that it has to try all of the “probably stupid” moves it finds⁴ with the same priority as the the “probably smart” moves⁵.

I played about with enhancing it with some heuristics, and scoring different play states, and then running that in a pruned depth-first way, but it didn’t really help much. It came to an answer eventually, sure… but it took a long, long time, and it’s pretty easy to see why: the number of permutations for a shuffled deck of cards is greater than the number of atoms on Earth!

If you pull off a genuinely random shuffle, then – statistically-speaking – you’ve probably managed to put that deck into an order that no deck of cards has never been in before!⁶

And sure: the rules of the game reduce the number of possibilities quite considerably… but there’s still a lot of them.

So how are “guaranteed winnable” decks generated?

I think I’ve worked out the answer to this question: it came to me in a dream!

Show this puzzle to any smarter-than-average child and they’ll quickly realise that the fastest way to get the solution is not to start from each programmer and trace their path… but to start from the laptop and work backwards!

The trick to generating “guaranteed winnable” decks for FlipFlop Solitaire (and, probably, any similar game) is to work backwards.

Instead of starting with a random deck and checking if it’s solvable by performing every permutation of valid moves… start with a “solved” deck (with all the cards stacked up neatly) and perform a randomly-selected series of valid reverse-moves! E.g.:

The first move is obvious: take one of the kings off the “finished” piles and put it into a column.
For the next move, you’ll either take a different king and do the same thing, or take the queen that was exposed from under the first king and place it either in an empty column or atop the first king (optionally, but probably not, flipping the king face down).
With each subsequent move, you determine what the valid next-reverse-moves are, choose one at random (possibly with some kind of weighting), and move on!

In computational complexity theory, you just transformed an NP-Hard problem⁷ into a P problem.

Once you eliminate repeat states and weight the randomiser to gently favour moving “towards” a solution that leaves the cards set-up and ready to begin the game, you’ve created a problem that may take an indeterminate amount of time… but it’ll be finite and its complexity will scale linearly. And that’s a big improvement.

I started implementing a puzzle-creator that works in this manner, but the task wasn’t as interesting as the near-impossible brute-force solver so I gave up, got distracted, and wrote some even more-pointless code instead.

If you go ahead and make an open source FlipFlop deck generator, let me know: I’d be interested to play with it!

Footnotes

¹ I don’t get much time to play videogames, nowadays, but I sometimes find that I’ve got time for a round or two of a simple “droppable” puzzle game while I’m waiting for a child to come out of school or similar. FlipFlop Solitaire is one of only three games I have installed on my phone for this purpose, the other two – both much less frequently-played – being Battle of Polytopia and the buggy-but-enjoyable digital version of Twilight Struggle.

² Okay, it feels slightly frustrating when you make a series of choices that are perfectly logical and the most-rational decision under the circumstances. But the game has an “undo” button, so it’s not that bad.

³ Mostly I was interested in whether such a thing was possible, but folks who’re familiar with how much I enjoy “cheating” at puzzles (don’t get me started on Jigidi jigsaws…) by writing software to do them for me will also realise that this is Just What I Am Like.

⁴ An example of a “probably stupid” move would be splitting a same-suit stack in order to sit it atop a card of a different suit, when this doesn’t immediately expose any new moves. Sometimes – just sometimes – this is an optimal strategy, but normally it’s a pretty bad idea.

⁵ Moving a card that can go into the completed stacks at the top is usually a good idea… although just sometimes, and especially in complex mid-game multi-suit scenarios, it can be beneficial to keep a card in play so that you can use it as an anchor for something else, thereby unblocking more flexible play down the line.

⁶ Fun fact: shuffling a deck of cards is a sufficient source of entropy that you can use it to generate cryptographic keystreams, as Bruce Schneier demonstrated in 1999.

⁷ I’ve not thought deeply about it, but determining if a given deck of cards will result in a winnable game probably lies somewhere between the travelling salesman and the halting problem, in terms of complexity, right? And probably not something a right-thinking person would ask their desktop computer to do for fun!

My Salary History

Jeremy Keith posted his salary history last week. I absolutely agree with him that employers exploit the information gap created by opaque salary advertisement, and I think that our industry of software engineering is especially troublesome for this.

So I’m joining him (and others) in choosing to share my salary history. I’ve set up a new page for that purpose, but here’s the summary of its initial state:

Understand

A few understandings and caveats:

For most of my career I’ve described myself as a “Full-Stack Web Applications Developer”, but I’ve worked outside of every one of those words and my job titles have often been more like “CMS Developer” or “Senior Engineer (Security)”.
My specialisms and “hot areas” are security engineering, web standards, performance, and accessibility.
When I worked multiple roles in a year, I’ve tried to capture that, but there’ll be some fuzziness around the edges.
The salaries are rounded slightly to make nice readable numbers.
I’ve not always worked full-time; all salaries are translated into “full-time equivalent”¹.
I’ve only included jobs that fit into my software engineering career².
If the table below looks out-of-date then I’ve probably just forgotten to update it. Let me know!

History

Year	Employer	Salary	Notes
2025 – 2026	Firstup	£80,000	Remote. + Stock (spread over four years).
2024 – 2025	Automattic	£111,000	Remote. + Stock (one-off bonus, worth ~£6,000).
2023	Automattic	£103,000	Remote.
2021 – 2022	Automattic	£98,000	Remote
2019 – 2020	Automattic	£89,000	Remote.
2015 – 2019	Bodleian Libraries + Freelance	£39,000	Hybrid.
2011 – 2014	Bodleian Libraries	£36,000	Practices salary transparency! ❤️
2010 – 2011	SmartData + Freelance	£26,000	Remote.
2007 – 2009	SmartData	£24,000
2004 – 2006	SmartData	£19,500
2002 – 2003	SmartData	£16,500	Alongside full-time study.
2002	CTA	£18,000
2001	Freelance	£4,500	Ad-hoc and hard to estimate. Alongside full-time study.

What does that look like?

I drew a graph, but I don’t like it. Mostly because I don’t see my salary as a “goal” to aim for or some kind of “score”.

It’s gone up; it’s gone down; but I’ve always been more-motivated by what I’m working on, with whom, and for what purpose than I have been on how much I get paid for it³. But if you want to see:

I’m not sure to what degree my career looks typical or not. But I guess I also don’t care! My motivations are probably different than most (a little-more idealistic, a little-less capitalistic), I’d guess.

Footnotes

¹ i.e. what I’d have earned if I had worked full-time

² That summer back in college that I worked in a factory building striplight fittings doesn’t appear, for example!

³ Pro-tip if you’re looking at my CV and pitching me an opportunity: mention what you expect to pay, sure, but if you’re trying to win me over then tell me about the problems I’ll be solving and how that’ll make the world a better place. That’s how you motivate me to accept your offer!

Chicory Keys

Towards the end of last week we picked up the keys to the Chicory House.¹ We’ve now officially moved in to the place we’ll be calling home for the next six months or so, while we wait for our Actual House to be repaired following our catastrophic flood in February.²

As part of my efforts to travel light, I use a pretty small wallet – a lump of carbon fibre about the size of a deck of cards³ that contains my ID, bank cards, and – in pocket at the back – my essential keys. Typically that’s my front door key and my bike lock key.

Minimalist carbon fibre wallet, balanced on two fingertips, with parts of a Halifax Mastercard credit card showing from behind an elasticated band. — The keys tuck in around the back, but there’s a “hook” on the end to which additional keys can be ringed. Sometimes I hook up a second-factor hardware token to it when I’m travelling with one.

And so when I received my front door key to the Chicory House, I had to decide: where does this key belong?

The obvious answer would have been to remove the front door key for my actual home from its special place within my wallet and replace it with the Chicory House’s front door key. That’s the one I’ll need most-often for the foreseeable future, right? My regular front door key can move to the supplementary hook, on a ring, and/or be removed entirely and taken with me only when I need to visit my uninhabitable home.

But that’s not what I did.

Reverse side of my wallet showing my regular house key folded-out from its special spot, and the Chicory House key attached to the hook. — I didn’t even think about what I was doing until I noticed, afterwards, that I’d chosen to put the Chicory House key on the “supplementary keys” hook rather than in the “primary keys” spot.

This made sense as an instinctive move: it’s where I’d clip on the key to any of the half-dozen or so AirBnBs I’ve lived in for the last couple of months, after all! But for a house I’m going to live in for half a year or more it doesn’t seem so rational.

But I haven’t put it back. I think I’m keeping it this way. My regular key gets to keep its special spot because it represents the lost status quo and the aspiration to return. Sure, it’s less-practical for me to keep it there, but its position is symbolic, not sensible.

Swapping the two over would feel like giving in: like caving to the inevitability of us being out of our home for an extended period. Keeping the key where it is means that every time I put my hand in my pocket I’m reminded that the current arrangement is temporary; things will go back to normal. And that’s nice.⁴

Footnotes

¹ The house isn’t actually called that, of course. That’s our nickname for it, on account of it being a substitute for the real thing.

² The flood was exactly two months ago today, which makes today “F-Day plus 60”. We’ve spent most of the intervening time hopping from AirBnB to AirBnB.

³ As somebody who often carries a deck of cards, this is a pretty-convenient size to me!

⁴ That said, the Chicory House is way better than most of the AirBnB’s we’ve been living in, and I’m especially loving being able to sleep on my own familiar mattress again! While I wouldn’t want to live here forever like I’d be happy to in the place we’ve called home since 2020, it’ll certainly suffice for the immediate future. A stepping-stone back towards the lives we’d built before.

Letting Games Die

Letting code (and games) die

Mike Cook wrote a provocative blog post this weekend; an anti-preservationist argument for video games. The essence of his arguments seem to boil down to:

Emphasising creation over preservation is liberating, as demonstrated by the imagination in the livecoding community.
Archiving without intensive curation is building an emotional or intellectual safety net you never expect to be used.
Digital preservation is a lossy process: effort spent on accurately preserving some media is at the expense of other media, whose lossy preservation paints in inaccurate picture of what is lost.
Recreation, rather than strict preservation, ensures the continuity of the most culturally-important parts of games

He concludes to say:

60 games are released on Steam every day.

There are 294 game jams active on Steam as I write this.

Preserve nothing. Make more.

To make is to preserve.

Let games die.

Digital preservationism

Philosophically-speaking, there’s no doubt that I am a digital preservationist. I argue against unnecessary URI changes. I donate to The Internet Archive. Back at the Bodleian, I used to carve out free time from project work to spend time making sure the University’s “older” exhibition websites could be made to survive¹. My approach to running out of hard drive space is to buy more hard drives. Even my blog retains content going back into the last millennium²!

But I like this kind of conversation. For World Digital Preservation Day a few years back I re-implemented Pong as a modern application but using retro controllers. Within its micro-exhibition, I used this as an excuse to get people to discuss what does it mean to preserve a videogame?

Two participants play Pong on the Heritage Window in Blackwell Hall at the Bodleian Library. — My reimplementation of *Pong* had several distinct differences from the original… but to a layperson – for whom *Pong* are the target audience! – those differences are irrelevant. To what level fidelity matters depends on many factors, and the biggest problem is that we don’t know what those factors are until it’s time to retrieve these historical media.

Similarly, back in 2021 I reverse-engineered and re-implemented “lost” piece of advertainment Axe Feather, mostly because I felt that a slightly-modernised version belonged in the “commons”.

A woman lies on a bed with her legs crossed, playfully wagging her finger. The mouse cursor is shaped like a feather. — This screenshot isn’t from the original site but from my homage to it. More on that later.

This makes it seem like I’m very much on the side of recreation, rather than preservation, but that’s not the case. In both of these projects I started by disassembling the original works.

That I chose to make them accessible to a modern audience by reimplementation rather than by emulation was an artistic choice. I opted for lower fidelity by making something mildly-transformative. I chose to appeal to the widest possible audience, at the expense of presenting an experience that was totally in-keeping with the original.

But I couldn’t have done that without access to the originals. Had I recreated Pong from memory rather than from re-playing it, I’d have doubtless introduced inconsistencies that would have “felt wrong” to people whose memories of the game, while fundamentally accurate, differed from mine. Had I recreated Axe Feather without first coming up with a mechanism to extract and reformat the video clips in the original I’d have failed to tap into the specific nostalgia of some of its users, which was tied to the specific actor who performed in it³.

So I guess it’s important to me that somebody is preserving these things. So that I can use them to create new things. I stand for preservation for culture’s sake, so that I personally can enjoy the benefits for nostalgia’s sake.

But I get what Mike’s saying

For all that I feel like I’m making the case for “preserve everything; work out what’s important later”, Mike’s argument gives me an uncomfortable cognitive dissonance. Because I’ve also come to discover a joy in the ephemeral, too.

Screenshot from M1 on ARCC, showing the high-score table with DAN50 holding the top score of 14963 (second place scored 12204). — I don’t know who’ll preserve ARCC, with its permanently-capped 500-playerbase limit, but I’m happy that I’ll probably *always* hold the highscore on driving/racing minigame M1.

Increasingly, I’m okay with just taking the experience of something with me. It bothers me that my memory is fallible and that I can’t necessarily recreate a digital experience whose technology has been lost to time, but I am, for the most part, okay with it.

Some of the best gaming experiences I’ve ever had are impossible to “capture” in an archive anyway. They were conversations over the tabletop roleplaying table, or moments of tension resulting from a videogame’s emergent gameplay, or random occurrences unlikely to be replicated. Those get preserved in my memory alone, retold as stories with gradually-decreasing accuracy as new memories take their place.

That said…

Who decides what games get preserved?

I feel like the decision about what to preserve and how should be in the hands of the audience of a piece of art, not its creators. If a videogame (or film, or book, or whatever) is culturally-significant enough to warrant a high-fidelity preservation, it ought to be ultimately up to the members of that culture to make that decision!

Transport Tycoon Deluxe met that bar, and it’s possible to play both faithful recreations or modern reimplementations (the latter having excellent new features) courtesy of the OpenTTD project⁴.

But modern videogames are, perhaps, getting harder to preserve. Always-online features, insidious DRM, digital distribution, live updates, and games-as-a-service streaming all shift the balance of power more-firmly into the hands of publishers⁵ rather than players. It’s already hard to play a randomly-selected thirty-year-old videogame today; I reckon it’ll be almost impossible to do the same thirty years hence.

Saying “let games die” feels a bit like giving up to that inevitability. Like saying to the slimier publishers “it’s okay, we didn’t care about keeping that anyway” when they shut down servers or remotely kill games. I know that’s not what Mike’s saying, but it could be wilfully misinterpreted that way.

Anyway: I don’t have a nice conclusion to any of this. Just a lot of mixed-up feelings.

Footnotes

¹ A policy which, since my departure, does not seem to have continued.

² Even where those writings don’t really represent me well any more.

³ It turns out that, for a significant number of folks who are mostly younger-than-me, this advertisement represented a kind of sexual awakening, based on some of the comments and emails I’ve received about it!

⁴ Which I’ve also donated too. Turns out I’m happy to invest in both pure preservation and in spiritual-successor reimplementation!

⁵ Supposing that Sonic Rumble Party somehow wasn’t a catastrophic pay-to-win nightmare and somehow was deemed culturally-significant… how would you go about archiving it? Without Sega/Sonic Team’s consent, you’d be totally out of luck.

Wood-Fired

This week I’m at Three Rings‘ annual “3Camp” event. Owing to Some Plot, we had a gap in the cooking rota, and, seeing that there was a pizza oven in the back garden, I figured… I can make a couple of dozen pizzas to feed everyone, right?

Dan, a white man with a ponytail of blue hair and a goatee beard, uses his hands to gather a huge pile of flour on a marble worksurface in a spacious kitchen. — Step one, as previously-indicated, was to make a *lot* of dough.

There was no mixing bowl large enough to accommodate the 4.5kg of flour so I just dumped it onto a surface, added some salt and sugar, made a well in the middle, and introduced my oil, water and rehydrated yeast right into the middle of it.

Minus a few minor spills, it broadly worked as a technique.

A small wood fire burns inside an outdoor brick pizza oven. — We weren’t able to find the woodpile at the house we’re staying at, so I eventually had to seek a volunteer to go ~~and forage~~ to B&Q to buy a couple of sacks of wood. I can’t wait to hear our treasurer’s response to this unusual expenses claim!

After an initial rise I knocked-back the dough and separated it into balls, and got started on building the fire.

I own a small, portable Ooni pizza oven that’s fired by woodchips, and I find it pretty challenging to use. It eats fuel pretty quickly and loses heat through its thin walls just as fast, and so it’s hard to maintain a consistent temperature while simultaneously maintaining the supply of wood and cooking pizza.

This brick-built oven, though, was a different kind of beast.

The same brick pizza oven, now seen from a few steps back with its chimney and base visible. — Compared to my small metal oven, this brick oven took a *lot* longer – on the region of an hour – to get up to temperature… but once hot, it maintained the heat much better.

I set up a prep station nearby and had Three Rings volunteers “build their own” pizzas: stretching or rolling the dough, adding sauce and cheese and other toppings, etc. And then I rotated them through the oven, up to two at a time.

My arms were already tired from the workout of hand-kneading the enormous pile of dough, and it was hot and tiring work to keep making, moving, and turning pizzas… but it was also… amazingly fun.

As the pizzas started to come out, Three Rings volunteers did too, gathering around the fire pit and in the covered dining areas of the garden, glasses in hand, to enjoy freshly-baked hot slices of crispy pizza, while they talked about volunteering, history, the future, and a diversity of other random topics beside (space travel, politics, music, teaching…).

Awesome.

So yeah… now I really want to build a brick pizza oven of my very own.

Obviously I’ve got other priorities right now (like having somewhere to live following the house-wrecking flood), but maybe that’s something I could look at in a future year.

A crispy, misshapen, slightly charred pepperoni and mushroom pizza on a paper plate. — The first pizza out of the oven was probably the ugliest, but it was also the one I remembered to photograph.

3Camp remains an annual tradition that I love dearly: the camaraderie, the doing-good-in-the-world, the opportunity to work alongside so many kind and talented volunteers, the chance to play with exciting technology, and whole experience… but the pizzas on the penultimate evening have got to go down as a special highlight this year.