Water Science #2

Back in 2019, the kids – so much younger back then! – and I helped undertake some crowdsourced citizen science for the Thames WaterBlitz. This year, we’re helping out again.

Screenshot from FreshWater Watch's slightly-shonky dataviewer, showing that 3 participants took a sample from Oxford Canal bridge 228 on 21 September 2019.
It really is “open data”. Look: I found the record that was created as a result of the kids’ and my participation back in 2019!

We’ve moved house since then, but we’re still within the Thames basin and can provide value by taking part in this weekend’s sampling activity. The data that gets collected on nitrate and phosphate levels in local water sources –  among other observations – gets fed into an open dataset for the benefit of scientists and laypeople.

Two young children assess the colour of some canal water, beneath a bridge.
The kids were smaller last time we did this.

It’d have been tempting to be exceptionally lazy and measure the intermittent water course that runs through our garden! It’s an old, partially-culverted drainage ditch1, but it’s already reached the “dry” part of its year and taking a sample wouldn’t be possible right now.

Two children wearing wellies stand in a ditch, breaking ice into chunks.
The ditch in our garden is empty 75% and full of water 25% of the time. Oh, and full of ice for a few days each winter, to the delight of children who love smashing things. (It’s also full of fallen wood and leaf detritus most of the year and JTA spends a surprising amount of time dredging it so that it drains properly into its next section.)

But more-importantly: the focus of this season’s study is the River Evenlode, and we’re not in its drainage basin! So we packed up a picnic and took an outing to the North Leigh Roman Villa, which I first visited last year when I was supposed to be on the Isle of Man with Ruth.

Dan kneels on a striped picnic mat with a 7-year-old and a 10-year old child alongside some sandwiches, iced fingers, Pepperami, fruit, and pretzels.
“Kids, we’re going outside…” / “Awww! Noooo!” / “…for a picnic and some science!” / “Yayyy!”

Our lunch consumed, we set off for the riverbank, and discovered that the field between us and the river was more than a little waterlogged. One of the two children had been savvy enough to put her wellies on when we suggested, but the other (who claims his wellies have holes in, or don’t fit, or some other moderately-implausible excuse for not wearing them) was in trainers and Ruth and I needed to do a careful balancing act, holding his hands, to get him across some of the tougher and boggier bits.

A 7-year-old boy in a grey camo coat balances on a blank over a large muddy puddle: he's about to attempt to cross a log to a gate into the next field (which also looks pretty wet). Ruth, who doesn't much like featuring in photos, has been digitally-removed from this one (she was standing at the far side ready to catch the balancing child!).
Trainers might not have been the optimal choice of footwear for this particular adventure.

Eventually we reached the river, near where the Cotswold Line crosses it for the fifth time on its way out of Oxford. There, almost-underneath the viaduct, we sent the wellie-wearing eldest child into the river to draw us out a sample of water for testing.

Map showing the border between Gloucestershire and Oxfordshire as defined by the original path of the River Evenlode near Kingham, but the Evenlode has been redirected as part of the construction of the railway, putting two small bits of Gloucestershire on the "wrong side" of the river.
As far as Moreton-in-Marsh, the Cotswold Line out of Oxford essentially follows the River Evenlode. In some places, such as this one near Kingham, the river was redirected to facilitate the construction of the railway. Given that the historic Gloucestershire/Oxfordshire boundary was at this point defined by the river, it’s not clear whether this represents the annexation of two territories of Gloucestershire by Oxfordshire. I doubt that anybody cares except map nerds.

Looking into our bucket, we were pleased to discover that it was, relatively-speaking, teeming with life: small insects and a little fish-like thing wriggled around in our water sample2. This, along with the moorhen we disturbed3 as we tramped into the reeds, suggested that the river is at least in some level of good-health at this point in its course.

A 10-year old girl wearing sunglasses and purple wellies holds her skirt up out of the water as she wades up the muddy bank of a river carrying a tub of water.
I’m sure our eldest would have volunteered to be the one to traipse through the mud and into the river even if she hadn’t been the only one of that was wearing wellies.

We were interested to observe that while the phosphate levels in the river were very high, the nitrate levels are much lower than they were recorded near this spot in a previous year. Previous years’ studies of the Evenlode have mostly taken place later in the year – around July – so we wondered if phosphate-containing agricultural runoff is a bigger problem later in the Spring. Hopefully our data will help researchers answer exactly that kind of question.

Children stand around at a riverside stile while a colour-changing chemical in a vial does its thing.
The chemical experiments take up to 5 minutes each to develop before you can read their colours, so the kids had plenty of time to write-up their visual observations while they waited.

Regardless of the value of the data we collected, it was a delightful excuse for a walk, a picnic, and to learn a little about the health of a local river. On the way back to the car, I showed the kids how to identify wild garlic, which is fully in bloom in the woods nearby, and they spent the rest of the journey back chomping down on wild garlic leaves.

A 7-year-old wearing his coat inside-out and as a cape runs excitedly into a forest path overgrown with wild garlic.
Seriously, that’s a lot of wild garlic.

The car now smells of wild garlic. So I guess we get a smelly souvenir from this trip, too4!

Footnotes

1 Our garden ditch, long with a network of similar channels around our village, feeds into Limb Brook. After a meandering journey around the farms to the East this eventually merges with Chill Brook to become Wharf Stream. Wharf Stream passes through a delightful nature reserve before feeding into the Thames near Swinford Toll Bridge.

2 Needless to say, we were careful not to include these little animals in our chemical experiments but let them wait in the bucket for a few minutes and then be returned to their homes.

3 We didn’t catch the moorhen in a bucket, though, just to be clear.

4 Not counting the smelly souvenir that was our muddy boots after splodging our way through a waterlogged field, twice

× × × × × × × × ×

Automattic Switch

Earlier this year, I changed team at work1.

Sankey diagram showing team Alpha splitting into Fire and Fuel; later Desire forms from parts of Fire and Fuel; later still Experiments forks from Fire and the remainder of Fire eventually become part of Desire. Team Masamune appears part-way along the timeline and runs parallel and independent to it. Dan's trajectory starts in Alpha, becomes part of Fire, and is eventually in the portion that merges into Desire.
This simplified diagram shows my journey so-far (dotted white line) through Automattic.

When I started at Automattic in 2019 I was assigned to Team Alpha, the engineering team responsible for WooCommerce.com. In 2020 the team later forked into teams Fire and Fuel, and I landed on the Fire side of the split2.

This winter, though, Fire became underpopulated. We lost a few folks to a newly-formed Experiments team, and several individual team members moved to other parts of the company. Once we got small enough it wasn’t worthwhile being a team in our own right. Our focus areas got split between Desire and Masume, and those of us who were left got absorbed into Team Desire. That’s where you’ll now find me.

Intranet identity card for Dan Q (he/him/his), Code Magician on Desire.
I miss being “on Fire”, because it sounded cool. Maybe I should suggest a patch for our intranet to allow teams to choose the preposition used when referring to their members, e.g. from “on”, “in”, “of” etc. Then I could be the “Code Magician of Desire”, which is a cool job title once again.

I was initially a bit bummed about the dissolution of my old team3 and struggled to find my place in my new team. The work is similar and the codebases overlap, but even sibling teams can have different rituals and approaches to problems that provide a learning barrier4.

I think I’ve begun to find my feet now, and next week I’m excited to meet many of my new team in-person for the first time at a Desire-wide meetup in Amsterdam5.

Footnotes

1 Strangely, this isn’t directly related to Automattic’s recent re-organisation, which I’ve written about previously, but is a result of more-local changes within my division coupled with the natural flow of Automatticians around the company. But it does make it feel from my perspective like a lot of things are getting jiggled about simultaneously!

2 When Alpha were first discussing the upcoming split, I suggested that we might like to give our new teams a “pair” of names that linked to one another, and threw out a few ideas to get the ball rolling. One of those ideas was “Fuel and Fire”; I jokingly added that “it was like the Metallica song, which also gave us ‘Desire’ as a possible third team name should the need arise”. This wasn’t supposed to be taken seriously, but apparently it was taken seriously enough because my suggestion was the winner and I soon ended up “on Fire”.

3 Many of my old teammates and I did at least manage to get together for one final (virtual) social event, culminating in a symbolic “extinguishing of fire” as a candle that had been left burning through the meeting was put out at the end.

4 A team’s rituals aren’t just about the way they hold their meetings or run their retros; for example my new team are very disciplined about announcing their appearance on a morning with a friendly greeting in our social channel, which are for some reason generally responded-to with a barrage of “waving Pikachu” slackmoji. I don’t know why Pikachu is the mascot of our mornings, but I’ve joined in because it’s a fun gesture of the team’s distinct collective personality. Also it’s a cute GIF: it’s nice to get waved-at by Pikachu on a morning.

5 Doubly-awesome, the destination’s proximity means that I get to travel by Eurostar rather than having to fly.

× ×

Council Disenfranchisement

Like much of the UK, there are local elections where I live next month. After coming home from a week of Three Rings volunteering I found my poll card on the doormat. Can you spot the bleeding-obvious mistake?

Poll card for West Oxfordshire District Council (and other) local elections on 2 May 2024 addressed to "Dan Que".
Also interesting was that this year the poll card came in a tamper-evidence tear-to-open envelope rather than just being a piece of card. Does the government now think that postal workers are routinely stealing voter identities? Cohabitees?

This’ll be the first election for which I’ve needed to bring photographic ID to the polling station. That shouldn’t be a problem: I have a passport and driving license and whatnot.

But just to be absolutely certain, I had the local council – the same people who issued me the polling card! – supply me with a voter authority certificate:

Voter Authority Certificate for "Dan Q".
Note that this document, also issued by West Oxfordshire District Council, spells my name correctly.

So now I’m in a pickle. West Oxfordshire District Council are asking me to produce photo ID in the wrong name when I turn up at a polling station next month. It doesn’t even match the name on the photo ID that they themselves issued me.

This would be less-infuriating were it not for the fact that they had my name wrong in the same way on an electoral roll form they sent me in August 20221. When I contacted them to have them fix it, they promised that the underlying problem was solved2 so this very thing wouldn’t happen.

And yet here we are.

Sign on the fence of a school playground reading, in block capitals: "Polling station for Dan Q's imaginary friends only".

Hopefully they’ll be able to fix their records promptly or else I guess I’ll have to apply for a proxy vote, to allow the ballot of my imaginary friend “Dan Que” to be cast by me, Dan Q, instead.

And if that isn’t the most bizarre form of election fraud you’ve ever heard of, I don’t know what is.

Update: True to their word, the council had managed to correct their records by the time I reached the polling station this morning. It’s still a little annoying that they somehow mucked it up in the first place, but I appreciate the efficiency with which they corrected their mistake.

Footnotes

1 They’d had my name right before August 2022, including on previous poll cards; I can only assume that some human operator “corrected” it to the wrong thing at some point.

2 They didn’t fix the problem immediately in August 2022. Initially, they demanded that I produce proof of my change of name from “Dan Que” (which has, of course, never been my name!) to “Dan Q”, and only later backed down and admitted that they’d made a mistake and would correct the PII they were holding about me.

× × ×

Reversing Sandwiches

Fixing Sandwiches

RotatingSandwiches.com is a website showcasing animated GIF files of rotating sandwiches1. But it’s got a problem: 2 of the 51 sandwiches rotate the wrong way. So I’ve fixed it:

The Eggplant Parm Sub is one of two sandwiches whose rotation doesn’t match the rest.

My fix is available as a userscript on GreasyFork, so you can use your favourite userscript manager2 to install it and the rotation will be fixed for you too. Here’s the code (it’s pretty simple):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    // ==UserScript==
    // @name        Standardise sandwich rotation on rotatingsandwiches.com
    // @namespace   rotatingsandwiches.com.danq.me
    // @match       https://rotatingsandwiches.com/*
    // @grant       GM_addStyle
    // @version     1.0
    // @author      Dan Q <https://danq.me/>
    // @license     The Unlicense / Public Domain
    // @description Some sandwiches on rotatingsandwiches.com rotate in the opposite direction to the majority. 😡 Let's fix that.
    // ==/UserScript==
     
    GM_addStyle('.q23-image-216, .q23-image-217 { transform: scaleX(-1); }');

Unless you’re especially agitated by irregular sandwich rotation, this is perhaps the most-pointless userscript ever created. So why did I go to the trouble?

Fixing Websites

Obviously, I’m telling you this as a vehicle to talk about userscripts in general and why you should be using them.

Userscripts fix and improve websites for you. Whether you want to force Reddit to stay “old Reddit” as long as possible,  make Geocaching.com’s maps more-powerful, show Twitter images uncropped by default, re-add the “cached” link to Google search results, show prices on shopping sites in terms of hours-of-your-life of work they “cost”, or just automatically click Netflix’s “yes, I’m still here, keep playing” button for maximum binge-mode, there’s a script for you. They’re like tiny browser plugins.

Screenshot from Pinterest showing many kittens, not logged-in.
Want to get endless-scroll in Pinterest without getting nagged to make a Pinterest account? There’s a userscript for that.

But the real magic is being able to remix the web your way. With just a little bit of CSS or JavaScript experience you can stop complaining that a website’s design has changed in some way you don’t like or that some functionality you use isn’t as powerful or convenient as you’d like and you can fix it.

A website I used disables scrolling until all their (tracking, advertising, etc.) JavaScript loads, and my privacy blocker blocks those files: I could cave and disable my browser’s privacy tools… but it was almost as fast to add setInterval(()=>document.body.style.overflow='', 200); to a userscript and now it’s fixed.

Don’t want a Sports section on your BBC News homepage (not just the RSS feed!)? document.querySelector('a[href="/sport"]').closest('main > div').remove(). Sorted.

I’m a huge fan of building your own tools to “scratch your own itch”. Userscripts are a highly accessible introduction to doing so that even beginner programmers can get on board with and start getting value from. More-advanced scripts can do immensely clever and powerful things, but even if you just use them to apply a few light CSS touches to your favourite websites, that’s still a win.

Footnotes

1 Remember when a website’s domain name used to be connected to what it was for? RotatingSandwiches.com does.

2 I favour ViolentMonkey.

×

My Geo*ing Limits

I thought it might be fun to try to map the limits of my geocaching/geohashing. That is, to draw the smallest possible convex polygon that surrounds all of the geocaches I’ve found and geohashpoints I’ve successfully visited.

World map showing the outer extent of the areas in which Dan has geocached/geohashed. A shaded polygon covers the UK (except the far North of Scotland), parts of California, Cape Town, and parts of Italy and Austria.

Mathematically, such a shape is a convex hull – the smallest polygon encircling a set of points without concavity. Here’s how I made it:

1. Extract all the longitude/latitude pairs for every successful geocaching find and geohashpoint expedition. I keep them in my blog database, so I was able to use some SQL to fetch them:

SELECT DISTINCT coord_lon.meta_value lon, coord_lat.meta_value lat
FROM wp_posts
LEFT JOIN wp_postmeta expedition_result ON wp_posts.ID = expedition_result.post_id AND expedition_result.meta_key = 'checkin_type'
LEFT JOIN wp_postmeta coord_lat ON wp_posts.ID = coord_lat.post_id AND coord_lat.meta_key = 'checkin_latitude'
LEFT JOIN wp_postmeta coord_lon ON wp_posts.ID = coord_lon.post_id AND coord_lon.meta_key = 'checkin_longitude'
LEFT JOIN wp_term_relationships ON wp_posts.ID = wp_term_relationships.object_id
LEFT JOIN wp_term_taxonomy ON wp_term_relationships.term_taxonomy_id = wp_term_taxonomy.term_taxonomy_id
LEFT JOIN wp_terms ON wp_term_taxonomy.term_id = wp_terms.term_id
WHERE wp_posts.post_type = 'post' AND wp_posts.post_status = 'publish'
AND wp_term_taxonomy.taxonomy = 'kind'
AND wp_terms.slug = 'checkin'
AND expedition_result.meta_value IN ('Found it', 'found', 'coordinates reached', 'Attended');

2. Next, I determine the convex hull of these points. There are an interesting variety of algorithms for this so I adapted the Monotone Chain approach (there are convenient implementations in many languages). The algorithm seems pretty efficient, although that doesn’t matter much to me because I’m caching the results for a fortnight.

Animation showing an algorithm draw lines from point to point, selecting each point by avoiding counter-clockwise turns.
I watched way too many animations of different convex hull algorithms before selecting this one… pretty-much arbitrarily.

3. Finally, I push the hull coordinates into Geoapify, who provide mapping services to me. My full source code is available.

An up-to-date (well, no-more than two weeks outdated) version of the map appears on my geo* stats page. I don’t often get to go caching/hashing outside the bounds already-depicted, but I’m excited to try to find opportunities to push the boundaries outwards as I continue to explore the world!

(I could, I suppose, try to draw a second larger area of places I’ve visited: the difference between the smaller and larger areas would represent all of the opportunities I’d missed to find a hashpoint!)

× ×

April Features!

I’m testing a handful of highly-experimental new features on my personal website using multivariate (“A/B”) testing.

Screenshot of the recent "Quickly Solving JigsawExplorer Puzzles" blog post with a new "Dark mode" switch hovering over it.
“Dark Mode” is just one of the new features I’m testing out.

If you visit within the next day or so you’re likely to be randomly-selected to try out one of them. (If you’re not selected, you can manually enable one of the experiments.)

I’d love to hear your feedback on these Very Serious New Features! Let me know which one(s) you see and whether you think they should become permanent fixtures on my site.

Quickly Solving JigsawExplorer Puzzles

Background

I was contacted this week by a geocacher called Dominik who, like me, loves geocaching…. but hates it when the coordinates for a cache are hidden behind a virtual jigsaw puzzle.

A popular online jigsaw tool used by lazy geocache owners is Jigidi: I’ve come up with several techniques for bypassing their puzzles or at least making them easier.

Dominik had been looking at a geocache hidden last week in Eastern France and had discovered that it used JigsawExplorer, not Jigidi, to conceal the coordinates. Let’s take a look…

Unsolved approx. 1000 piece jigsaw puzzle.
Not just any puzzle; the geocache used an ~1000 piece puzzle! Ugh!

I experimented with a few ways to work-around the jigsaw, e.g. dramatically increasing the “snap range” so dragging a piece any distance would result in it jumping to a neighbour, and extracting original image URLs from localStorage. All were good, but none were perfect.

For a while, making pieces “snap” at any range seemed to be the best hacky workaround.

Then I realised that – unlike Jigidi, where there can be a congratulatory “completion message” (with e.g. geocache coordinates in) – in JigsawExplorer the prize is seeing the completed jigsaw.

Dialog box reading "This puzzle's box top preview is disabled for added challenge."
You can click a button to see the “box” of a jigsaw, but this can be disabled by the image uploader.

Let’s work on attacking that bit of functionality. After all: if we can bypass the “added challenge” we’ll be able to see the finished jigsaw and, therefore, the geocache coordinates. Like this:

Hackaround

Here’s how it’s done. Or keep reading if you just want to follow the instructions!
  1. Open a jigsaw and try the “box cover” button at the top. If you get the message “This puzzle’s box top preview is disabled for added challenge.”, carry on.
  2. Open your browser’s debug tools (F12) and navigate to the Sources tab.
  3. Find the jigex-prog.js file. Right-click and select Override Content (or Add Script Override).
  4. In the overridden version of the file, search for the string – e&&e.customMystery?tt.msgbox("This puzzle's box top preview is disabled for added challenge."): – this code checks if the puzzle has the “custom mystery” setting switched on and if so shows the message, otherwise (after the :) shows the box cover.
  5. Carefully delete that entire string. It’ll probably appear twice.
  6. Reload the page. Now the “box cover” button will work.

The moral, as always, might be: don’t put functionality into the client-side JavaScript if you don’t want the user to be able to bypass it.

Or maybe the moral is: if you’re going to make a puzzle geocache, put some work in and do something clever, original, and ideally with fieldwork rather than yet another low-effort “upload a picture and choose the highest number of jigsaw pieces to cut it into from the dropdown”.

× ×

Oxford’s Area Code at OGN 57

Ever wondered why Oxford’s area code is 01865? The story is more-complicated than you’d think.

Modern desk telephone with an 01865 number (masked) and a keypad with letters on the buttons, 2=ABC through 9=WXYZ, with superimposed question marks between the two.
As a child, I was told that city STD codes were usually associated to the letters that appear on some telephones… but that wouldn’t make any sense for Oxford’s code!

I’ll share the story on my blog, of course. But before then, I’ll be telling it from the stage of the Jericho Tavern at 21:15 on Wednesday 17 April as my third(?) appearance at Oxford Geek Nights! So if you’re interested in learning about some of the quirks of UK telephone numbering history, I can guarantee that this party’s the only one to be at that Wednesday night!

Not your jam? That’s okay: there’s plenty of more-talented people than I who’ll be speaking, about subjects as diverse as quantum computing with QATboxen, bringing your D&D experience to stakeholder management (!), video games without screens, learnings from the Horizon scandal, and whatever Freyja Domville means by The Unreasonable Effectiveness of the Scientific Method (but I’m seriously excited by that title).

Oxford Geek Night logo

Anyway: I hope you’ll be coming along to Oxford Geek Nights 57 next month, if not to hear me witter on about the fossils in our telecommunications networks then to enjoy a beer and hear from the amazing speakers I’ll be sharing the stage with. The event’s always a blast, and I’m looking forward to seeing you there!

×

Home Page, 50% Off!

In August, I celebrated my blog – with its homepage weighing-in at a total of just 481kbbeing admitted to Kev Quirk‘s 512kb club. 512kb club celebrates websites (often personal sites) whose homepage are neither “ultra minimal” or “link pages” but have a total size, including all assets, of under half a megabyte. It’s about making a commitment to a leaner, more-efficient Web.

My relatively-heavyweight homepage only just slipped in under the line. But, feeling inspired perhaps by some performance enhancements I’ve been planning this week at work, I decided to try to shave a little more off:

Now, at ~234kb, danq.me just beats the excellent gomakethings.com (it’s all those heavyweight fonts, Chris!).

Here’s what I changed:

  1. The “recent article” tiles are dynamically sized based on their number, type, and the visitor’s screen resolution. But apart from the top one they’re almost never very large. Using thumbnail images for the non-first tile shaved off almost 160kb.
Illustration showing how the smaller tiles only need thumbnail images.
You can see the difference, but it’s still acceptable to look at, I think.
  1. Not space-saving, but while I was in there I ensured that the first tile’s image – which almost-certainly comprises part of the Largest Contentful Paint – is never delivered with loading="lazy".
  2. I was providing a shortcut icon in .ico format (<link rel="shortcut icon" href="/_q23t/icons/favicon-16-32-48-64-128.ico" />), which is pretty redundant nowadays because all modern browsers (and even IE11) support .png icons. I was already providing .png and .svg versions, but it turns out that some browsers favour the one with the (harmful?) rel="shortcut icon" over rel="icon" if both are present, and .ico files are – being based on Windows Bitmaps – horrendously inefficient.

By getting under the 250kb threshold, I’ve jumped up a league from Blue Team to Orange Team, so that’s nice too. I can’t see a meaningful path from where I’m at to Green Team (under 100kb) though, so this level might have to suffice.

Last-minute additions:

×

My Favourite Video Game Easter Egg

This post is also available as a video. If you'd prefer to watch/listen to me talk about this topic, give it a look.

Spoiler: it’s Alorik’s crystal ball in Ultima Underworld II.

This blog post is also available as a vlog: why not watch & listen to me as I demonstrate my favourite video game Easter egg!

Video preview showing Dan overlaid on a screenshot of Ultima Underworld II.

Ultima Underworld II

My favourite video game Easter egg is found in Ultima Underworld II: Labyrinth of Worlds1.

Released early in 1993 after missing a target of Christmas 19922, it undersold despite being almost universally well-received by reviewers3.

Composite of PC Zone's review of Ultima Underworld II, overlaid with their final score: "94%: Just go out now and buy it, okay."
I don’t know how print magazine video game reviews read today, but back in the early 90s they seemed to be frivolous and flippant approximately 100% of the time.

Developed by Looking Glass Technologies, it used an enhanced version of the engine they’d used for the game’s prequel a year earlier4.

The engine is particularly cool for it’s time; it’s sometimes compared to Wolfenstein5, but that’s not entirely fair… on Wolfenstein! The original version of Underworld‘s 3D engine predated Wolfenstein… and yet supported several features that Wolfenstein lacked, like the ability for the player to look up and down and jump over chasms, for example.

Screenshots of the same piles of bones and detritus in Ultima Underworld I, seen from two different angles.
Move around and you’ll see the 3D walls shift to your new perspective, but you’ll always see the same side of the 2D objects on the floor. Turn as you might, you’ll never see the back of that skull!

The team’s expertise and code would eventually be used to produce System Shock in 1994. The team’s producer, Warren Spector, would eventually draw from his experience of the Ultima Underworld games when he went on to make Thief: The Dark Project and Deus Ex.

But the technology of Ultima Underworld II and its prequel aren’t as interesting as its approach to storytelling and gameplay. They’re:

  • real-time
  • first-person
  • non-linear
  • fantasy roleplaying games

This was a highly innovative combination6.

Highlighted part of the Designers Notes, reading: "We feel that one of Underworld’s strongest features is its capacity to generate situations and strategies that we did not script, but that arose from the scenarios we created. For example: Suppose there’s a Despoiler daemon guarding the key to that door. Maybe I should come back when I can take him hand-to-hand … or bring up my picklock skill … or throw this sling stone at him, see if he’ll chase me … or maybe I can sneak past him — how good is a Despoiler’s eyesight? And can he swim …"
Underworld II encourages lateral thinking and provides multiple solutions to many challenges, perhaps best-described by this section in the Designers’ Notes page at the back of the manual.

What’s being described there is what we’d now call emergent gameplay, and while it wasn’t completely new in 19937 it was still uncommon enough to be noteworthy.

The Easter Egg

The Ultima series are riddled with Easter eggs, but my favourite is one that I feel is well-hidden, beautiful… and heavily laden with both fan service and foreshadowing!

Screenshot showing a frozen fountain in an icy realm.
The third of the interdimensional planes the Avatar visits in the game contains a city frozen solid by the Guardian’s magic.

To find the Easter egg, you must first travel to Anodunos. This city was once the capital of a tropical city-state which had become allied to the Guardian, the the principal antagonist of Ultima VII through IX.

Box art showing a hero dressed in leather armour wielding a staff with a glowing tip in a frozen cave.
The original box art for Underworld II, painted by Denis Loubet (who’ll draw your RPG character for you if you like!) showed the Avatar in the ice cavern, wielding an important late-game artefact.

After the city’s major, Beatrice, attempted to put an end to the red titan’s growing demands, the Guardian cursed the city fountain to radiate out a magical cold that eventually froze the entire settlement under a cave of ice.

Screenshot of the conversation interface as Dan talks to a ghost called Beatrice.
Mayor Beatrice lives on here, as a ghost. While she’s a great source of information about the ice caverns as a whole and has information highly useful to the primary quest, that’s not the thing we’re interested in right now.

On the Eastern bank of the city’s river we find the remnants of the workshop of the magician Alorik, and in it – if we look in the right place8 – a secret door. We can’t open it though: unusually for a secret door in this game, it’s locked.

Looking at a blank patch of wall in a frozen room, the player is told "You see a secret door."
In the original, unpatched release of the game, you couldn’t even cast the “open” spell on it to unlock it: the only way through was to use a powerful late-game spell like “portal” to teleport to the other side.

I didn’t even find this chamber on my first playthrough of the game. It was only on my second, while using the Map Area spell to help me to draw accurate maps of the entire game world, that I found the room… and even then I spent some time hunting for a switch on the “outside” before eventually giving up and teleporting into the secret room.

An open secret door (the one from before, but viewed from the other side), and a thrown switch.
Once you’re on the other side, there’s a switch. I guess it’s a Alorik’s panic room?

There’s valuable treasure here including a sceptre of mana restoration, a “grav” runestone (probably still easier to get than the one at the Scintillus Academy), but what’s most interesting is the crystal ball, which the player can look into to see a vision of another place and, in the case of this orb, another time.

The first time you look into it, you’re told:

You see yourself striving against the forces of Mondain and Minax in the lands of ancient Sosaria.9

Mondain and Minax are the antagonists in Ultima I and Ultima II. We’re seeing the earliest parts of the player character’s adventures.

In the game: looking into the crystal ball.
It’s long been argued that the “Stranger” – the protagonist of Ultima I through Ultima III – is a different person to the Avatar of Ultima IV but I feel like this vision is canonical proof that they’re supposed to be interpreted as being the same person…

If we look into the crystal ball a second time:

You see yourself climbing to the peak of Olympus Mons on the planet Mars.

This is a reference to the plot of Ultima: Worlds of Adventure 2: Martian Dreams… which is a… weird choice of game to reference.

In my mind, a more logical leap forward in time might have been to jump to Ultima IV10, in which the protagonist first becomes the Avatar of the Eight Virtues and the Hero of Britannia. Martian Dreams is… a sequel to a spinoff of Ultima VI. So why pick that?

Screenshot from Martian Dreams showing Dr. Spector.
Little-known and oft-forgotten, Martian Dreams is apparently nonetheless a source of pride for the real Spector.

Martian Dreams starts with a friend of the Avatar’s from Earth facilitating the Avatar and their companions to set out on an adventure to the planet Mars. That friend is called Dr. Spector, obviously named for Warren Spector, who helped develop Ultima VI and, of course, this game. This usual choice of vision of the past is a cryptic nod to the producer of Underworld II.

Let’s look again:

You see yourself in the Deep Forest, speaking with the peace-loving simian race of Emps.

This one’s a reference to Ultima VII, the game whose story immediately precedes this one. The Deep Forest seems an strange part of the adventure to choose, though. The Avatar goes to the Deep Forest where, via some emps and then a wisp are eventually lead to the Time Lord11. The Time Lord provides a whole heap of exposition and clues that the Avatar needs to eventually close the Black Gate and win the game.

Ultima VII: the Avatar converses with the Time Lord in the Shrine of Spirituality. He says "Thou must free me and we must work together in battling the Guardian. The fate of thy people depends upon it. Dost thou accept?"
Tardis noises intensify.

Do these references serve to hint that this crystal ball, too, is a source of exposition and guidance? Let’s see what it says next.

You see yourself peering into a crystal ball.

I remember the moment I first saw this happen in the game: serious chills! You’ve just found a long-lost, centuries-buried secret chamber, in which there’s a crystal ball. You peer into it and observe a series of moments from throughout your life. You continue to watch, and eventually you see yourself, staring into the crystal ball: you’re seeing the present. So what’s next?

Frame from 1984's The NeverEnding Story, showing Atreyu seeing the prophetic painting that shows the beast he's about to meet.
Alorik’s crystal ball from Ultima Underworld II and the mural of G’mork from The NeverEnding Story totally have the same energy.

If you look again, you’re asking to see… the unwritten future:

You see yourself winding a great war horn in the throne room of Castle British.

To save Britannia in Ultima Underworld II, the Avatar needs to exploit symmetries implicit in The Guardian’s spellcasting to travel to eight different parallel worlds, find a place from which His power stems, dispel it, encase themselves in a shell of basilisk oil-infused magic mud, immerse themselves in lava to bake it on, find a magic sigil, consume a djinn… it’s a whole thing. But ultimately it all leads to a climactic end scene in which the Avatar raises a horn retrieved from the Tomb of Praecor Loth and blows it to shatter a dome of blackrock.

Under a starry sky, a blackrock dome grows green cracks and shatters.
Blackrock’s really heavy, right? I’m not sure that shattering it above the building you’re inside is wise.

If you happen to find this clue on your first playthrough, it’s helpful exposition.

But that’s the end of this game, right? How can we possibly peer into the orb again?

You see yourself sailing through majestic pillars cropping up out of the sea, on a voyage of discovery.

What’s being described there is the opening scene from the next game in the series, the as-yet-unreleased Ultima VII Part 2: Serpent Isle!

Ultima VII Part 2 introduction, showing a boat sailing through an icy sea corridor towards two serpent pillars.
Serpent Isle begins with the Avatar and their companions sailing out to some special pillars in pursuit of a bad guy.

This vision is a teaser of what’s to come. That’s just… magical, for both the character and the player.

The character uses fortune-telling magic to see their future, but the player is also seeing their future: if they’re playing Ultima Underworld II at or close to its release date, or they’re playing through the games in chronological order, they’re in a literal sense being shown what comes next in their life. That’s really cool.12

Let’s look again:

You see the obscure form of an old and dear friend, as he sacrifices his life for the good of all.

Some time after the party arrives on Serpent Isle, the Avatar’s companions are possessed by the Banes of Chaos and go on a murderous rampage. Later, there’s a ritual that will save the world, but at the cost of the death of one of the heroes. The Avatar is willing to make the ultimate sacrifice, but in the end Sir Dupre takes his place, unwilling to live within himself after seeing the carnage he has wrought.

Sir Dupre says "I cannot live with the shame of the deaths I have caused... If I can redeem myself by becoming the required martyr, so be it!"
The death of Sir Dupre, who first appeared way back in Ultima II, is somewhat undercut by his resurrection in Ultima IX. But perhaps you’ll let me pretend for a paragraph longer that Ultima IX doesn’t exist, okay?

At the end of Serpent Isle, the Avatar is plucked out of space and time and deposited into Pagan, The Guardian’s home base. The plot of Ultima VIII and Ultima IX revolve around the Avatar working to return to a radically-changed Britannia, attempting to fight The Guardian and bring to an end the Age of Armageddon, and ultimately merging and become one with Him before vanishing completely from the world.

An ankh symbol picked out in the constellations.
I don’t disagree with how Ultima IX got to where it was going, but I don’t like the route it took to get there. Or the hilariously-terrible bugs.

Which is why it’s perhaps quite fitting that if the Avatar in Underworld II looks into the orb one final time, they’re told…

You see nothing.

That’s it. That’s the end.

The end of the vision, certainly, but also: a vision of the end.

Depending on how you count the Ultima games13, this is the 13th of 17 in the series. We’re approaching the final chapter, and this Easter egg foreshadows that finale.

I feel hugely privileged that I got to experience it “organically”, by accident, as its authors presumably intended, back in 1993. But it also makes me happy to be able to share the story of it with you14.

If you haven’t seen it yet, you might enjoy watching the vlog version of this post, through which my enthusiasm for the topic might be more-palpable.

Footnotes

1 I’ve doubtless mentioned Ultima Underworld II before: for example both it and Ultima VII, as well as NetHack (mentioned elsewhere in this post) made it into my 2007 list of top 10 computer games that stole my life.

2 The release was delayed owing to testing revealing just too-many bugs, the penultimate of which was squashed on 18 December leaving just one more that the team couldn’t reproduce until the New Year

3 It suffered perhaps for the time of year it was released, but perhaps also for the fact that 1993 was a big year for video games and it was competing with The 7th Guest, Star Wars: Rebel Assault, Return to Zork , Myst, Disney’s Aladdin and, of course – later in the year – Doom.

4 Director/designer Paul Neurath apparently sang the praises of his team for improving texture mapping and viewport size constraints, and he’s right: they’re a huge improvement on Underworld I‘s. Neurath would later go on create the crowdfunded “spiritual successor” Underworld Ascendant, which was critically panned, which just goes to show that sometimes it’s better to get a tight team together and make it “until it’s done” than to put your half-baked idea on Kickstarter and hope you can work it out what you’re making before the money runs out.

5 Like Wolfenstein, the engine uses a mixture of software-rendered 3D (for walls and furniture) overlaid with traditionally-produced sprites (for characters and items).

6 All executed over a year before the release of the very first Elder Scrolls game. Just sayin’.

7 That king of emergent gameplay NetHack was showcasing emergent gameplay in a fantasy roleplaying game way back in the 1980s!

8 An interesting quirk of the game was that if you turned the graphics settings down to their lowest, secret doors would become just as visible as regular doors. If you’re sure there is one but you can’t quite find it, tweaking your graphics settings is much easier than casting a spell!

9 Do you like the “in the style of Underworld II” scrolls I’ve used in this post? I’ve made available the source code you need if you want to use them yourself.

10 Ultima IV is my personal favourite Ultima game, but I see the argument of people who claim that Ultima VII is the best of the series.

11 The Time Lord turns up throughout the game series. Way back in Ultima III, he appears in the Dungeon of Time where he provides a clue essential to defeating Exodus, and he appears or is referenced in most games from Ultima VII onwards. He doesn’t seem to appear in Ultima IV through Ultima VI, except… in Ultima IX, which wouldn’t be released until six years after Underworld II, it’s revealed that the Time Lord is the true identity of the seer Hawkwind… who provided the same kind of exposition and guidance in Ultima IV!

12 How did the Underworld II team know with such certainty what was being planned for Serpent Isle? At some point in 1992 project director Jeff George left Origin Studios and was replaced by lead designer Bill Armintrout, and the role of producer was assigned to… Warren Spector again! For some time, Spector was involved with both projects, providing an easy conduit for inter-team leaks.

13 How you count Ultima games and what specifically should be counted is a source of controversy in fan circles.

14 I’m sure many people reading this will have heard me talk about this particular Easter egg in-person before, over the last couple of decades. Some of you might even have heard me threaten to write a blog post about it, someday. Well: now I have. Tada! It only took me thirty years after experiencing it to write about it here, which is still faster than some things I’ve blogged about!

× × × × × × × × × × × × × × × ×

The Stupidest CSS

Yesterday, I wrote the stupidest bit of CSS of my entire career.

Screenshot of the WooCommerce Extensions Marketplace, with two sponsored products annotated to show (a) a pretty stripe at the top, (b) adding the word "Sponsored", and (c) a strange green blob alongside each.
Two new visual elements and one textual one will make it clear where a product’s placement in the marketplace is sponsored.

Owners of online shops powered by WooCommerce can optionally “connect” their stores back to Woo.com. This enables them to manage their subscriptions to any extensions they use to enhance their store1. They can also browse a marketplace of additional extensions they might like to consider, which is somewhat-tailored to them based on e.g. their geographical location2

In the future, we’ll be adding sponsored products to the marketplace listing, but we want to be transparent about it so yesterday I was working on some code that would determine from the appropriate API whether an extension was sponsored and then style it differently to make this clear. I took a look at the proposal from the designer attached to the project, which called for

  1. the word “Sponsored” to appear alongside the name of the extension’s developer,
  2. a stripe at the top in the brand colour of the extension, and
  3. a strange green blob alongside it

That third thing seemed like an odd choice, but I figured that probably I just didn’t have the design or marketing expertise to understand it, and I diligently wrote some appropriate code.3

I even attached to my PR a video demonstrating how my code reviewers could test it without spoofing actual sponsored extensions.

After some minor tweaks, my change was approved. The designer even swung by and gave it a thumbs-up. All I needed to do was wait for the automated end-to-end tests to complete, and I’d be able to add it to WooCommerce ready to be included in the next-but-one release. Nice.

In the meantime, I got started on my next bit of work. This one also included some design work by the same designer, and wouldn’t you know it… this one also had a little green blob on it?

A design for a completely different part of the WooCommerce Extensions Management tool, again with a little green blob. Dan's floating head looks thoughtful above it, with a thought-bubble reading "wow, we're really going all-in on these little blobs, aren't we?"
I’m almost embarrassed to admit that my first thought was that this must be part of some wider design strategy to put little green blobs everywhere.

Then it hit me. The blobs weren’t part of the design at all, but the designer’s way of saying “look at this bit, it’s important!”. Whoops!

So I got to rush over to my (already-approved, somehow!) changeset and rip out the offending CSS: the stupidest bit of CSS of my entire career. Not bad code per se, but reasonable code resulting from a damn-stupid misinterpretation of a designer’s wishes. Brilliant.

Footnotes

1 WooCommerce extensions serve loads of different purposes, like handling bookings and reservations and integrating with parcel tracking services.

2 There’s no point us suggesting an extension that helps you calculate Royal Mail shipping rates if you’re not shipping from the UK, for example!

3 A fun side-effect of working on open-source software is that my silly mistake gets immortalised somewhere where you can go and see it any time you like!

× ×

Zap

This week, I received a ~240V AC electric shock. I can’t recommend it.

A 10-year-old girl hangs from a scaffolding pole outside a white house.
As you may have guessed based on photos in previous posts, our house is currently wrapped in a convenient climbing frame scaffolding.

We’re currently having our attic converted, so we’ve had some electricians in doing the necessary electrical wiring. Shortly after they first arrived they discovered that our existing electrics were pretty catastrophic, and needed to make a few changes including a new fusebox and disconnecting the hilariously-unsafe distribution board in the garage.

Lifted floor showing central heating pipes and a tangle of electrical cables.
The owner before last of our house worked for SSEN and did all of his own wiring, and left us a rats’ nest of spaghetti wiring that our electricians described as being unlike anything they’d ever seen before. Also a literal rats’ nest under the decking, but we got rid of that already.

After connecting everything new up they began switching everything back on and testing the circuits… and we were surprised to hear arcing sounds and see all the lights flickering.

The electricians switched everything off and started switching breakers back on one at a time to try to identify the source of the fault, reasonably assuming that something was shorting somewhere, but no matter what combination of switches were enabled there always seemed to be some kind of problem.

Electricity shining a torch into a cupboard containing an fusebox with an open cover; the switches are in a mixture of on and off positions.
You know those escape room puzzles where you have to get the right permutation of switch combinations? This was a lot less fun than that.

Noticing that the oven’s clock wasn’t just blinking 00:00 (as it would after a power cut) but repeatedly resetting itself to 00:00, I pointed this out to the electricians as an indicator that the problem was occurring on their current permutation of switches, which was strange because it was completely different to the permutation that had originally exhibited flickering lights.

I reached over to point at the oven, and the tip of my finger touched the metal of its case…

Blam! I felt a jolt through my hand and up my arm and uncontrollably leapt backwards across the room, convulsing as I fell to the floor. I gestured to the cooker and shouted something about it being live, and the electricians switched off its circuit and came running with those clever EM-field sensor pens they use.

Somehow the case of the cooker was energised despite being isolated at the fusebox? How could that be?

Dan, bare-chested, lies in a hospital bed with an ECG hooked-up to him.
Buy one ECG appointment. Get a free partial chest-shaving free!

I missed the next bits of the diagnosis of our electrical system because I was busy getting my own diagnosis: it turns out that if you get a mains electric shock – even if you’re conscious and mobile – the NHS really want you to go to A&E.

At my suggestion, Ruth delivered me to the Minor Injuries unit at our nearest hospital (I figured that what I had wasn’t that serious, and the local hospital generally has shorter wait times!)… who took one look at me and told me that I ought to be at the emergency department of the bigger hospital over the way.

ECG printout showing a report of "Abnormal ECG: sinus arrhythmia".
The first hospital were kind enough to hook me up to an ECG before sending me on to the A&E department. It indicated possible cardiac arrhythmia in the sinus node – basically: my heart’s natural pacemaker was firing somewhat irregularly – which is kinda what you’d expect from an AC zap.

Off at the “right” hospital I got another round of ECG tests, some blood tests (which can apparently be used to diagnose muscular damage: who knew?), and all the regular observations of pulse and blood pressure and whatnot that you might expect.

And then, because let’s face it I was probably in better condition than most folks being dropped off at A&E, I was left to chill in a short stay ward while the doctors waited for test results to come through.

Two electricians, one in a hard hat, look in an outdoor metering cupboard by torchlight.
Apparently our electricity meter blew itself up somewhere along the way, leaving us with even less of a chance to turn the power back on again.

Meanwhile, back at home our electricians had called-in SSEN, who look after the grid in our area. It turns out that the problem wasn’t directly related to our electrical work at all but had occurred one or two pylons “upstream” from our house. A fault on the network had, from the sounds of things, resulted in “live” being sent down not only the live wire but up the earth wire too.

That’s why appliances in the house were energised even with their circuit breakers switched-off: they were connected to an earth that was doing pretty-much the opposite of what an earth should: discharging into the house!

Linesmen examining an electricity pylon by torchlight.
For the next day or so, a parade of linesmen climbed up and down all the pylons in the field behind our house, hunting for the source of the problem.

It seems an inconceivable coincidence to me that a network fault might happen to occur during the downtime during which we happened to have electricians working, so I find myself wondering if perhaps the network fault had occurred some time ago but only become apparent/dangerous as a result of changes to our household configuration.

I’m no expert, but I sketched a diagram showing how such a thing might happen (click to embiggen). I’ll stress that I don’t know for certain what went wrong: I’m just basing this on what I’ve been told my SSEN plus a little speculation:

MS Paint grade illustration showing how a faulty transformer on a power pole, crossing live with earth, might go unnoticed as a result of the earth spike behind our garage until the garage's (unsafe) circuit is disconnected.

By the time I was home from the hospital the following day, our driveway was overflowing with the vehicles of grid engineers to the point of partially blocking the main street outside (which at least helped ensure that people obeyed our new 20mph limit for a change).

A driveway full of vehicles spills out onto the nearby road.
We weren’t even able to get our own car onto our driveway when we got back from the hospital.

Two and a half days later, I’m back at work and mostly recovered. I’ve still got some discomfort in my left hand, especially if I try to grip anything tightly, but I’m definitely moving in the right direction.

It’s actually more-annoying how much my chest itches from having various patches of hair shaved-off to make it possible to hook up ECG electrodes!

A linesman hangs from his climbing belt at the top of a pole, while two others look on from the ground.
The actual conversation at this point seemed to consist of the guy at the top of the pole confirming that yes, he really had disconnected the live wire from our house, and one at the bottom saying he can’t have because he’s still seeing electricity flowing. Makes sense now, doesn’t it?

Anyway, the short of it is that I recommend against getting zapped by the grid. If it had given me superpowers it might have been a different story, but I guess it just gave me sore muscles and a house with a dozen non-working sockets.

× × × × × × × × × ×

Yours Quim-cerely

A lovely letter from the Vagina Museum – which I’ve not had the opportunity to visit yet – came through my letterbox:

Printed letter on Vagina Museum headed paper, reading: Dear Dan Q, I'm assuming you donated £50 on 1st February in response to my desperate plea for help on Mastodon. I'm pleased to say this was printed on our brand spanking new Brother Laser. The entire Vagina Museum thanks you for your generosity in our hour of need. Yours quim-cerely, Zoe Williams; Head of Communications and Fundraising
“Yours quim-cerely,” doesn’t appear in any style guide but is now the best sign-off in any letter I’ve ever received.

This moment of joy was kick-started when I casually dropped in on a conversation about printer recommendations. I’ve got a big ol’ Brother printer here, and it’s great, not least because even though it’s got a tonne of features like duplexing and (double-sided) scanning and photocopying and it’s even got a fax machine built in for some reason… it doesn’t try to be any more “smart” than it needs to be. It doesn’t talk to Alexa or order itself more toner (it even gets-by with knockoff toner!) or try to do anything well… except print things, which it does wonderfully.

For this and other reasons I recommended they buy a Brother.

Then, alongside some other Fediversians, I chipped in to help them buy one.

Totally worth it for the letter alone. Now I just need to find an excuse to visit an exhibition!

×

BBC News… without the crap

Did I mention recently that I love RSS? That it brings me great joy? That I start and finish almost every day in my feed reader? Probably.

I used to have a single minor niggle with the BBC News RSS feed: that it included sports news, which I didn’t care about. So I wrote a script that downloaded it, stripped sports news, and re-exported the feed for me to subscribe to. Magic.

RSS reader showing duplicate copies of the news story "Barbie 2? 'We'd love to,' says Warner Bros boss", and an entry from BBC Sounds.
Lately my BBC News feed has caused me some annoyance and frustration.

But lately – presumably as a result of technical changes at the Beeb’s side – this feed has found two fresh ways to annoy me:

  1. The feed now re-publishes a story if it gets re-promoted to the front pagebut with a different <guid> (it appears to get a #0 after it when first published, a #1 the second time, and so on). In a typical day the feed reader might scoop up new stories about once an hour, any by the time I get to reading them the same exact story might appear in my reader multiple times. Ugh.
  2. They’ve started adding iPlayer and BBC Sounds content to the BBC News feed. I don’t follow BBC News in my feed reader because I want to watch or listen to things. If you do, that’s fine, but I don’t, and I’d rather filter this content out.

Luckily, I already have a recipe for improving this feed, thanks to my prior work. Let’s look at my newly-revised script (also available on GitHub):

#!/usr/bin/env ruby
require 'bundler/inline'

# # Sample crontab:
# # At 41 minutes past each hour, run the script and log the results
# */20 * * * * ~/bbc-news-rss-filter-sport-out.rb > ~/bbc-news-rss-filter-sport-out.log 2>>&1

# Dependencies:
# * open-uri - load remote URL content easily
# * nokogiri - parse/filter XML
gemfile do
  source 'https://rubygems.org'
  gem 'nokogiri'
end
require 'open-uri'

# Regular expression describing the GUIDs to reject from the resulting RSS feed
# We want to drop everything from the "sport" section of the website, also any iPlayer/Sounds links
REJECT_GUIDS_MATCHING = /^https:\/\/www\.bbc\.co\.uk\/(sport|iplayer|sounds)\//

# Load and filter the original RSS
rss = Nokogiri::XML(open('https://feeds.bbci.co.uk/news/rss.xml?edition=uk'))
rss.css('item').select{|item| item.css('guid').text =~ REJECT_GUIDS_MATCHING }.each(&:unlink)

# Strip the anchors off the <guid>s: BBC News "republishes" stories by using guids with #0, #1, #2 etc, which results in duplicates in feed readers
rss.css('guid').each{|g|g.content=g.content.gsub(/#.*$/,'')}

File.open( '/www/bbc-news-no-sport.xml', 'w' ){ |f| f.puts(rss.to_s) }
It’s amazing what you can do with Nokogiri and a half dozen lines of Ruby.

That revised script removes from the feed anything whose <guid> suggests it’s sports news or from BBC Sounds or iPlayer, and also strips any “anchor” part of the <guid> before re-exporting the feed. Much better. (Strictly speaking, this can result in a technically-invalid feed by introducing duplicates, but your feed reader oughta be smart enough to compensate for and ignore that: mine certainly is!)

You’re free to take and adapt the script to your own needs, or – if you don’t mind being tied to my opinions about what should be in BBC News’ RSS feed – just subscribe to my copy at: https://fox.q-t-a.uk/bbc-news-no-sport.xml

×

RSS > ActivityPub

RSS is better than ActivityPub1.

Photograph of a boxing match, but with the heads of the competitors replaced with the ActivityPub and RSS logos (and "AP" or "RSS" written on their clothes, respectively). RSS is delivering a powerful uppercut to ActivityPub.
A devastating blow by RSS against a competitor 19 years his junior! For updates on this bout as it develops, don’t forget to subscribe… using either protocol.

When I subscribe to content, I want:

  • Resilient failsafes. ActivityPub has many points-of-failure. A notification might fail to complete transmission as a result of downtime, faults, or network conditions, and the receiving server might never know. A feed reader, conversely, can tell you that an address 404’d or the server was down.
  • Retroactive access. Once you fix the problem above… you still don’t get the message you missed: it’s probably gone forever – there’s no retroactive access. The same is true when your ActivityPub server connects with a peer for the first time: you only ever get new content after that point. RSS, on the other hand, provides some number of “recent” items the moment you first subscribe.
  • Simple subscriptions. RSS can be served from a statically-hosted single file, which makes it suitable to deploy anywhere as well as consume using anything. It can be read, after a fashion, in anything from Lynx upwards.

RSS ticks all these boxes. If I can choose between RSS and ActivityPub to subscribe to your content, and I don’t need a real-time update, I’m probably going to choose RSS.

About a month later, Matthias Pfefferle wrote a great post that makes a good “next stop” if you’re on a deep dive…

Footnotes

1 I feel like this statement needs a few clarifications and caveats, but my hot take looks spicier if I bury them in a footnote!

  • By RSS, I mean whichever pull-based basic HTTP you like, be that Atom, JSON Feed, h-entry, or even just properly-marked-up HTML5: did you know that the <article> element is intended to be suitable for syndication use?
  • Obviously I appreciate that RSS and ActivityPub are different tools for different jobs, and there are doubtless use-cases for which ActivityPub is clearly the superior solution.
  • I certainly don’t object to services providing both RSS and ActivityPub as syndication options, like Mastodon does, where both might be good choices.
×