Somewhat-Effective Spam Filters

I’ve tried a variety of unusual strategies to combat email spam over the years.

Here are some of them (each rated in terms the geekiness of its implementation and its efficacy), in case you’d like to try any yourself. They’re all still in use in some form or another:

Spam filters

Geekiness: 1/10
Efficacy: 5/10

A colander filters spam email out of a stream of emails.

Your email provider or your email software probably provides some spam filters, and they’re probably pretty good. I use Proton‘s and, when I’m at my desk, Thunderbird‘s. Double-bagging your spam filter only slightly reduces the amount of spam that gets through, but increases your false-positive rate and some non-spam gets mis-filed.

A particular problem is people who email me for help after changing their name on FreeDeedPoll.org.uk, probably because they’re not only “new” unsolicited contacts to me but because by definition many of them have strange and unusual names (which is why they’re emailing me for help in the first place).

Frankly, spam filters are probably enough for many people. Spam filtering is in general much better today than it was a decade or two ago. But skim the other suggestions in case they’re of interest to you.

Unique email addresses

Geekiness: 3/10
Efficacy: 8/10

If you give a different email address to every service you deal with, then if one of them misuses it (starts spamming you, sells your data, gets hacked, whatever), you can just block that one address. All the addresses come to the same inbox, for your convenience. Using a catch-all means that you can come up with addresses on-the-fly: you can even fill a paper form with a unique email address associated with the company whose form it is.

On many email providers, including the ever-popular GMail, you can do this using plus-sign notation. But if you want to take your unique addresses to the next level and you have your own domain name (which you should), then you can simply redirect all email addresses on that domain to the same inbox. If Bob’s Building Supplies wants your email address, give them bobs@yourname.com, which works even if Bob’s website erroneously doesn’t accept email addresses with plus signs in them.

This method actually works for catching people misusing your details. On one occasion, I helped a band identify that their mailing list had been hacked. On another, I caught a dodgy entrepreneur who used the email address I gave to one of his businesses without my consent to send marketing information of a different one of his businesses. As a bonus, you can set up your filtering/tagging/whatever based on the incoming address, rather than the sender, for the most accurate finding, prioritisation, and blocking.

Emails to multiple email addresses reach the same inbox. Spam emails are blocked based on the addresses they're sent to.

Also, it makes it easy to have multiple accounts with any of those services that try to use the uniqueness of email addresses to prevent you from doing so. That’s great if, like me, you want to be in each of three different Facebook groups but don’t want to give Facebook any information (not even that you exist at the intersection of those groups).

Signed unique email addresses

Geekiness: 10/10
Efficacy: 2/10

Unique email addresses introduce two new issues: (1) if an attacker discovers that your Dreamwidth account has the email address dreamwidth@yourname.com, they can probably guess your LinkedIn email, and (2) attackers will shotgun “likely” addresses at your domain anyway, e.g. admin@yourname.com, management@yourname.com, etc., which can mean that when something gets through you get a dozen copies of it before your spam filter sits up and takes notice.

What if you could assign unique email addresses to companies but append a signature to each that verified that it was legitimate? I came up with a way to do this and implemented it as a spam filter, and made a mobile-friendly webapp to help generate the necessary signatures. Here’s what it looked like:

  1. The domain directs all emails at that domain to the same inbox.
  2. If the email address is on a pre-established list of valid addresses, that’s fine.
  3. Otherwise, the email address must match the form of:
    • A string (the company name), followed by
    • A hyphen, followed by
    • A hash generated using the mechanism described below, then
    • The @-sign and domain name as usual

The hashing algorithm is as follows: concatenate a secret password that only you know with a colon then the “company name” string, run it through SHA1, and truncate to the first eight characters. So if my password were swordfish1 and I were generating a password for Facebook, I’d go:

  1. SHA1 ( swordfish1 : facebook) [ 0 ... 8 ] = 977046ce
  2. Therefore, the email address is facebook-977046ce@myname.com
  3. If any character of that email address is modified, it becomes invalid, preventing an attacker from deriving your other email addresses from a single point (and making it hard to derive them given multiple points)

I implemented the code, but it soon became apparent that this was overkill and I was targeting the wrong behaviours. It was a fun exercise, but ultimately pointless. This is the one method on this page that I don’t still use.

Honeypots

Geekiness: 8/10
Efficacy: ?/10

Emails to multiple email addresses reach an inbox, but senders who reach a "honeypot" inbox are blocked from reaching the real inbox.

A honeypot is a “trap” email address. Anybody who emails it get aggressively marked as a spammer to help ensure that any other messages they send – even to valid email addresses – also get marked as spam.

I litter honeypots all over the place (you might find hidden email addresses on my web pages, along with text telling humans not to use them), but my biggest source of honeypots is formerly-valid unique addresses, or “guessed” catch-all addresses, which already attract spam or are otherwise compromised!

I couldn’t tell you how effective it is without looking at my spam filter’s logs, and since the most-effective of my filters is now outsourced to Proton, I don’t have easy access to that. But it certainly feels very satisfying on the occasions that I get to add a new address to the honeypot list.

Instant throwaways

Geekiness: 5/10
Efficacy: 6/10

OpenTrashmail is an excellent throwaway email server that you can deploy in seconds with Docker, point some MX records at, and be all set! A throwaway email server gives you an infinite number of unique email addresses, like other solutions described above, but with the benefit that you never have to see what gets sent to them.

Emails are delivered to an inbox and to a trash can, depending on the address they're sent to. The inbox subscribes to the trash can using RSS.

If you offer me a coupon in exchange for my email address, it’s a throwaway email address I’ll give you. I’ll make one up on the spot with one of my (several) trashmail domains at the end of it, like justgivemethedamncoupon@danstrashmailserver.com. I can just type that email address into OpenTrashmail to see what you sent me, but then I’ll never check it again so you can spam it to your heart’s content.

As a bonus, OpenTrashmail provides RSS feeds of inboxes, so I can subscribe to any email-based service using my feed reader, and then unsubscribe just as easily (without even having to tell the owner).

Summary

With the exception of whatever filters your provider or software comes with, most of these options aren’t suitable for regular folks. But you’re only a domain name (assuming you don’t have one already) away from being able to give unique email addresses to everybody you deal with, and that’s genuinely a game-changer all by itself and well worth considering, in my opinion.

× × × ×

Framing Device

Doors

As our house rennovations/attic conversions come to a close, I found myself up in what will soon become my en suite, fitting a mirror, towel rail, and other accessories.

Wanting to minimise how much my power tool usage disturbed the rest of the house, I went to close the door separating my new bedroom from my rest of my house, only to find that it didn’t properly fit its frame and instead jammed part-way-closed.

“Oh,” I said, as the door clearly failed to shut, “Damn.”

Somehow we’d never tested that this door closed properly before we paid the final instalment to the fitters. And while I’m sure they’d have come back to repair the problem if I asked, I figured that it’d be faster and more-satisfying to fix it for myself.

Homes

As a result of an extension – constructed long before we moved in – the house in Preston in which spent much of my childhood had not just a front and a back door but what we called the “side door”, which connected the kitchen to the driveway.

Unfortunately the door that was installed as the “side door” was really designed for interior use and it suffered for every winter it faced the biting wet North wind.

A partially-pebbledashed house.
The side door isn’t visible in this picture: it’s concealed behind the corner of the house, to the left of the car.

My father’s DIY skills could be rated as somewhere between mediocre and catastrophic, but his desire to not spend money “frivolously” was strong, and so he never repaired nor replaced the troublesome door. Over the course of each year the wood would invariably absorb more and more water and swell until it became stiff and hard to open and close.

The solution: every time my grandfather would visit us, each Christmas, my dad would have his dad take down the door, plane an eighth of an inch or so off the bottom, and re-hang it.

Sometimes, as a child, I’d help him do so.

A grey-haired white man wearing spectacles and a boiler suit leans comfortably on a railing alongside industrial machinery.
My paternal grandfather was a practical and hand-on engineer and a reasonable carpenter.

Planes

The first thing to do when repairing a badly-fitting door is work out exactly where it’s sticking. I borrowed a wax crayon from the kids’ art supplies, coloured the edge of the door, and opened and closed it a few times (as far as possible) to spot where the marks had smudged.

Fortunately my new bedroom door was only sticking along the top edge, so I could get by without unmounting it so long as I could brace it in place. I lugged a heavy fence post rammer from the garage and used it to brace the door in place, then climbed a stepladder to comfortably reach the top.

A small box plane perched atop a sloping door.
I figured I’d only need to remove a few millimetres, so I didn’t mind doing it from atop a stepladder. Hey: here’s a fun thing – when I think about planing a door with my grandfather, I think in inches; when I think about doing it myself, I think in metric!

Loss

After my paternal grandfather died, there was nobody left who would attend to the side door of our house. Each year, it became a little stiffer, until one day it wouldn’t open at all.

Surely this would be the point at which he’d pry open his wallet and pay for it to be replaced?

A middle-aged man carrying walking poles on an urban riverbank drags a car tyre that's chained to his waist.
I’m not sure there’s a more apt metaphor for my dad’s ability to be stubborn than this photo of him dragging a tyre around Gateshead as a training activity for an Arctic expedition.

Nope. Instead, he inexpertly screwed a skirting board to it and declared that it was now no-longer a door, but a wall.

I suppose from a functionalist perspective he was correct, but it still takes a special level of boldness to simply say “That door? It’s a wall now.”

Sand

Of all the important tasks a carpenter (or in this case, DIY-er) must undertake, hand sanding must surely be the least-satisfying.

Dan rubs sandpaper atop a wooden door.
You wear your fingers out rubbing a piece of wood smooth, and your only reward is getting to do it again with a slightly finer grade of paper.

But reaching the end of the process, the feel of a freshly-planed, carefully-sanded piece of wood is fantastic. This surface represented chaos, and now it represents order. Order that you yourself have brought about.

Often, you’ll be the only one to know. When my grandfather would plane and sand the bottom edge of our house’s side door, he’d give it a treatment of oil (in a doomed-to-fail attempt to keep the moisture out) and then hang it again. Nobody can see its underside once it’s hung, and so his handiwork was invisible to anybody who hadn’t spent the last couple of months swearing at the stiffness of the door.

A paintbrush applies white paint to the top of a door.
Swish, swish. Now I’m glad I sanded.

Even though the top of my door is visible – particularly visible, given its sloping face – nobody sees the result of the sanding because it’s hidden beneath a layer of paint.

A few brush strokes provide the final touch to a spot of DIY… that in provided a framing device for me to share a moment of nostalgia with you.

Sweep away the wood shavings. Keep the memories.

× × × × × ×

Don’t Ask, Don’t Teach

Politics and pundits

The UK’s Conservative government, having realised that their mandate is worthless, seems to be in a panicked rush to try to get the voters to ignore any of the real issues. Instead, they say, we should be focussed on things like ludicrously-expensive and ineffective ways to handle asylum seekers and making life as hard as possible for their second-favourite scapegoat: trans and queer people.

Screengrab from Last Week Tonight with John Oliver. John Oliver is subtitled as saying: In the end, Sunak did an end-run around the ruling that Rwanda was too dangerous by simply having his government officially declare Rwanda a "safe country".
By the time John Oliver’s doing a segment about you, perhaps it’s time to realise you’ve fucked up? But our main story tonight is about sex education…

The latest move in that second category seems likely to be a plan to, among other things, discourage teachers from talking about gender identity in schools, with children of any age. From the article I linked:

The BBC has not seen the new guidelines but a government source said they included plans to ban any children being taught about gender identity.

If asked, teachers will have to be clear gender ideology is contested.

Needless to say, such guidance is not likely to be well-received by teachers:

Pepe Di’Iasio, headteacher at a school in Rotherham, told Today that he believes pupils are being used “as a political football”.

Teachers “want well informed and evidence-based decisions”, he said, and not “politicised” guidance.

Cringey political poster reading "Is this Labour's idea of a comprehensive education? Take the politics out of education, vote Conservative", alongside three books: Young gay & proud, Police: Out of School!, and The playbook for kids about sex.
I can only assume that the Tories still have a stack of this genuine 1987 billboard poster (ugh) in stock, and are hoping to save money by reusing them.

People and pupils

This shit isn’t harmless. Regardless of how strongly these kinds of regulations are enforced, they can have a devastating chilling effect in schools.

I speak from experience.

A group of teenagers stand around awkwardly.
I don’t know if this is the “most-90s” photo I own of myself, but it’s gotta be close. Taken at the afterparty from a school production of South Pacific, so probably at least a little disproportionately-queer gathering.

Most of my school years were under the shadow of Section 28. Like I predict for the new Conservative proposals, Section 28 superficially didn’t appear to have a major impact: nobody was ever successfully prosecuted under it, for example. But examining its effects in that way completely overlooks the effect it had on how teachers felt they had to work.

For example…

In around 1994, I witnessed a teacher turn a blind eye to homophobic bullying of a pupil by their peer, during a sex education class. Simultaneously, the teacher coolly dismissed the slurs of the bully, saying that we weren’t “talking about that in this class” and that the boy should “save his chatter for the playground”. I didn’t know about the regulations at the time: only in hindsight could I see that this might have been a result of Section 28. All I got to see at the time was a child who felt that his homophobic harassment of his classmate had the tacit endorsement of the teachers, so long as it didn’t take place in the classroom.

A gay friend, who will have been present but not involved in the above event, struggled with self-identity and relationships throughout his teenage years, only “coming out” as an adult. I’m confident that he could have found a happier, healthier life had he felt supported – or at the very least not-unwelcome – at school. I firmly believe that the long-running third-degree side-effects of Section 28 effectively robbed him of a decade of self-actualisation about his identity.

The long tail of those 1980s rules were felt long-after they were repealed. And for a while, it felt like things were getting better. But increasingly it feels like we’re moving backwards.

A pride rainbow painted down the back of a white person's first, held in the air.
As a country and as a society, we can do better than this.

With general elections coming up later this year, it’ll soon be time to start quizzing your candidates on the issues that matter to you. Even (perhaps especially) if your favourite isn’t the one who wins, it can be easiest to get a politicians’ ear when they and their teams are canvassing for your vote; so be sure to ask pointed questions about the things you care about.

I hope that you’ll agree that not telling teachers to conceal from teenagers the diversity of human identity and experience is something worth caring about.

Update: Only a couple of hours after I posted this, the awesome folks (whom I’ve mentioned before) at the Vagina Museum tooted a thread about the long tail of Section 28. It’s well-worth a read.

× × × ×

A completely plaintext WordPress Theme

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

This is a silly idea. But it works. I saw Dan Q wondering about plaintext WordPress themes – so I made one.

This is what this blog looks like using it:

Screenshot showing my blog rendered just as text.

I clearly nerdsniped Terence at least a little when I asked whether a blog necessarily had to be HTML, because he went on to implement a WordPress theme that delivers content entirely in plain text.

Naturally, I’ve also shared his accomplishment on my own text/plain blog (which uses a much simpler CMS based on static files).

×

Does a blog have to be HTML?

Terence Eden wrote about his recent experience of IndieWebCamp Brighton, in which he mentioned that somebody – probably Jeremy Keith – had said, presumably to provoke discussion:

A blog post doesn’t need a title.

Terence disagrees, saying:

In a literal sense, he was wrong. The HTML specification makes it clear that the <title> element is mandatory. All documents have title.

But I think that’s an overreach. After all, where is it written that a blog must be presented in HTML?

Non-HTML blogs

There are plenty of counter-examples already in existence, of course:

But perhaps we can do better…

A totally text/plain blog

We’ve looked at plain text, which as a format clearly does not have to have a title. Let’s go one step further and implement it. What we’d need is:

  1. A webserver configured to deliver plain text files by preference, e.g. by adding directives like index index.txt; (for Nginx).5
  2. An index page listing posts by date and URL. Most browser won’t render these as “links” so users will have to copy-paste or re-type them, so let’s keep them short,
  3. Pages for each post at those URLs, presumably without any kind of “title” (just to prove a point), and
  4. An RSS feed: usually I use RSS as shorthand for all feed types, but this time I really do mean RSS and not e.g. Atom because RSS, strangely, doesn’t require that an <item> has a <title>!

I’ve implemented it! it’s at textplain.blog.

textplain.blog in Lynx
Unlike other sites, I didn’t need to test textplain.blog in Lynx to know it’d work well. But I did anyway.

In the end I decided it’d benefit from being automated as sort-of a basic flat-file CMS, so I wrote it in PHP. All requests are routed by the webserver to the program, which determines whether they’re a request for the homepage, the RSS feed, or a valid individual post, and responds accordingly.

It annoys me that feed discovery doesn’t work nicely when using a Link: header, at least not in any reader I tried. But apart from that, it seems pretty solid, despite its limitations. Is this, perhaps, an argument for my .well-known/feeds proposal?

Anyway, I’ve open-sourced the entire thing in case it’s of any use to anybody at all, which is admittedly unlikely! Here’s the code.

Footnotes

1 no-ht.ml technically does use HTML, but the same content could easily be delivered with an appropriate non-HTML MIME type if he’d wanted.

2 Again, I suppose this technically required HTML, even if what was delivered was an empty file!

3 Gemtext is basically Markdown, and doesn’t require a title.

4 Plain text obviously doesn’t require a title.

5 There’s no requirement that default files served by webservers are HTML, although it’s highly-unsual for that not to be the case.

Test your site in Lynx

When was the last time you tested your website in a text-only browser like Lynx (or ELinks, or one of several others)? Perhaps you should.

I’m a big fan of CSS Naked Day. I love the idea of JS Naked Day, although I missed it earlier this month (I was busy abroad, plus my aggressive caching, including in service workers, makes it hard to reliably make sweeping changes for short periods). I’m a big fan of the idea that, for the vast majority of websites, if it isn’t at least usable without any CSS or JavaScript, it should probably be considered broken.

This year, I thought I’d celebrate the events by testing DanQ.me in the most-limited browser I had to-hand: Lynx. Lynx has zero CSS or JavaScript support, along with limited-to-no support for heading levels, tables, images, etc. That may seem extreme, but it’s a reasonable analogue for the level of functionality you might routinely expect to see in the toughest environments in which your site is accessed: slow 2G connections from old mobile hardware, people on the other side of highly-restrictive firewalls or overenthusiastic privacy and security software, and of course users of accessibility technologies.

Here’s what broke (and some other observations):

<link rel="alternate">s at the top

I see the thinking that Lynx (and in an even more-extreme fashion, ELinks) have with showing “alternate versions” of a page at the top, but it’s not terribly helpful: most of mine are designed to help robots, not humans!

Screenshot showing four alternate links at the top of DanQ.me as viewed in Lynx.
Four alternates is pretty common for a WordPress site: post feed, comments feed, and two formats of oEmbed.

I wonder if switching from <link rel="alternate"> elements to Link: HTTP headers would indicate to Lynx that it shouldn’t be putting these URLs in humans’ faces, while still making them accessible to all the services that expect to find them? Doing so would require some changes to my caching logic, but might result in a cleaner, more human-readable HTML file as a side-effect. Possibly something worth investigating.

Fortunately, I ensure that my <link rel="alternate">s have a title attribute, which is respected by Lynx and ELinks and makes these scroll-past links slightly less-confusing.

Lynx screenshot from IKEA.com, showing no fewer than 113 anonymous "alternate" links at the top of the page.
Not all sites title their alternate links. IKEA.com requires you to scroll through 113 anonymous links for their alternate language versions, because Lynx doesn’t understand the hreflang attribute.

Post list indentation

Posts on the homepage are structured a little like this:

<li>
  <a href="...">
    <h2>Post Title</h2>
    <p>...post metadata, image, and things...</p>
  </a>
</li>

Strictly-speaking, that’s not valid. Heading elements are only permitted within flow elements. I chose to implement it that way because it seemed to be the most semantically-correct way to describe the literal “list of posts”. But probably my use of <h2> is not the best solution. Let’s see how Lynx handles it:

Screenshot from Lynx showing headings "outdented" from the list items they're children of.
Lynx “outdents” headings so they stand out, and “indents” lists so they look like lists. This causes a quirky clash where a heading is inside a list.

It’s not intolerable, but it’s a little ugly.

CSS lightboxes add a step to images

I use a zero-JavaScript approach to image lightboxes: you can see it by clicking on any of the images in this post! It works by creating a (closed) <dialog> at the bottom of the page, for each image. Each <dialog> has a unique id, and the inline image links to that anchor.

Originally, I used a CSS :target selector to detect when the link had been clicked and show the <dialog>. I’ve since changed this to a :has(:target) and directed the link to an element within the dialog, because it works better on browsers without CSS support.

It’s not perfect: in Lynx navigating on an inline image scrolls down to a list of images at the bottom of the page and selects the current one: hitting the link again now offers to download the image. I wonder if I might be better to use a JavaScript-powered lightbox after all!

gopher: and finger: links work perfectly!

I was pleased to discover that gopher: and finger: links to alternate copies of a post… worked perfectly! That shouldn’t be a surprise – Lynx natively supports these protocols.

Lynx screenshot showing DanQ.me via the Finger protocol.
I can’t conceive that many people access DanQ.me via the Finger protocol, but the option’s there.

In a fun quirk and unusually for a standard of its age, the Finger specification did not state the character encoding that ought to be used. I guess the authors just assumed everybody reading it would use ASCII. But both my WordPress-to-Finger bridge and Lynx instead assume that UTF-8 is acceptable (being a superset of ASCII, that seems fair!) which means that emoji work (as shown in the screenshot above). That’s nuts, isn’t it?

You can’t react to anything

Back in November I added the ability to “react” to a post by clicking an emoji, rather than typing out a full comment. Because I was feeling lazy, the feature was (and remains) experimental, and I didn’t consider it essential functionality, I implemented it mostly in JavaScript. Without JavaScript, all you can do is see what others have clicked.

Emoji reactions screenshot showing thumbs up, star eyes, surprise, muscle flex, nausea, and a dinosaur.
The available emoji vary from post to post; I sometimes like to throw a weird/fun one in there, knowing that it’ll invariably be Ruth that clicks it first.

In a browser with no JavaScript but with functional CSS, the buttons correctly appear disabled.

But with neither technology available, as in Lynx, they look like they should work, but just… don’t. Oops.

Screenshot from Lynx showing a "Bad HTML!! No form action defined." error when clicking an emoji reaction button.
Lynx is correct; this is sloppy code. Without CSS support, it even shows the instruction that implies the buttons will work, but they don’t.

If I decide to keep the reaction buttons long-term, I’ll probably reimplement them so that they function using plain-old HTML and HTTP, using a <form>, and refactor my JavaScript to properly progressively-enhance the buttons for those that support it. For now, this’ll do.

Comment form honeypot

The comment form on my blog posts works… but there’s a quirk:

At the end of the comments form, an additional <textarea> appears!

That’s an annoyance. It turns out it’s a honeypot added by Akismet: a fake comments field, normally hidden, that tries to trick spam bots into filling it (and thus giving themselves away): sort-of a “reverse CAPTCHA” where the robots do something extra, unintentionally, to prove their inhumanity. Lynx doesn’t understand the code that Akismet uses to hide the form, and so it’s visible to humans, which is suboptimal both because it’s confusing but also because a human who puts details into it is more-likely to be branded a spambot!

I might look into suppressing Akismet adding its honeypot field in the first place, or else consider one of the alternative anti-spam plugins for WordPress. I’ve heard good things about Antispam Bee; I ought to try it at some point.

Overall, it’s pretty good

On the whole, DanQ.me works reasonably well in browsers without any JavaScript or CSS capability, with only a few optional features failing to function fully. There’s always room for improvement, of course, and I’ve got a few things now to add to my “one day” to-do list for my little digital garden.

Obviously, this isn’t really about supporting people using text-mode browsers, who probably represent an incredible minority. It’s about making a real commitment to the semantic web, to accessibility, and to progressive enhancement! That making your site resilient, performant, and accessible also helps make it function in even the most-uncommon of browsers is just a bonus.

× × × × ×

Window Tax

Duration

Podcast Version

This post is also available as a podcast. Listen here, download for later, or subscribe wherever you consume podcasts.

…in England and Wales

From 1696 until 1851 a “window tax” was imposed in England and Wales1. Sort-of a precursor to property taxes like council tax today, it used an estimate of the value of a property as an indicator of the wealth of its occupants: counting the number of windows provided the mechanism for assessment.

Graph showing the burden of window tax in 1696 and 1794. In the former year a flat rate of 1 shiling was charged, doubling for a property when it reached 10 and 20 windows respectively. In the latter year charging began at 10 windows and the price per-window jumped up at 15 at 20 windows. Both approaches result in a "stepped" increase.
The hardest thing about retrospectively graphing the cost of window tax is thinking in “old money”2.
Window tax replaced an earlier hearth tax, following the ascension to the English throne of Mary II and William III of Orange. Hearth tax had come from a similar philosophy: that you can approximate the wealth of a household by some aspect of their home, in this case the number of stoves and fireplaces they had.

(A particular problem with window tax as enacted is that its “stepping”, which was designed to weigh particularly heavily on the rich with their large houses, was that it similarly weighed heavily on large multi-tenant buildings, whose landlord would pass on those disproportionate costs to their tenants!)

1703 woodcut showing King William III and Queen Mary II.
It’d be temping to blame William and Mary for the window tax, but the reality is more-complex and reflects late renaissance British attitudes to the limits of state authority.

Why a window tax? There’s two ways to answer that:

  • A window tax – and a hearth tax, for that matter – can be assessed without the necessity of the taxpayer to disclose their income. Income tax, nowadays the most-significant form of taxation in the UK, was long considered to be too much of an invasion upon personal privacy3.
  • But compared to a hearth tax, it can be validated from outside the property. Counting people in a property in an era before solid recordkeeping is hard. Counting hearths is easier… so long as you can get inside the property. Counting windows is easier still and can be done completely from the outside!
Dan points to a bricked-up first storey window on a stone building used by a funeral services company.
If you’re in Britain, finding older buildings with windows bricked-up to save on tax is pretty easy. I took a break from writing this post, walked for three minutes, and found one.4

…in the Netherlands

I recently got back from a trip to Amsterdam to meet my new work team and get to know them better.

Dan, by a game of table football, throws his arms into the air as if in self-celebration.
There were a few work-related/adjacent activities. But also a table football tournament, among other bits of fun.

One of the things I learned while on this trip was that the Netherlands, too, had a window tax for a time. But there’s an interesting difference.

The Dutch window tax was introduced during the French occupation, under Napoleon, in 1810 – already much later than its equivalent in England – and continued even after he was ousted and well into the late 19th century. And that leads to a really interesting social side-effect.

Dan, with four other men, sit in the back of a covered boat on a canal.
My brief interest in 19th century Dutch tax policy was piqued during my team’s boat tour.

Glass manufacturing technique evolved rapidly during the 19th century. At the start of the century, when England’s window tax law was in full swing, glass panes were typically made using the crown glass process: a bauble of glass would be spun until centrifugal force stretched it out into a wide disk, getting thinner towards its edge.

The very edge pieces of crown glass were cut into triangles for use in leaded glass, with any useless offcuts recycled; the next-innermost pieces were the thinnest and clearest, and fetched the highest price for use as windows. By the time you reached the centre you had a thick, often-swirly piece of glass that couldn’t be sold for a high price: you still sometimes find this kind among the leaded glass in particularly old pub windows5.

Multi-pane window with distinctive crown glass "circles".
They’re getting rarer, but I’ve lived in houses with small original panes of crown glass like these!

As the 19th century wore on, cylinder glass became the norm. This is produced by making an iron cylinder as a mould, blowing glass into it, and then carefully un-rolling the cylinder while the glass is still viscous to form a reasonably-even and flat sheet. Compared to spun glass, this approach makes it possible to make larger window panes. Also: it scales more-easily to industrialisation, reducing the cost of glass.

The Dutch window tax survived into the era of large plate glass, and this lead to an interesting phenomenon: rather than have lots of windows, which would be expensive, late-19th century buildings were constructed with windows that were as large as possible to maximise the ratio of the amount of light they let in to the amount of tax for which they were liable6.

Hotel des Pays-Bas, Nieuwe Doelenstraat 11 (1910 photo), showing large windows.
Look at the size of those windows! If you’re limited in how many you can have, but you’ve got the technology, you’re going to make them as large as you possibly can!

That’s an architectural trend you can still see in Amsterdam (and elsewhere in Holland) today. Even where buildings are renovated or newly-constructed, they tend – or are required by preservation orders – to mirror the buildings they neighbour, which influences architectural decisions.

Pre-WWI Neighbourhood gathering in Amsterdam, with enormous windows (especially on the ground floor) visible.
Notice how each building has only between one and three windows on the ground floor, letting as much light in while minimising the tax burden.

It’s really interesting to see the different architectural choices produced in two different cities as a side-effect of fundamentally the same economic choice, resulting from slightly different starting conditions in each (a half-century gap and a land shortage in one). While Britain got fewer windows, the Netherlands got bigger windows, and you can still see the effects today.

…and social status

But there’s another interesting this about this relatively-recent window tax, and that’s about how people broadcast their social status.

Modern photo, taken from the canal, showing a tall white building in Amsterdam with large windows on the ground floor and also basement level, and an ornamental window above the front door. Photo from Google Street View.
This Google Street Canal (?) View photo shows a house on Keizersgracht, one of the richest parts of Amsterdam. Note the superfluous decorative window above the front door and the basement-level windows for the servants’ quarters.

In some of the traditionally-wealthiest parts of Amsterdam, you’ll find houses with more windows than you’d expect. In the photo above, notice:

  • How the window density of the central white building is about twice that of the similar-width building on the left,
  • That a mostly-decorative window has been installed above the front door, adorned with a decorative leaded glass pattern, and
  • At the bottom of the building, below the front door (up the stairs), that a full set of windows has been provided even for the below-ground servants quarters!

When it was first constructed, this building may have been considered especially ostentatious. Its original owners deliberately requested that it be built in a way that would attract a higher tax bill than would generally have been considered necessary in the city, at the time. The house stood out as a status symbol, like shiny jewellery, fashionable clothes, or a classy car might today.

Cheerful white elderly man listening to music through headphones that are clearly too large for him.
I originally wanted to insert a picture here that represented how one might show status through fashion today. But then I remembered I don’t know anything about fashion7. But somehow my stock image search suggested this photo, and I love it so much I’m using it anyway. You’re welcome.
How did we go wrong? A century and a bit ago the super-wealthy used to demonstrate their status by showing off how much tax they can pay. Nowadays, they generally seem more-preoccupied with getting away with paying as little as possible, or none8.

Can we bring back 19th-century Dutch social status telegraphing, please?9

Footnotes

1 Following the Treaty of Union the window tax was also applied in Scotland, but Scotland’s a whole other legal beast that I’m going to quietly ignore for now because it doesn’t really have any bearing on this story.

2 The second-hardest thing about retrospectively graphing the cost of window tax is finding a reliable source for the rates. I used an archived copy of a guru site about Wolverhampton history.

3 Even relatively-recently, the argument that income tax might be repealed as incompatible with British values shows up in political debate. Towards the end of the 19th century, Prime Ministers Disraeli and Gladstone could be relied upon to agree with one another on almost nothing, but both men spoke at length about their desire to abolish income tax, even setting out plans to phase it out… before having to cancel those plans when some financial emergency showed up. Turns out it’s hard to get rid of.

4 There are, of course, other potential reasons for bricked-up windows – even aesthetic ones – but a bit of a giveaway is if the bricking-up reduces the number of original windows to 6, 9, 14 or 19, which are thesholds at which the savings gained by bricking-up are the greatest.

5 You’ve probably heard about how glass remains partially-liquid forever and how this explains why old windows are often thicker at the bottom. You’ve probably also already had it explained to you that this is complete bullshit. I only mention it here to preempt any discussion in the comments.

6 This is even more-pronounced in cities like Amsterdam where a width/frontage tax forced buildings to be as tall and narrow and as close to their neighbours as possible, further limiting opportunities for access to natural light.

7 Yet I’m willing to learn a surprising amount about Dutch tax law of the 19th century. Go figure.

8 Obligatory Pet Shop Boys video link. Can that be a thing please?

9 But definitely not 17th-century Dutch social status telegraphing, please. That shit was bonkers.

× × × × × × × × × ×

5 Cool Apps for your Unraid NAS

I’ve got a (now four-year-old) Unraid NAS called Fox and I’m a huge fan. I particularly love the fact that Unraid can work not only as a NAS, but also as a fully-fledged Docker appliance, enabling me to easily install and maintain all manner of applications.

A cube-shaped black computer sits next to a battery pack on a laminated floor. A sign has been left atop it, reading "Caution: Generator connected to this installation."
There isn’t really a generator attached to Fox, just a UPS battery backup. The sign was liberated from our shonky home electrical system.

I was chatting this week to a colleague who was considering getting a similar setup, and he seemed to be taking notes of things he might like to install, once he’s got one. So I figured I’d round up five of my favourite things to install on an Unraid NAS that:

  1. Don’t require any third-party accounts (low dependencies),
  2. Don’t need any kind of high-powered hardware (low specs), and
  3. Provide value with very little set up (low learning curve).
Dan, his finger to his lips and his laptop on his knees, makes a "shush" action. A coworker can be seen working behind him.
It’d have been cooler if I’d have secretly written this blog post while sitting alongside said colleague (shh!). But sadly it had to wait until I was home.

Here we go:

Syncthing

I’ve been raving about Syncthing for years. If I had an “everyday carry” list of applications, it’d be high on that list.

Syncthing screenshot for computer Rebel, sharing with Fox, Idiophone, Lemmy and Maxine.
Syncthing’s just an awesome piece of set-and-forget software that facilitates file synchronisation between all of your devices and can also form part of a backup strategy.

Here’s the skinny: you install Syncthing on several devices, then give each the identification key of another to pair them. Now you can add folders on each and “share” them with the others, and the two are kept in-sync. There’s lots of options for power users, but just as a starting point you can use this to:

  • Manage the photos on your phone and push copies to your desktop whenever you’re home (like your favourite cloud photo sync service, but selfhosted).
  • Keep your Obsidian notes in-sync between all your devices (normally costs $4/month).1
  • Get a copy of the documents from all your devices onto your NAS, for backup purposes (note that sync’ing alone, even with versioning enabled, is not a good backup: the idea is that you run an actual backup from your NAS!).

Huginn

You know IFTTT? Zapier? Services that help you to “automate” things based on inputs and outputs. Huginn’s like that, but selfhosted. Also: more-powerful.

Screenshot showing Huginn workflows.
When we first started looking for a dog to adopt (y’know, before we got this derper), I set up Huginn watchers to monitor the websites of several rescue centres, filter them by some of our criteria, and push the results to us in real-time on Slack, giving us an edge over other prospective puppy-parents.

The learning curve is steeper than anything else on this list, and I almost didn’t include it for that reason alone. But once you’ve learned your way around its idiosyncrasies and dipped your toe into the more-advanced Javascript-powered magic it can do, you really begin to unlock its potential.

It couples well with Home Assistant, if that’s your jam. But even without it, you can find yourself automating things you never expected to.

FreshRSS

I’ve written a lot about how and why FreshRSS continues to be my favourite RSS reader. But you know what’s even better than an awesome RSS reader? An awesome selfhosted RSS reader!

FreshRSS screenshot.
Yes, I know I have a lot of “unread” items. That’s fine, and I can tell you why.

Many of these suggested apps benefit well from you exposing them to the open Web rather than just running them on your LAN, and an RSS reader is probably the best example (you want to read your news feeds when you’re out and about, right?). What you need for that is a reverse proxy, and there are lots of guides to doing it super-easily, even if you’re not on a static IP address.2. Alternatively you can just VPN in to your home: your router might be able to arrange this, or else Unraid can do it for you!

Open Trashmail

You know how sometimes you need to give somebody your email address but you don’t actually want to. Like: sure, I’d like you to email me a verification code for this download, but I don’t trust you not to spam me later! What you need is a disposable email address.3

Open Trashmail screenshot showing a subscription to Thanks for subscribing to Dan Q's Spam-Of-The-Hour List!
How do you feel about having infinite email addresses that you can make up on-demand (without even having access to a computer), subscribe to by RSS, and never have to see unless you specifically want to.

You just need to install Open Trashmail, point the MX records of a few domain names or subdomains (you’ve got some spare domain names lying around, right? if not; they’re pretty cheap…) at it, and it will now accept email to any address on those domains. You can make up addresses off the top of your head, even away from an Internet connection when using a paper-based form, and they work. You can check them later if you want to… or ignore them forever.

Couple it with an RSS reader, or Huginn, or Slack, and you can get a notification or take some action when an email arrives!

  • Need to give that escape room your email address to get a copy of your “team photo”? Give them a throwaway, pick up the picture when you get home, and then forget you ever gave it to them.
  • Company give you a freebie on your birthday if you sign up their mailing list? Sign up 366 times with them and write a Huginn workflow that puts “today’s” promo code into your Obsidian notetaking app (Sync’d over Syncthing) but filters out everything else.
  • Suspect some organisation is selling your email address on to third parties? Give them a unique email address that you only give to them and catch them in a honeypot.

YOURLS

Finally: a URL shortener. The Internet’s got lots of them, but they’re all at the mercy of somebody else (potentially somebody in a country that might not be very-friendly with yours…).

YOURLS screenshot (Your Own URL Shortener).
It isn’t pretty, but… it doesn’t need to be! Nobody actually sees the admin interface except you anyway.

Plus, it’s just kinda cool to be able to brand your shortlinks with your own name, right? If you follow only one link from this post, let it be to watch this video that helps explain why this is important: danq.link/url-shortener-highlights.

I run many, many other Docker containers and virtual machines on my NAS. These five aren’t even the “top five” that I use… they’re just five that are great starters because they’re easy and pack a lot of joy into their learning curve.

And if your NAS can’t do all the above… consider Unraid for your next NAS!

Footnotes

1 I wrote the beginnings of this post on my phone while in the Channel Tunnel and then carried on using my desktop computer once I was home. Sync is magic.

2 I can’t share or recommend one reverse proxy guide in particular because I set my own up because I can configure Nginx in my sleep, but I did a quick search and found several that all look good so I imagine you can do the same. You don’t have to do it on day one, though!

3 Obviously there are lots of approachable to on-demand disposable email addresses, including the venerable “plus sign in a GMail address” trick, but Open Trashmail is just… better for many cases.

× × × × × × ×

Council Disenfranchisement

Like much of the UK, there are local elections where I live next month. After coming home from a week of Three Rings volunteering I found my poll card on the doormat. Can you spot the bleeding-obvious mistake?

Poll card for West Oxfordshire District Council (and other) local elections on 2 May 2024 addressed to "Dan Que".
Also interesting was that this year the poll card came in a tamper-evidence tear-to-open envelope rather than just being a piece of card. Does the government now think that postal workers are routinely stealing voter identities? Cohabitees?

This’ll be the first election for which I’ve needed to bring photographic ID to the polling station. That shouldn’t be a problem: I have a passport and driving license and whatnot.

But just to be absolutely certain, I had the local council – the same people who issued me the polling card! – supply me with a voter authority certificate:

Voter Authority Certificate for "Dan Q".
Note that this document, also issued by West Oxfordshire District Council, spells my name correctly.

So now I’m in a pickle. West Oxfordshire District Council are asking me to produce photo ID in the wrong name when I turn up at a polling station next month. It doesn’t even match the name on the photo ID that they themselves issued me.

This would be less-infuriating were it not for the fact that they had my name wrong in the same way on an electoral roll form they sent me in August 20221. When I contacted them to have them fix it, they promised that the underlying problem was solved2 so this very thing wouldn’t happen.

And yet here we are.

Sign on the fence of a school playground reading, in block capitals: "Polling station for Dan Q's imaginary friends only".

Hopefully they’ll be able to fix their records promptly or else I guess I’ll have to apply for a proxy vote, to allow the ballot of my imaginary friend “Dan Que” to be cast by me, Dan Q, instead.

And if that isn’t the most bizarre form of election fraud you’ve ever heard of, I don’t know what is.

Update: True to their word, the council had managed to correct their records by the time I reached the polling station this morning. It’s still a little annoying that they somehow mucked it up in the first place, but I appreciate the efficiency with which they corrected their mistake.

Footnotes

1 They’d had my name right before August 2022, including on previous poll cards; I can only assume that some human operator “corrected” it to the wrong thing at some point.

2 They didn’t fix the problem immediately in August 2022. Initially, they demanded that I produce proof of my change of name from “Dan Que” (which has, of course, never been my name!) to “Dan Q”, and only later backed down and admitted that they’d made a mistake and would correct the PII they were holding about me.

× × ×

you are a printer we are all printers

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Anyway, here’s the best printer for 2024: a Brother laser printer. You can just pick any one you like; I have one with a sheet feeder and one without a sheet feeder. Both of them have reliably printed return labels and random forms and pictures for my kid to color for years now, and I have never purchased replacement toner for either one. Neither has fallen off the WiFi or insisted I sign up for an ink-related hostage situation or required me to consider the ongoing schemes of HP executives who seem determined to make people hate a legendary brand with straightforward cash grabs and weird DRM ideas.

It’s sort-of alarming that Brother are the only big player in the printer space who subscribe to a philosophy of “don’t treat the customers like livestock”. Presumably all it’d take is a board-level decision to flip the switch from “not evil” to “evil” and we’d lose something valuable. Thankfully, for now at least, they still clearly see the value of the positive marketing the world gives them. Positive marketing like like this article.

The article is excellent, by the way. I know that I’m “supposed” to stir up hatred about the fact that its conclusion is written by an AI but… well, just read it for yourself and you’ll see why I don’t mind even one bit. Top notch reporting. Consider following the links within it to stories about how other printer manufacturers continue to show exactly how shitty they can be.

I recommended a Brother printer to the Vagina Museum the other month. I assume it ‘s still working out fine for them (and not ripping them off, spying on them, and/or contributing to the destruction of the the planet).

Quickly Solving JigsawExplorer Puzzles

Background

I was contacted this week by a geocacher called Dominik who, like me, loves geocaching…. but hates it when the coordinates for a cache are hidden behind a virtual jigsaw puzzle.

A popular online jigsaw tool used by lazy geocache owners is Jigidi: I’ve come up with several techniques for bypassing their puzzles or at least making them easier.

Dominik had been looking at a geocache hidden last week in Eastern France and had discovered that it used JigsawExplorer, not Jigidi, to conceal the coordinates. Let’s take a look…

Unsolved approx. 1000 piece jigsaw puzzle.
Not just any puzzle; the geocache used an ~1000 piece puzzle! Ugh!

I experimented with a few ways to work-around the jigsaw, e.g. dramatically increasing the “snap range” so dragging a piece any distance would result in it jumping to a neighbour, and extracting original image URLs from localStorage. All were good, but none were perfect.

For a while, making pieces “snap” at any range seemed to be the best hacky workaround.

Then I realised that – unlike Jigidi, where there can be a congratulatory “completion message” (with e.g. geocache coordinates in) – in JigsawExplorer the prize is seeing the completed jigsaw.

Dialog box reading "This puzzle's box top preview is disabled for added challenge."
You can click a button to see the “box” of a jigsaw, but this can be disabled by the image uploader.

Let’s work on attacking that bit of functionality. After all: if we can bypass the “added challenge” we’ll be able to see the finished jigsaw and, therefore, the geocache coordinates. Like this:

Hackaround

Here’s how it’s done. Or keep reading if you just want to follow the instructions!
  1. Open a jigsaw and try the “box cover” button at the top. If you get the message “This puzzle’s box top preview is disabled for added challenge.”, carry on.
  2. Open your browser’s debug tools (F12) and navigate to the Sources tab.
  3. Find the jigex-prog.js file. Right-click and select Override Content (or Add Script Override).
  4. In the overridden version of the file, search for the string – e&&e.customMystery?tt.msgbox("This puzzle's box top preview is disabled for added challenge."): – this code checks if the puzzle has the “custom mystery” setting switched on and if so shows the message, otherwise (after the :) shows the box cover.
  5. Carefully delete that entire string. It’ll probably appear twice.
  6. Reload the page. Now the “box cover” button will work.

The moral, as always, might be: don’t put functionality into the client-side JavaScript if you don’t want the user to be able to bypass it.

Or maybe the moral is: if you’re going to make a puzzle geocache, put some work in and do something clever, original, and ideally with fieldwork rather than yet another low-effort “upload a picture and choose the highest number of jigsaw pieces to cut it into from the dropdown”.

× ×

Oxford’s Area Code at OGN 57

Ever wondered why Oxford’s area code is 01865? The story is more-complicated than you’d think.

Modern desk telephone with an 01865 number (masked) and a keypad with letters on the buttons, 2=ABC through 9=WXYZ, with superimposed question marks between the two.
As a child, I was told that city STD codes were usually associated to the letters that appear on some telephones… but that wouldn’t make any sense for Oxford’s code!

I’ll share the story on my blog, of course. But before then, I’ll be telling it from the stage of the Jericho Tavern at 21:15 on Wednesday 17 April as my third(?) appearance at Oxford Geek Nights! So if you’re interested in learning about some of the quirks of UK telephone numbering history, I can guarantee that this party’s the only one to be at that Wednesday night!

Not your jam? That’s okay: there’s plenty of more-talented people than I who’ll be speaking, about subjects as diverse as quantum computing with QATboxen, bringing your D&D experience to stakeholder management (!), video games without screens, learnings from the Horizon scandal, and whatever Freyja Domville means by The Unreasonable Effectiveness of the Scientific Method (but I’m seriously excited by that title).

Oxford Geek Night logo

Anyway: I hope you’ll be coming along to Oxford Geek Nights 57 next month, if not to hear me witter on about the fossils in our telecommunications networks then to enjoy a beer and hear from the amazing speakers I’ll be sharing the stage with. The event’s always a blast, and I’m looking forward to seeing you there!

×

BBC News… without the crap

Did I mention recently that I love RSS? That it brings me great joy? That I start and finish almost every day in my feed reader? Probably.

I used to have a single minor niggle with the BBC News RSS feed: that it included sports news, which I didn’t care about. So I wrote a script that downloaded it, stripped sports news, and re-exported the feed for me to subscribe to. Magic.

RSS reader showing duplicate copies of the news story "Barbie 2? 'We'd love to,' says Warner Bros boss", and an entry from BBC Sounds.
Lately my BBC News feed has caused me some annoyance and frustration.

But lately – presumably as a result of technical changes at the Beeb’s side – this feed has found two fresh ways to annoy me:

  1. The feed now re-publishes a story if it gets re-promoted to the front pagebut with a different <guid> (it appears to get a #0 after it when first published, a #1 the second time, and so on). In a typical day the feed reader might scoop up new stories about once an hour, any by the time I get to reading them the same exact story might appear in my reader multiple times. Ugh.
  2. They’ve started adding iPlayer and BBC Sounds content to the BBC News feed. I don’t follow BBC News in my feed reader because I want to watch or listen to things. If you do, that’s fine, but I don’t, and I’d rather filter this content out.

Luckily, I already have a recipe for improving this feed, thanks to my prior work. Let’s look at my newly-revised script (also available on GitHub):

#!/usr/bin/env ruby
require 'bundler/inline'

# # Sample crontab:
# # At 41 minutes past each hour, run the script and log the results
# */20 * * * * ~/bbc-news-rss-filter-sport-out.rb > ~/bbc-news-rss-filter-sport-out.log 2>>&1

# Dependencies:
# * open-uri - load remote URL content easily
# * nokogiri - parse/filter XML
gemfile do
  source 'https://rubygems.org'
  gem 'nokogiri'
end
require 'open-uri'

# Regular expression describing the GUIDs to reject from the resulting RSS feed
# We want to drop everything from the "sport" section of the website, also any iPlayer/Sounds links
REJECT_GUIDS_MATCHING = /^https:\/\/www\.bbc\.co\.uk\/(sport|iplayer|sounds)\//

# Load and filter the original RSS
rss = Nokogiri::XML(open('https://feeds.bbci.co.uk/news/rss.xml?edition=uk'))
rss.css('item').select{|item| item.css('guid').text =~ REJECT_GUIDS_MATCHING }.each(&:unlink)

# Strip the anchors off the <guid>s: BBC News "republishes" stories by using guids with #0, #1, #2 etc, which results in duplicates in feed readers
rss.css('guid').each{|g|g.content=g.content.gsub(/#.*$/,'')}

File.open( '/www/bbc-news-no-sport.xml', 'w' ){ |f| f.puts(rss.to_s) }
It’s amazing what you can do with Nokogiri and a half dozen lines of Ruby.

That revised script removes from the feed anything whose <guid> suggests it’s sports news or from BBC Sounds or iPlayer, and also strips any “anchor” part of the <guid> before re-exporting the feed. Much better. (Strictly speaking, this can result in a technically-invalid feed by introducing duplicates, but your feed reader oughta be smart enough to compensate for and ignore that: mine certainly is!)

You’re free to take and adapt the script to your own needs, or – if you don’t mind being tied to my opinions about what should be in BBC News’ RSS feed – just subscribe to my copy at: https://fox.q-t-a.uk/bbc-news-no-sport.xml

×

RSS > ActivityPub

RSS is better than ActivityPub1.

Photograph of a boxing match, but with the heads of the competitors replaced with the ActivityPub and RSS logos (and "AP" or "RSS" written on their clothes, respectively). RSS is delivering a powerful uppercut to ActivityPub.
A devastating blow by RSS against a competitor 19 years his junior! For updates on this bout as it develops, don’t forget to subscribe… using either protocol.

When I subscribe to content, I want:

  • Resilient failsafes. ActivityPub has many points-of-failure. A notification might fail to complete transmission as a result of downtime, faults, or network conditions, and the receiving server might never know. A feed reader, conversely, can tell you that an address 404’d or the server was down.
  • Retroactive access. Once you fix the problem above… you still don’t get the message you missed: it’s probably gone forever – there’s no retroactive access. The same is true when your ActivityPub server connects with a peer for the first time: you only ever get new content after that point. RSS, on the other hand, provides some number of “recent” items the moment you first subscribe.
  • Simple subscriptions. RSS can be served from a statically-hosted single file, which makes it suitable to deploy anywhere as well as consume using anything. It can be read, after a fashion, in anything from Lynx upwards.

RSS ticks all these boxes. If I can choose between RSS and ActivityPub to subscribe to your content, and I don’t need a real-time update, I’m probably going to choose RSS.

About a month later, Matthias Pfefferle wrote a great post that makes a good “next stop” if you’re on a deep dive…

Footnotes

1 I feel like this statement needs a few clarifications and caveats, but my hot take looks spicier if I bury them in a footnote!

  • By RSS, I mean whichever pull-based basic HTTP you like, be that Atom, JSON Feed, h-entry, or even just properly-marked-up HTML5: did you know that the <article> element is intended to be suitable for syndication use?
  • Obviously I appreciate that RSS and ActivityPub are different tools for different jobs, and there are doubtless use-cases for which ActivityPub is clearly the superior solution.
  • I certainly don’t object to services providing both RSS and ActivityPub as syndication options, like Mastodon does, where both might be good choices.
×

Sonarr > Huginn > Slack

I use a tool called Sonarr to, uhh1, keep track of when new episodes of television shows are released, regardless of what platform they’re on (Netflix, Prime, iPlayer, whatever) and notify me so I remember to watch it.

For several years, I’ve used IFTTT as the intermediary, receiving webhooks from Sonarr and translating them for Slack:

A series of webhooks sent from Sonarr to IFTTT to Slack.
This worked for years, but it’s time to retire it.

IFTTT‘s move to kill its Legacy Pro plan2 – which I was on – gave me reason to re-assess this configuration. It turns that the only Pro feature I was using was an IFTTT “filter” to convert the Sonarr webhooks to a Slack-friendly-format.

Given that I’m running an installation of Huginn on my home network anyway, I resolved to re-implement this flow in Huginn and cancel my IFTTT subscription.

A series of webhooks sent from Sonarr to Huginn to Slack.
Raven-powered automation is the new hotness.

This turned out to be so easy I wonder why I never did it before.

First, I created a Webhook Agent and gave the URL to Sonarr.

Then I connected that to a Slack Agent with the following configuration:

{
  "webhook_url": "https://hooks.slack.com/services/...",
  "channel": "#sonarr",
  "username": "Sonarr",
  "message": "*<https://thetvdb.com/?tab=series&id={{series.tvdbId}}|{{series.title}}>*\nNew episodes:{% for episode in episodes %}\n• S{{episode.seasonNumber}}E{{episode.episodeNumber}} {{episode.title}}{% endfor %}",
  "icon": ":tv:"
}
I’ve omitted my Slack webhook URL so you don’t spam me. I tried for far too long to get the pluralize filter to work so it’d say “episode” or “episodes” as appropriate before realising I didn’t care enough and gave up.

Then all I needed to do was re-emit some of the previous webhooks to test it:

Slack chat window showing notifications: (1) a new episode of Resident Alien, announced by IFTTT, (b) the same episode, announced by "Sonarr", (c) two episodes of Marvel's Spidey and His Amazing Friends, also announced by "Sonarr".
As a bonus, I swapped out the IFTTT logo for Slackmoji’s :tv: icon and added the “Sonarr” username, as shown in my code sample.

Now I’ll continue to know when there’s new television to watch3!

I love the power and flexibility that Huginn provides to help automate your life. It does many of the things that I used to do with a handful of cron jobs and shell scripts, but all in one convenient place.

Footnotes

1 I’ve heard there are other uses for the tool. Your mileage may vary. Don’t forget to pay for your content, if possible.

2 Like many others, I originally signed up to the plan under the promise that the price would be honoured forever. Turns out “forever” means “three years”: who knew?

3 It’s especially useful when you’re between seasons or a show is on hiatus to be reminded that it’s back and I should go and watch it. Hey, there’s a thought: I wonder if I can extract the subtitles from shows and run them through a summarising LLM to give me a couple of paragraphs reminding me “what happened last series” if the show’s been on a long break?

× × ×