After a morning of optimising a nonprofit’s reverse proxy configuration, I feel like I’ve earned my lunch! Four cheese, mushroom and jalapeño quesapizzas, mmm…
Tag: three rings
Sabbatical Magic
A couple of weeks ago, I kicked off my first sabbatical since starting at Automattic a little over five years ago1.
The first fortnight of my sabbatical has consisted of:
- Three Rings CIC’s AGM weekend and lots of planning for the future of the organisation and how we make it a better place to volunteer, and better value for our charity users,
- building a first draft of Three Rings’ new server architecture, which turns out to mostly work but still needs some energy thrown at it,
- a geohashing expedition with the dog, and
- a family holiday to Catalonia, Spain.
The trip to Spain followed a model for European family breaks that we first tried in Paris last year2, but was extended to give us a feel for more of the region than a simple city break would. Ultimately, we ended up in three separate locations:
- Barcelona, where we stayed in a wonderful skyscraper hotel with fantastic breakfasts and, after I was able to get enough sleep, explored the obvious touristy bits of the city (e.g. la Sagrada Família3 and other Gaudían architecture, the chocolate museum, the fort at Montjuic, and because it’s me, of course, a widely varied handful of geocaches).
- The PortAventura World theme park, whose accommodation was certainly a gear shift after the 5-star hotel we’d come from4 but whose rides kept us and the kids delighted for a couple of days (Shambhala was a particular hit with the eldest kid and me).
- A villa in el Vilosell – a village of only 190 people – at which the kids mostly played in the outdoor pool (despite the sometimes pouring rain) but we did get the chance to explore the local area a little. Also, of course, some geocaching: some local caches are 1-2 years old and yet had so few finds that I was able to be only the tenth or even just the third person to sign the logbooks!
I’d known – planned – that my sabbatical would involve a little travel. But it wasn’t until we began to approach the end of this holiday that I noticed a difference that a holiday on sabbatical introduces, compared to any other holiday I’ve taken during my adult life…
Perhaps because of the roles I’ve been appointed to – or maybe as a result of my personality – I’ve typically found that my enjoyment of the last day or two of a week-long trip are marred somewhat by intrusive thoughts of the work week to follow.
If I’m back to my normal day job on Monday, then by Saturday I’m already thinking about what I’ll need to be working on (in my case, it’s usually whatever I left unfinished right before I left), contemplating logging-in to work to check my email or Slack, and so on5.
But this weekend, that wasn’t even an option. I’ve consciously and deliberately cut myself off from my usual channels of work communication, and I’ve been very disciplined about not turning any of them back on. And even if I did… my team aren’t expecting me to sign into work for about another 11 weeks anyway!
Monday and Tuesday are going to mostly be split between looking after the children, and voluntary work for Three Rings (gotta fix that new server architecture!). Probably. Wednesday? Who knows.
That’s my first taste of the magic of a sabbatical, I think. The observation that it’s possible to unplug from my work life and, y’know, not start thinking about it right away again.
Maybe I can use this as a vehicle to a more healthy work/life balance next year.
Footnotes
1 A sabbatical is a perk offered to Automatticians giving them three months off (with full pay and benefits) after each five years of work. Mine coincidentally came hot on the tail of my last meetup and soon after a whole lot of drama and a major shake-up, so it was a very welcome time to take a break… although of course it’s been impossible to completely detach from bits of the drama that have spilled out onto the open Web!
2 I didn’t get around to writing about Paris, but I did write about how the hotel we stayed at introduced our eldest, and by proxy re-introduced me, to Wonder Boy, ultimately leading to me building an arcade cabinet on which I finally, beat the game, 35 years after first playing it.
3 Whose construction has come on a lot since the last time I toured inside it.
4 Although alcohol helped with that.
5 I’m fully aware that this is a symptom of poor work/life balance, but I’ve got two decades of ingrained bad habits working against me now; don’t expect me to change overnight!
Bluesky and enshittification
This is a repost promoting content originally published elsewhere. See more things Dan's reposted.
…
Any system where users can leave without pain is a system whose owners have high switching costs and whose users have none. An owner who makes a bad call – like removing the block function say, or opting every user into AI training – will lose a lot of users. Not just those users who price these downgrades highly enough that they outweigh the costs of leaving the service. If leaving the service is free, then tormenting your users in this way will visit in swift and devastating pain upon you.
…
There’s a name for this dynamic, from the world of behavioral economics. It’s called a “Ulysses Pact.” It’s named for the ancient hacker Ulysses, who ignored the normal protocol for sailing through the sirens’ sea. While normie sailors resisted the sirens’ song by filling their ears with wax, Ulysses instead had himself lashed to the mast, so that he could hear the sirens’ song, but could not be tempted into leaping into the sea, to be drowned by the sirens.
Whenever you take a measure during a moment of strength that guards against your own future self’s weakness, you enter into a Ulysses Pact – think throwing away the Oreos when you start your diet.
…
Wise words from Cory about why he isn’t on Bluesky, which somewhat echo my own experience. If you’ve had the experience in recent memory of abandoning an enshittified Twitter (and if you didn’t yet… why the fuck not?), TikTok, or let’s face it Reddit… and you’ve looked instead to services like Bluesky or arguably Threads… then you haven’t learned your lesson at all.
Freedom to exit is fundamental, and I’m a big fan of systems with a built-in Ulysses Pact. In non-social or unidirectionally-social software it’s sufficient for the tools to be open source: this allows me to host a copy myself if a hosted version falls to enshittification. But for bidirectional social networks, it’s also necessary for them to be federated, so that I’m not disadvantaged by choosing to drop any particular provider in favour of another or my own.
Bluesky keeps promising a proper federation model, but it’s not there yet. And I’m steering clear until it is.
I suppose I also enjoyed this post of Cory’s because it helped remind me of where I myself am failing to apply the Ulysses Pact. Right now, Three Rings is highly-centralised, and while I and everybody else involved with it know our exit strategy should the project have to fold (open source it, help charities migrate to their own instances, etc.) right now that plan is less “tie ourselves to the mast” than it is “trust one another to grab us if we go chasing sirens”. We probably ought to fix that.
Bad Names for Servers
Six or seven years ago our eldest child, then a preschooler, drew me a picture of the Internet1. I framed it and I keep it on the landing outside my bedroom – y’know, in case I get lost on the Internet and need a map:
I found myself reminded of this piece of childhood art when she once again helped me with a network map, this weekend.
As I kick off my Automattic sabbatical I’m aiming to spend some of this and next month building a new server architecture for Three Rings. To share my plans, this weekend, I’d been drawing network diagrams showing my fellow volunteers what I was planning to implement. Later, our eldest swooped in and decided to “enhance” the picture with faces and names for each server:
I noted that she named the read-replica database server Demmy2, after our dog.
It’s a cute name for a server, but I don’t think I’m going to follow it. The last thing I want is for her to overhear me complaining about some possible future server problem and misinterpret what I’m saying. “Demmy is a bit slow; can you give her a kick,” could easily cause distress, not to mention “Demmy’s dying; can we spin up a replacement?”
I’ll stick to more-conventional server names for this new cluster, I think.
Note #24767
Kicking off day #1 of my three-month sabbatical from work at a hotel in Reading, at a meeting with fellow Three Rings volunteers to discuss our organisational culture and values.
Sick words
This evening I pushed against my illness-addled brain to try to sit in on the fortnightly Zoom call with the Three Rings dev team. Unfortunately it seems like the primary symptom of my cold is an inability to string words together.
At one point, I apologised to by colleague “Beff” (I meant “Bev”, but I had just been talking about “Geoff”) that I couldn’t work out how to stop “scaring my screen” (well, I suppose Halloween is coming up…). Then, realising my mistake, explained that it was a bit of a “ting-twister”.
I should just go to bed, right?
Note #24388
Me: “This weekend I need to update these buildout scripts for Debian 12 and implement a new provider middleware.”
Dog’s expression: “Or we could just go for a walk?”
Me: “Or we could just go for a walk, sure.”
Roll Your Own Antispam
Three Rings operates a Web contact form to help people get in touch with us: the idea is that it provides a quick and easy way to reach out if you’re a charity who might be able to make use of the system, a user who’s having difficulty with the features of the software, or maybe a potential new volunteer willing to give your time to the project.
But then the volume of spam it received increased dramatically. We don’t want our support team volunteers to spend all their time categorising spam: even if it doesn’t take long, it’s demoralising. So what could we do?
Our conventional antispam tools are configured pretty liberally: we don’t want to reject a contact from a legitimate user just because their message hits lots of scammy keywords (e.g. if a user’s having difficulty logging in and has copy-pasted all of the error messages they received, that can look a lot like a password reset spoofing scam to a spam filter). And we don’t want to add a CAPTCHA, because not only do those create a barrier to humans – while not necessarily reducing spam very much, nowadays – they’re often terrible for accessibility, privacy, or both.
But it didn’t take much analysis to spot some patterns unique to our contact form and the questions it asks that might provide an opportunity. For example, we discovered that spam messages would more-often-than-average:
- Fill in both the “name” and (optional) “Three Rings username” field with the same value. While it’s cetainly possible for Three Rings users to have a login username that’s identical to their name, it’s very rare. But automated form-fillers seem to disproportionately pair-up these two fields.
- Fill the phone number field with a known-fake phone number or a non-internationalised phone number from a country in which we currently support no charities. Legitimate non-UK contacts tend to put international-format phone numbers into this optional field, if they fill it at all. Spammers often put NANP (North American Numbering Plan) numbers.
- Include many links in the body of the message. A few links, especially if they’re to our services (e.g. when people are asking for help) is not-uncommon in legitimate messages. Many links, few of which point to our servers, almost certainly means spam.
- Choose the first option for the choose -one question “how can we help you?” Of course real humans sometimes pick this option too, but spammers almost always choose it.
None of these characteristics alone, or any of the half dozen or so others we analysed (including invisible checks like honeypots and IP-based geofencing), are reason to suspect a message of being spam. But taken together, they’re almost a sure thing.
To begin with, we assigned scores to each characteristic and automated the tagging of messages in our ticketing system with these scores. At this point, we didn’t do anything to block such messages: we were just collecting data. Over time, this allowed us to find a safe “threshold” score above which a message was certainly spam.
Once we’d found our threshold we were able to engage a soft-block of submissions that exceeded it, and immediately the volume of spam making it to the ticketing system dropped considerably. Under 70 lines of PHP code (which sadly I can’t share with you) and we reduced our spam rate by over 80% while having, as far as we can see, no impact on the false-positive rate.
Where conventional antispam solutions weren’t quite cutting it, implementing a few rules specific to our particular use-case made all the difference. Sometimes you’ve just got to roll your sleeves up and look at the actual data you do/don’t want, and adapt your filters accordingly.
Dan Q found GC51F07 Knapwell one and a half
This checkin to GC51F07 Knapwell one and a half reflects a geocaching.com log entry. See more of Dan's cache logs.
The second of two caches found on a morning walk from the nearby Cambridge Belfry Hotel, where some fellow volunteers and I met yesterday for a meeting. This cache looked so close, but being on the other side of the A428 meant that my route to get from one to the other side of the trunk road necessitated a long and circuitous route around half a dozen (ill-maintained) pegasus crossings around the perimeter of two large roundabouts! Thankfully traffic was quiet at this point if a Saturday morning.
Cache itself was worth the effort though. Feels like it’s increasingly rare to find a large, appropriately-camouflaged, well looked-after cache in a nice location, so FP awarded. TFTC!
Dan Q found GC10ZT3 Off Yer Trolley! (Cambourne)
This checkin to GC10ZT3 Off Yer Trolley! (Cambourne) reflects a geocaching.com log entry. See more of Dan's cache logs.
Even early on a Saturday morning, after a volunteering event the previous day at the hotel across the road, this highly-exposed GZ made me feel vulnerable! It’s not as though anybody were actually watching me as I stood around nonchalantly at the GZ waiting for an opportunity to make a search: a couple of shop workers setting up, maybe, and a handful of drivers going past… but what got me was that every time I looked up from my rummaging I spotted, in the corner of my eye, a police officer standing to attention just on the other side of the car park, staring intently in my direction!
The copper in question, of course, was nothing more than a cardboard cut-out designed to spook shoplifters, but man that’s a chilling thing to spot in your peripheral vision when you’re rooting around in the bushes for a concealed container in a quiet car park!
Signed the log and took a selfie with my law enforcement friend (attached) before getting back to my day. TFTC!
Dan Q found GC8TNPE Incy Wincy
This checkin to GC8TNPE Incy Wincy reflects a geocaching.com log entry. See more of Dan's cache logs.
The elder child and I are staying nearby and couldn’t resist coming to a nearby cache with so many FPs. The name gave us a bit of a clue what we would be looking for but nothing could have prepared us for for this imaginative and unusual container! FP awarded. Attached is very non-spoiler photo of us with our very own Incy Wincies. Greetings from Oxfordshire!
[Bloganuary] Not The Lottery
This post is part of my attempt at Bloganuary 2024. Today’s prompt is:
What would you do if you won the lottery?
I know what I’d do, and I’ll get to that. But first, let me tell you about the lottery game I play.
Not the lottery
I don’t generally play the lottery1. I’ve made interactive widgets (now broken) to illustrate quite how many losers there are in these games and hopefully help highlight that while “it could be you”… it won’t be.
But if I ever happen to be somewhere that the lottery results are being announced, I sometimes like to play a game I call Not The Lottery.2 Here’s how you play:
- Set aside the money it would have cost for a ticket.
- Think of the numbers you’d have played.
- When those numbers don’t come up, congratulations: you just won not-wasting-your-money!3
Want to play Not The Lottery retroactively? Cool. I’ve made and open-sourced a tool for that. Hopefully it’ll load below and you can choose some numbers (or take a Lucky Dip) and have it played through the entirety of EuroMillions history and see how much money you’d have won if you’d only played them every week. Or, to look at things from a brighter perspective, how much you’ve saved by not playing. It’s almost-certainly in the thousands.
Loading game… please wait… (if it never loads, Dan
probably broke it; sorry!)
Winning the lottery
But that’s not what the question’s really about, is it? We don’t ask people “what would they do if they won the lottery?” because we think it’s likely to happen4 We ask them because… well, because it’s fun to fantasise.
And I sort-of gave the answer away on day 20 of Bloganuary: I’d do my “dream job”. I’d work (for free) for Three Rings, like I already do, except instead of spending a couple of hours a week on it on average I’d spend about ten times that. I’d use the luxury of not having to work to focus on things that I know I can do to make the world a better place.
Sure, there’s other things I’d do. They’re mostly obvious things that I’d hope anybody in my position would do. Pay off the mortgage (and for all the works currently being done to
infuriate the dog improve the house). Arrange some kind of slow-access trust or annuity for the people closest to me so that
they need not worry about money, nor about having to work out how to spend, save, or invest a lump sum. Maybe a holiday or two. Certainly some charitable donations. Perhaps buy
really expensive ketchup: the finest dijon ketchup5.
But mostly I’d just want to be able to live as comfortably as I do now, or perhaps slightly more, and spend a greater proportion of my time than I already do making charities work better.
I don’t know if that makes me insufferably self-righteous or insufferably simple-minded, but it’s probably one of those.
Footnotes
1 I’ve been caught describing it as “a tax on people who are bad at maths”, but I don’t truly believe that (although I am concerned about how readily we let people get addicted to problematic gambling and then keep encouraging them to play with dark patterns that hide how low the odds truly are). I’ve even been known to buy a ticket or two, some years.
2 While writing this, I decided to retroactively play for last Friday, having not seen whatever numbers came up. I guessed only one of them. Hurrah! That means I saved £2.50 by not playing!
3 There are, of course, other possible outcomes. You could have missed out on winning a small prize – the odds aren’t that low – but the solution to this is simple: just keep playing Not The Lottery and you, as the “house”, will come out on top in the end. Alternatively, it’s just-about possible that you could pluck the jackpot numbers from thin air, in which case: well done! You’re doing better than Derren Brown when in 2009 he performed a pretty good magic trick but then turned it into a turd when he “explained” it using pseudoscience (why not just stick with “I’m a magician, duh”; when you play the Uri Geller card you just make yourself look like an idiot). Let’s find a way to use those superpowers for good. Because what you’ve got is a superpower. For context: if you played Not The Lottery twice a week, every week, without fail, for 393 years… you’d still only have a 1% chance of having ever predicted a jackpot in your five-lifetimes.
4 What if we lived in a world where we did use statistics to think about the hypothetical questions we ask people? Would we ask “what would you do if you were stuck by lightning?”, given that the lifetime chance of being killed by lightning is significantly greater than the chance of winning the jackpot, even if you play every draw!
5 Y’know, to keep in the fridge in the treehouse.
[Bloganuary] Dream Job
This post is part of my attempt at Bloganuary 2024. Today’s prompt is:
What’s your dream job?
It feels like a bit of a cop-out to say I’m already doing it, but that’s true. Well, mostly (read on and I’ll make a counterpoint!).
Automattic
I’m incredibly fortunate that my job gets to tick so many of the boxes I’d put on a “dream job wishlist”:
- I work on things that really matter. Automattic’s products make Web publishing and eCommerce available to the world without “lock-in” or proprietary bullshit. I genuinely believe that Automattic’s work helps to democratise the Internet and acts, in a small way, as a counterbalance to the dominance of the big social media silos.
- I get to make the world a better place by giving away as much intellectual property as possible. Automattic’s internal policy is basically “you don’t have to ask to open source something; give away anything you like so long as it’s not the passwords”.1 Open Source is one of the most powerful ideas of our generation, and all that.
- We work in a distributed, asynchronous way. I work from where I want, when I want. I’m given the autonomy to understand what my ideal working environment is and make the most of it. Some mornings I’m just not feeling that coding flow, so I cycle somewhere different and try working the afternoon in a different location. Some weekends I’m struck by inspiration and fire up my work laptop to make the most of it, because, y’know, I’m working on things that really matter and I care about them.
- I work with amazing people who I learn from and inspire me. Automattic’s home to some incredibly talented people and I love that I’ve managed to find a place that actively pushes me to study new things every day.
- Automattic’s commitment to diversity & inclusion is very good-to-excellent. As well as getting work work alongside people from a hundred different countries and with amazingly different backgrounds, I love that I get to work in one of the queerest and most queer-affirming environments I’ve ever been paid to be in.
Did I mention that we’re hiring?2
Three Rings
But you know where else ticks all of those boxes? My voluntary work with Three Rings. Let me talk you through that wishlist again:
- I work on things that really matter. We produce the longest-running volunteer management system in the world3 We produce it as volunteers ourselves, because we believe that volunteering matters and we want to make it as easy as possible for as many people as possible to do as much good as possible, and this allows us to give it away as cheaply as possible: for free, to the smallest and poorest charities.
- I get to make the world a better place by facilitating the work of suicide helplines, citizens advice bureaus, child support services, environmental charities, community libraries and similar enterprises, museums, theatres, charity fundraisers, and so many more good works. Back when I used to to helpline volunteering I might do a three hour shift and help one or two people, and I was… okay at it. Now I get to spend those three hours making tools that facilitate many tens of thousands of volunteers to provide services that benefit an even greater number of people across six countries.
- We work in a distributed, asynchronous way. Mostly I work from home; sometimes we get together and do things as a team (like in the photo above). Either way, I’m trusted with the autonomy to produce awesome things in the way that works best for me, backed with the help and support of a team that care with all their hearts about what we do.
- I work with amazing people who I learn from and inspire me. I mentioned one of them yesterday. But seriously, I could sing the praises of any one of our two-dozen strong team, whether for their commitment to our goals, their dedication to making the world better, their passion for quality and improvement, their focus when producing things that meet our goals, or their commitment to sticking with us for years or decades, without pay, simply because they know that what we do is important and necessary for so many worthy causes. And my fellow development/devops volunteers continue to introduce me to new things, which scratches my “drive-to-learn” itch.
- Three Rings’ commitment to diversity & inclusion is very good, and improving. We skew slightly queer and have moderately-diverse gender mix, but I’m especially impressed with our age range these days: there’s at least 50 years between our oldest and youngest volunteers with a reasonably-even spread throughout, which is super cool (and the kind of thing many voluntary organisations dream of!).
The difference
The biggest difference between these two amazing things I get to work on is… only one of them pays me. It’s hard to disregard that.
Sometimes at Automattic, I have to work on something that’s not my favourite project in the world. Or the company’s priorities clash with my own, and I end up implementing something that my gut tells me isn’t the best use of my time from a “make the world a better place” perspective. Occasionally they take a punt on something that really pisses me off.
That’s all okay, of course, because they pay me, and I have a mortgage to settle. That’s fine. That’s part of the deal.
My voluntary work at Three Rings is more… mine. I’m the founder of the project; I 100% believe in what it’s trying to achieve. Even though I’ve worked to undermine the power of my “founder privilege” by entrusting the organisation to a board and exec that I know will push back and challenge me, I feel safe fully trusting that everything I give to Three Rings will be used in the spirit of the original mission. And even though I might sometimes disagree with others on the best way forward, I accept that whatever decision is made comes from a stronger backing than if I’d acted alone.
Three Rings, of course, doesn’t pay me4. That’s why I can only give them a few hours a week of my time. If I could give more, I would, but I have bills to pay so my “day job” is important too: I’m just so incredibly fortunate that that “day job” touches upon many of the same drives that are similarly satisfied by my voluntary work.
If I didn’t have bills to pay, I could happily just volunteer for Three Rings. I’d miss Automattic, of course: there are some amazing folks there whom I love very much, and I love the work. But if they paid me as little as Three Rings did – that is, nothing! – I’d choose Three Rings in a heartbeat.
But man, what a privileged position I’m in that I can be asked what my dream job is and I can answer “well, it’s either this thing that I already do, or this other thing that I already do, depending on whether this hypothetical scenario considers money to be a relevant factor.” I’m a lucky, lucky man.
Footnotes
1 I’m badly-paraphrasing Matt, but you get the gist.
2 Automattic’s not hiring as actively nor voraciously as it has been for the last few years – a recent downtown in the tech sector which you may have seen have heavily affected many tech companies has flooded the market with talent, and we’ve managed to take our fill of them – we’re still always interested to hear from people who believe in what we do and have skills that we can make use of. And because we’re a community with a lot of bloggers, you can find plenty of first-hand experiences of our culture online if you’d like to solicit some opinions before you apply…
3 Disclaimer: Three Rings is the oldest still-running volunteer management system we’re aware of: our nearest surviving “competitor”, which provides similar-but-different features for a price that’s an order of magnitude greater, launched later in the same year we started. But maybe somebody else has been running these last 22 years that we haven’t noticed, yet: you never know!
4 Assuming you don’t count a Christmas dinner each January – yes, really! (it turns out to be cheaper to celebrate Christmas in January) – as payment.
Dan Q found GC5F425 Lovers Walk
This checkin to GC5F425 Lovers Walk reflects a geocaching.com log entry. See more of Dan's cache logs.
My GPSr dropped me next to a far older bit of architecture than the one that hosts the cache, but found after a short search. I’m staying nearby as part of a charity hackathon for a nonprofit I’m involved with, but came out for a walk and an explore while between other tasks. SL, TFTC.
Dan Q found GC97WKD 2 BH
This checkin to GC97WKD 2 BH reflects a geocaching.com log entry. See more of Dan's cache logs.
A nonprofit I volunteer with has, years ago, held our Christmas bash at the nearby Fairlawns Hotel. We haven’t been in several years and – even though we missed Christmas itself by a full month! – decided to return here this year.
I’m often an early riser, especially when away from home, and enjoy making the most of the first light with a walk. Last time I was here there wasn’t a geocache in sight, so imagine my delight to find that now there’s one right on the doorstep! Armed with a torch to fight off the renaming pre-dawn darkness, I braved the cold and came out to explore.
Found the obvious hiding spot quickly, but my sore back (Fairlawns’ mattress was somewhat softer than I enjoy!) made retrieval challenging! Still, a success once I was on my hands and knees! TFTC, and Merry Christmas I guess!