Blog

More than you expected?

You're reading everything on Dan's blog - including notes, reposts, checkins, videos and comics.
That might be more than you wanted to see, if you're only interested in blog posts (articles) Dan has written.

Creed Cult-ure

My employer, Automattic, has a creed. Right now it reads:

I will never stop learning. I won’t just work on things that are assigned to me. I know there’s no such thing as a status quo. I will build our business sustainably through passionate and loyal customers. I will never pass up an opportunity to help out a colleague, and I’ll remember the days before I knew everything. I am more motivated by impact than money, and I know that Open Source is one of the most powerful ideas of our generation. I will communicate as much as possible, because it’s the oxygen of a distributed company. I am in a marathon, not a sprint, and no matter how far away the goal is, the only way to get there is by putting one foot in front of another every day. Given time, there is no problem that’s insurmountable.

Lots of companies have something like this, even if it falls short of a “creed”. It could be a “vision”, or a set of “values”, or something in that line.

Of course, sometimes that just means they’ve strung three clichéd words together because they think it looks good under their company logo, and they might as well have picked any equally-meaningless words.

Future logo and values of of Any Company, Anywhere.

But while most companies (and their staff) might pay lip service to their beliefs, Automattic’s one of few that seems to actually live it. And not in an awkward, shoehorned-in way: people here actually believe this stuff.

By way of example:

A woman in a wheelchair waves to a colleague via her laptop screen; she's smiling and has a cup of coffee by her side. Photo by Marcus Aurelius from Pexels.
Coffee: check. Webcam: check. Let’s touch bases, random colleague!

We’ve got a bot that, among other things, pairs up people from across the company for virtual “watercooler chat”/”coffee dates”/etc. It’s cool: I pair-up with random colleagues in my division, or the whole company, or fellow queermatticians… and collectively these provide me a half-hour hangout about once a week. It’s a great way to experience the diversity of culture, background and interests of your colleagues, as well as being a useful way to foster idea-sharing and “watercooler effect” serendipity.

For the last six months or so, I’ve been bringing a particular question to almost every random-chat I’ve been paired into:

What part of the Automattic creed resonates most-strongly for you right now?

Two women in black dresses sit in a graveyard by candlelight and hold up the Automattic logo. Edited image based on original photo by Valeria Boltneva from Pexels (used with permission) and the Automattic logo (used under the assumption that they won't mind, given the context).
On a good day, I’m at least 90% certain I’m a senior software engineer and not a cult member.

I volunteer my own answer first. It’s varied over time. Often I’m most-attached to “I will never stop learning.” Other times I connect best to “I will communicate as much as possible…” or “I am in a marathon, not a sprint…”. Lately I’ve felt a particular engagement with “I will never pass up the opportunity to help a colleague…”.

It varies for other people too. But every single person I’ve asked this question has been able to answer it. And they’ve been able to answer it confidently and with justifications for or examples of their choice.

Have you ever worked anywhere before where seemingly all your coworkers profess a genuine belief in the corporate creed? Like, enough that some of them get it tattooed onto their bodies. Unless you’ve been brainwashed by a cult, the answer is probably no.

Dan sits in his office; behind him, four separate monitors show the Automattic logo.
If Automattic is a cult, then it might be too late for me.

Why are Automatticians like that?

For some folks, of course, the creed is descriptive rather than prescriptive. Regarding its initial creation, Matt says that “as a hack to introduce new folks to our culture, we put a beta Automattic Creed, basically a statement of things important to us, written in the first person.”

But this alone isn’t an explanation, because back then there were only around a hundred people in the company: nowadays there are over 1,500. So how can the creed continue to be such a pervasive influence? Or to put it another way: why are Automatticians… like that?

  • Do we simply attract like-minded individuals? The creed is highly visible and cross-referenced by our recruitment pages, so it wouldn’t be entirely surprising.
  • Maybe we filter for people who are ideologically-compatible with the creed? Insofar as the qualities it describes are essential to integrating into our corporate culture, yes: our recruitment process does a great job of testing for those qualities.
  • Perhaps we converge on these values as a result of our experience as Automatticians? Once you’re in, you’re indoctrinated into the tenets of the creed and internalise its ideas.
  • Or perhaps it’s a combination of the three, in some ratio or another. (What’s the ratio?)

I’ve been here 1⅔ years and don’t know the answer yet. But I’ll tell you this: it’s inspiring to be part of a team that really seem to believe in what they do.

The Automattic Creed presented as an infographic with icons accompanying each tenet.
People keep making infographics of the creed, just for fun. Even if they’re not Automatticians (any longer). That’s not creepy, right?

Incidentally: if the creed speaks to you too, you might like to look at some of the many open positions! I promise we’re not actually a cult.

Plus we’ve been doing “work anywhere” for longer than almost anybody else and we’re really, really good at it.

If you enjoyed this, you might also like other blog posts about my time with Automattic: the recruitment process, accepting an offer, my induction, and the experience of lockdown in a distributed company, among others.

Dan Q posted a note for GC8W7QW Forgotten Bridge

This checkin to GC8W7QW Forgotten Bridge reflects a geocaching.com log entry. See more of Dan's cache logs.

Dropped by while out for a walk and discovered that “Gina + Kylie”, a pair of presumably non-geocachers, found the cache on Tue 8 June and left a note in the logbook! This is cool, not just because it’s always nice to find a friendly muggle but also because it proves that this path isn’t exclusively used by me (and by geocachers following in my footsteps) as I’d thought. Awesome!

Handwritten note from the geocache log: "8/6/20 found by accident, friend thought it was a rat trap. Nice work. gina + kylie"

Map of 51.7652,-1.390367

Getting Twitter Avatars (without the Twitter API)

Among Twitter’s growing list of faults over the years are various examples of its increasing divergence from open Web standards and developer-friendly endpoints. Do you remember when you used to be able to subscribe to somebody’s feed by RSS? When you could see who follows somebody without first logging in? When they were still committed to progressive enhancement and didn’t make your browser download ~5MB of Javascript or else not show any content whatsoever? Feels like a long time ago, now.

Lighthouse Performance score for Twitter's Twitter account page on mobile, scoring 50%.
For one of the most-popular 50 websites in the world, this score is frankly shameful.

But those complaints aside, the thing that bugged me most this week was how much harder they’ve made it to programatically get access to things that are publicly accessible via web pages. Like avatars, for example!

If you’re a human and you want to see the avatar image associated with a given username, you can go to twitter.com/that-username and – after you’ve waited a bit for all of the mandatory JavaScript to download and run (I hope you’re not on a metered connection!) – you’ll see a picture of the user, assuming they’ve uploaded one and not made their profile private. Easy.

If you’re a computer and you want to get the avatar image, it used to be just as easy; just go to twitter.com/api/users/profile_image/that-username and you’d get the image. This was great if you wanted to e.g. show a Facebook-style facepile of images of people who’d retweeted your content.

But then Twitter removed that endpoint and required that computers log in to Twitter, so a clever developer made a service that fetched avatars for you if you went to e.g. twivatar.glitch.com/that-username.

But then Twitter killed that, too. Because despite what they claimed 5½ years ago, Twitter still clearly hates developers.

Dan Q's Twitter profile header showing his avatar image.
You want to that image? Well you’ll need a Twitter account, a developer account, an OAuth token set, a stack of code…

Recently, I needed a one-off program to get the avatars associated with a few dozen Twitter usernames.

First, I tried the easy way: find a service that does the work for me. I’d used avatars.io before but it’s died, presumably because (as I soon discovered) Twitter had made things unnecessarily hard for them.

Second, I started looking at the Twitter API documentation but it took me in the region of 30-60 seconds before I said “fuck that noise” and decided that the set-up overhead in doing things the official way simply wasn’t justified for my simple use case.

So I decided to just screen-scrape around the problem. If a human can just go to the web page and see the image, a computer pretending to be a human can do exactly the same. Let’s do this:

/* Copyright (c) 2021 Dan Q; released under the MIT License. */

const Puppeteer = require('puppeteer');

getAvatar = async (twitterUsername) => {
  const browser = await Puppeteer.launch({args: ['--no-sandbox', '--disable-setuid-sandbox']});
  const page = await browser.newPage();
  await page.goto(`https://twitter.com/${twitterUsername}`);
  await page.waitForSelector('a[href$="/photo"] img[src]');
  const url = await page.evaluate(()=>document.querySelector('a[href$="/photo"] img').src);
  await browser.close();
  console.log(`${twitterUsername}: ${url}`);
};

process.argv.slice(2).forEach( twitterUsername => getAvatar( twitterUsername.toLowerCase() ) );
The code is ludicrously simple. It took less time, energy, and code to write this than to follow Twitter’s “approved” procedure. You can download the code via Gist.

Obviously, using this code would violate Twitter’s terms of use for automation, so… don’t, I guess?

Given that I only needed to run it once, on a finite list of accounts, I maintain that my approach was probably kinder on their servers than just manually going to every page and saving the avatar from it. But if you set up a service that uses this approach then you’ll certainly piss off somebody at Twitter and history shows that they’ll take their displeasure out on you without warning.

$ node get-twitter-avatar.js alexsdutton richove geohashing TailsteakAD LilFierce1 ninjanails
alexsdutton: https://pbs.twimg.com/profile_images/740505937039986688/F9gUV0eK_200x200.jpg
lilfierce1: https://pbs.twimg.com/profile_images/1189417235313561600/AZ2eLjAg_200x200.jpg
richove: https://pbs.twimg.com/profile_images/1576438972/2011_My_picture4_200x200.jpeg
geohashing: https://pbs.twimg.com/profile_images/877137707939581952/POzWWV2d_200x200.jpg
ninjanails: https://pbs.twimg.com/profile_images/1146364466801577985/TvCfb49a_200x200.jpg
tailsteakad: https://pbs.twimg.com/profile_images/1118738807019278337/y5WWkLbF_200x200.jpg
This output shows the avatar URLs of a half a dozen Twitter accounts. It took minutes to write the code and takes seconds to run, but if I’d have done it the “right” way I’d still be unnecessarily wading through Twitter’s sprawling documentation.

But it works. It was fast and easy and I got what I was looking for.

And the moral of the story is: if you make an API and it’s terrible, don’t be surprised if people screen-scape your service instead. (You can’t spell “scraping” without “API”, amirite?)

Hey @LloydsBank! 2009 called and asked if you’re done sending your customers links to unencrypted HTTP endpoints yet. How do you feel about switching this to a HTTPS link rather than relying on an interceptable/injectable HTTP request?

Text message: "Follow this link to download your free Lloyds Bank Mobile Banking app. http://www.lloydsbank.com/mobileapp"

Ireland and the UK Aren’t In The Same Timezone!

This weekend, while investigating a bug in some code that generates iCalendar (ICS) feeds, I learned about a weird quirk in the Republic of Ireland’s timezone. It’s such a strange thing (and has so little impact on everyday life) that I imagine that even most Irish people don’t even know about it, but it’s important enough that it can easily introduce bugs into the way that computer calendars communicate:

Most of Europe put their clocks forward in Summer, but the Republic of Ireland instead put their clocks backward in Winter.

If that sounds to you like the same thing said two different ways – or the set-up to a joke! – read on:

Map showing timezones of Europe. The UK and Ireland are grouped (along with Iceland) in a zone labelled as being UTC+0.
The timezones of Europe look pretty simple compared to some parts of the world, but the illustration of the British Isles hides an interesting eccentricity.

A Brief History of Time (in Ireland)

Poster titled "Time (Ireland) Act 1916", advising that "On and after Sunday 1st October 1916 Western European Time will be ovserved throughout Ireland" asking people to set their clocks and watches back 35 minutes.
Spring forward, fall back… just a little bit back, though. Not too much.

After high-speed (rail) travel made mean solar timekeeping problematic, Great Britain in 1880 standardised on Greenwich Mean Time (UTC+0) as the time throughout the island, and Ireland standardised on Dublin Mean Time (UTC-00:25:21). If you took a ferry from Liverpool to Dublin towards the end of the 19th century you’d have to put your watch back by about 25 minutes. With air travel not yet being a thing, countries didn’t yet feel the need to fixate on nice round offsets in the region of one-hour (today, only a handful of regions retain UTC-offsets of half or quarter hours).

That’s all fine in peacetime, but by the First World War and especially following the Easter Rising, the British government decided that it was getting too tricky for their telegraph operators (many of whom operated out of Ireland, which provided an important junction for transatlantic traffic) to be on a different time to London.

1885 GPO telegraph instrument from the Porthcurno Telegraph Museum, which Dan almost visited the other week but it was closed.
It’s widely believed that the world’s first “U UP? [STOP]” message never got a response as a direct result of Anglo-Irish timezone confusion.
So the Time (Ireland) Act 1916 was passed, putting Ireland on Greenwich Mean Time. Ireland put her clocks back by 35 minutes and synched-up with the rest of the British Isles. And from then on, everything was simple and because nothing ever went wrong in Ireland as a result of the way it was governed by by Britain, nobody ever had to think about the question of timezones on the island again.

Ah. Hmm.

December 1920 photograph showing St Patrick's Street, Cork, following the burning of the city by British forces.
“Those Irish people want to govern their own country, do they? After we so kindly shared our king with them? Right-ho: let’s set fire to their cities and see how they feel then.”

Following Irish independence, the keeping of time carried on in much the same way for a long while, which will doubtless have been convenient for families spread across the Northern Irish border. But then came the Second World War.

Summers in the 1940s saw Churchill introduce Double Summer Time which he believed would give the UK more daylight, saving energy that might otherwise be used for lighting and increasing production of war materiel.

Ireland considered using the emergency powers they’d put in place to do the same, as a fuel saving measure… but ultimately didn’t. This was possibly because aligning her time with Britain might be seen as undermining her neutrality, but was more likely because the government saw that such a measure wouldn’t actually have much impact on fuel use (it certainly didn’t in Britain). Whatever the reason, though, Britain and Northern Ireland were again out-of-sync with one another until the war ended.

Newspaper clipping advising that "Double Summer Time comes to an end on Saturday night, August 8-9, when all clocks and watches should be put back one hour, thus reverting to British Summer Time, which will probably be maintained throughout the winter."
I like to imagine that the development of powerful computers by the folks at Bletchley Park was

From 1968 to 1971 Britain experimented with “British Standard Time” – putting the clocks forward in Summer once, to UTC+1, and then leaving them there for three years. This worked pretty well except if you were Scottish in which case you’ll have found winter mornings to be even gloomier than you were used to, which was already pretty gloomy. Conveniently: during much of this period Ireland was also on UTC+1, but in their case it was part of a different experiment. Ireland were working on joining the European Economic Community, and aligning themselves with “Paris time” year-round was an unnecessary concession but an interesting idea.

But here’s where the quirk appears: the Standard Time Act 1968, which made UTC+1 the “standard” timezone for the Republic of Ireland, was not repealed and is still in effect. Ireland could have started over in 1971 with a new rule that made UTC+0 the standard and added a “Summer Time” alternative during which the clocks are put forward… but instead the Standard Time (Amendment) Act 1971 left UTC+1 as Ireland’s standard timezone and added a “Winter Time” alternative during which the clocks are put back.

Two clocks, both showing the same time. One has a sign reading "LONDON", the other "DUBLIN, I GUESS?"
It all seems so simple until you actually think about it.

(For a deeper look at the legal history of time in the UK and Ireland, see this timeline. Certainly don’t get all your history lessons from me.)

So what?

You might rightly be thinking: so what! Having a standard time of UTC+0 and going forward for the Summer (like the UK), is functionally-equivalent to having a standard time of UTC+1 and going backwards in the Winter, like Ireland, right? It’s certainly true that, at any given moment, a clock in London and a clock in Dublin should show the same time. So why would anybody care?

Perl Data::ICal::TimeZone implementation of Dublin timezone, incorrectly showing summer DST at +1 rather than winter DST of -1.
This code for Europe/Dublin, from the Perl module Data::ICal::TimeZone, is technically-incorrect because it states that the winter time is the standard and daylight savings of +1 hour apply in the summer, rather than the opposite.

But declaring which is “standard” is important when you’re dealing with computers. If, for example, you run a volunteer rota management system that supports a helpline charity that has branches in both the UK and Ireland, then it might really matter that the computer systems involved know what each other mean when they talk about specific times.

The author of an iCalendar file can choose to embed timezone information to explain what, in that file, a particular timezone means. That timezone information might say, for example, “When I say ‘Europe/Dublin’, I mean UTC+1, or UTC+0 in the winter.” Or it might say – like the code above! – “When I say ‘Europe/Dublin’, I mean UTC+0, or UTC+1 in the summer.” Both of these declarations would be technically-valid and could be made to work, although only the first one would be strictly correct in accordance with the law.

Stressed programmer hunched over a MacBook. Photo by Anna Shvets from Pexels.
Clients who need solid timezone support represent 50% of a programmer’s production of stress hormones. See also Falsehoods Programmers Believe About Time.

But if you don’t include timezone information in your iCalendar file, you’re relying  on the feed subscriber’s computer (e.g. their calendar software) to make a sensible interpretation.. And that’s where you run into trouble. Because in cases like Ireland, for which the standard is one thing but is commonly-understood to be something different, there’s a real risk that the way your system interprets and encodes time won’t necessarily be the same as the way somebody else’s does.

If I say I’ll meet you at 12:00 on 1 January, in Ireland, you rightly need to know whether I’m talking about 12:00 in Irish “standard” time (i.e. 11:00, because daylight savings are in effect) or 12:00 in local-time-at-the-time-of-the-meeting (i.e. 12:00). Humans usually mean the latter because we think in terms of local time, but when your international computer system needs to make sure that people are on a shift at the same time, but in different timezones, it needs to be very clear what exactly it means!

And when your daylight savings works “backwards” compared to everybody else’s… that’s sure to make a developer somewhere cry. And, possibly, blog about your weird legislation.

Dan Q performed maintenance for GC8W7QW Forgotten Bridge

This checkin to GC8W7QW Forgotten Bridge reflects a geocaching.com log entry. See more of Dan's cache logs.

The rain finally stopped this afternoon so I figured I’d take my next Zoom meeting outdoors with me and stretch my legs while I talked to work colleagues. And so I went, selfie-stick ahead of me and chatting to teammates in New York and Florence (what a world we live in!), out for a ramble and soon remembered that I was carrying Duck Race / Mustache Pink, a travel bug I’d picked up near Lands End this weekend. So I diverted my walk to come and check up on this cache (it’s looking fine!) and drop off the TB for the next leg of its journey!

Geocacher with a selfie-stick

Map of 51.7652,-1.390367

Dan Q couldn’t find GC1VQ7G Pooh Sticks Bridge

This checkin to GC1VQ7G Pooh Sticks Bridge reflects a geocaching.com log entry. See more of Dan's cache logs.

Coordinates brought me exactly to a tree that would match the hint… except for the fact that it’s been recently felled! (Picture attached.) No sign of cache, and anything else nearby that would fit the hint is on very-clearly-private land, so I’m concerned this cache might have vanished. :-(

Cheered myself up with a quick game of pooh sticks. I won, but that’s to be expected when you play solo.

Map of 51.6546,-2.443

Dan Q found GC5J6JR Would you believe another Almost Motorway Mayhem?

This checkin to GC5J6JR Would you believe another Almost Motorway Mayhem? reflects a geocaching.com log entry. See more of Dan's cache logs.

Stopped at the nearby services on a long journey from dropping my partner’s brother and his boat off Lands End (I can just about see my car from near the GZ: photo attached) for a hot drink and to remotely participate in a work meeting. Meeting’s not starting yet so I walked out the services’ staff exit and came up here to find this cache. Easy find, TFTC.

Map of 51.652383,-2.436317

Dan Q found GC4NTRC Motorway Mayhem M5 Michaelwood Northbound

This checkin to GC4NTRC Motorway Mayhem M5 Michaelwood Northbound reflects a geocaching.com log entry. See more of Dan's cache logs.

Stopped at the services on my way back to Oxford from Lands End, where I was dropping my partner’s brother and his skiff into the sea to begin his attempt to row the length of the UK! The boat trailer is wobbling in a curious way so I’ve been driving extra carefully, so it’s been a long journey so far (and I’ve still got the A40 to tackle!) so the opportunity for a break is a welcome one.

Cache was easy to sight – with the hint – and stealthing around the nearby truckers wasn’t hard, but prickly plants made retrieving the container a little challenging. Wish I’d brought gloves! SL, TFTC.

Map of 51.656333,-2.43345

Dan Q found GCYK2M Got A Light Boy ? Longships Lighthouse

This checkin to GCYK2M Got A Light Boy ? Longships Lighthouse reflects a geocaching.com log entry. See more of Dan's cache logs.

I just launched my partner’s brother – shown in free attached picture – out in his rowboat to begin his attempt to row from Land’s End to John O’ Groats. Naturally this first involves rowing South, around the headland and past the lighthouse, to get to Land’s End! So I came up the hill to watch him get started. And while I was at it, I figured I’d find this cache! Took travel bug. TFTC!

Robin rows away from Sennen Cove and towards St. Ives

Map of 50.07555,-5.705467

Dan Q found GC6VG1N “BOSISTOW FARM”

This checkin to GC6VG1N "BOSISTOW FARM" reflects a geocaching.com log entry. See more of Dan's cache logs.

I arrived yesterday at nearby Raftra Farm for a weekend, mostly to launch my partner’s brother into the sea to begin his attempt to row from Land’s End to John O’ Groats (making use of inland waterways as much as possible). After a bit of a lie-in this morning, I came out for a brief walk and to find this geocache. Probably this’ll be a highlight of my day, as much of the rest of it will be dominated by catching up on the work I didn’t get done yesterday (during the drive down here from Oxford), at least until the afternoon tide turns which is when we’re doing the first launch!

Easy to find cache hidden in the most likely location – I maintain one just like this near my old house North of Oxford! TFTC.

Dan, on a country lane in Cornwall, in front of a bright blue sky, waves to the camera.

Map of 50.05335,-5.670867

Dan Q found GC5VXQ3 Motorway Mayhem – M5 – Gloucester Services (South)

This checkin to GC5VXQ3 Motorway Mayhem - M5 - Gloucester Services (South) reflects a geocaching.com log entry. See more of Dan's cache logs.

A quick and easy find (though I was glad of the hint when I approached the obstacle at the GZ) while travelling from Oxford to Cornwall to dump my partner’s brother in the sea for the start of his personal challenge to row the length of the UK. (Photo of our boat in tow attached!)

Arthur (red car) and Lucy (rowboat in tow) parked at a service station alongside caravans and HGVs

Map of 51.818183,-2.22145