Science Weekend

This weekend was full of science.

Research

String of large electromagnets used for steering an electron beam.
If you wanna bend a stream of electrons travelling at nearly the speed of light, you’re gonna need a lot of big magnets.

This started on Saturday with a trip to the Harwell Campus, whose first open day in eight years provided a rare opportunity for us to get up close with cutting edge science (plus some very kid-friendly and accessible displays) as well as visit the synchrotron at Diamond Light Source.

Dan with a child in front of beamlines coming out of the Diamond Light Synchrotron ring.
It’s hard to convey the scale of the thing; turns out you need a big ol’ ring if you want to spin electrons fast enough to generate a meaningful amount of magnetobremsstrahlung radiation.

The whole thing’s highly-recommended if you’re able to get to one of their open days in the future, give it a look. I was particularly pleased to see how enthused about science it made the kids, and what clever questions they asked.

For example: the 7-year-old spent a long time cracking a variety of ciphers in the computing tent (and even spotted a flaw in one of the challenge questions that the exhibitors then had to hand-correct on all their handouts!); the 10-year-old enjoyed quizzing a researcher who’d been using x-ray crystallography of proteins.

Medicine

And then on Sunday I finally got a long-overdue visit to my nearest spirometry specialist for a suite of experiments to try to work out what exactly is wrong with my lungs, which continue to be a minor medical mystery.

Dan holds a piece of medical apparatus to his mouth.
“Once you’ve got your breath back, let’s fill you with drugs and do those experiments again…”

It was… surprisingly knackering. Though perhaps that’s mostly because once I was full of drugs I felt briefly superpowered and went running around the grounds of the wonderfully-named Brill Hill Windmill with the dog until was panting in pretty much the way that I might normally have been, absent an unusually-high dose of medication.

Computer screen graphs showing peak respiratory flow under a series of different experiments.
It’s got a graph; that makes it science, right? (I’m ignoring those party political histograms that outright lie about how narrow the margins are…)

For amusement purposes alone, I’d be more-likely to recommend the first day’s science activities than the second, but I can’t deny that it’s cool to collect a load of data about your own body and how it works in a monitorable, replicable way. And maybe, just maybe, start to get to the bottom of why my breathing’s getting so much worse these last few years!

× × × ×

Automattic Switch

Earlier this year, I changed team at work1.

Sankey diagram showing team Alpha splitting into Fire and Fuel; later Desire forms from parts of Fire and Fuel; later still Experiments forks from Fire and the remainder of Fire eventually become part of Desire. Team Masamune appears part-way along the timeline and runs parallel and independent to it. Dan's trajectory starts in Alpha, becomes part of Fire, and is eventually in the portion that merges into Desire.
This simplified diagram shows my journey so-far (dotted white line) through Automattic.

When I started at Automattic in 2019 I was assigned to Team Alpha, the engineering team responsible for WooCommerce.com. In 2020 the team later forked into teams Fire and Fuel, and I landed on the Fire side of the split2.

This winter, though, Fire became underpopulated. We lost a few folks to a newly-formed Experiments team, and several individual team members moved to other parts of the company. Once we got small enough it wasn’t worthwhile being a team in our own right. Our focus areas got split between Desire and Masume, and those of us who were left got absorbed into Team Desire. That’s where you’ll now find me.

Intranet identity card for Dan Q (he/him/his), Code Magician on Desire.
I miss being “on Fire”, because it sounded cool. Maybe I should suggest a patch for our intranet to allow teams to choose the preposition used when referring to their members, e.g. from “on”, “in”, “of” etc. Then I could be the “Code Magician of Desire”, which is a cool job title once again.

I was initially a bit bummed about the dissolution of my old team3 and struggled to find my place in my new team. The work is similar and the codebases overlap, but even sibling teams can have different rituals and approaches to problems that provide a learning barrier4.

I think I’ve begun to find my feet now, and next week I’m excited to meet many of my new team in-person for the first time at a Desire-wide meetup in Amsterdam5.

Footnotes

1 Strangely, this isn’t directly related to Automattic’s recent re-organisation, which I’ve written about previously, but is a result of more-local changes within my division coupled with the natural flow of Automatticians around the company. But it does make it feel from my perspective like a lot of things are getting jiggled about simultaneously!

2 When Alpha were first discussing the upcoming split, I suggested that we might like to give our new teams a “pair” of names that linked to one another, and threw out a few ideas to get the ball rolling. One of those ideas was “Fuel and Fire”; I jokingly added that “it was like the Metallica song, which also gave us ‘Desire’ as a possible third team name should the need arise”. This wasn’t supposed to be taken seriously, but apparently it was taken seriously enough because my suggestion was the winner and I soon ended up “on Fire”.

3 Many of my old teammates and I did at least manage to get together for one final (virtual) social event, culminating in a symbolic “extinguishing of fire” as a candle that had been left burning through the meeting was put out at the end.

4 A team’s rituals aren’t just about the way they hold their meetings or run their retros; for example my new team are very disciplined about announcing their appearance on a morning with a friendly greeting in our social channel, which are for some reason generally responded-to with a barrage of “waving Pikachu” slackmoji. I don’t know why Pikachu is the mascot of our mornings, but I’ve joined in because it’s a fun gesture of the team’s distinct collective personality. Also it’s a cute GIF: it’s nice to get waved-at by Pikachu on a morning.

5 Doubly-awesome, the destination’s proximity means that I get to travel by Eurostar rather than having to fly.

× ×

Reflecting on Bloganuary

Well that was Bloganuary! It was pressuring, exhausting, and – mostly! – fun. Let’s recap what I wrote about each day of January:

  1. My Biggest Challenge, for which I pointed at motivation in the winter and how that was a major part of my motivation for trying to participate in Bloganuary in the first place! I also touched on the difficulty of staying on-task.
Chart showing number of articles on DanQ.me by month of year, with a pronounced dip starting in January and continuing through until a rebound in April.
Early in January I shared this chart which indicates the severity of the “dip” I typically see in my blog output in the first few months of the year. Could I overcome this through sheer determination, I wondered?
  1. Playtime. I talked about some of the “play” activities I engage in, including roleplaying games, board games, videogames, escape rooms, and GNSS games.
  2. Alumnus: an exploration of the higher education establishments I’ve been part of.
  3. The Gift of Time, when I talked about being time-poor and seemingly perpetually-busy and expressed my love of gifts that help me reclaim that time.
  4. Nostalgia vs Futurism. I spend comparable amounts of time thinking about the future as the past, I reckon.
  5. Billboards: a silly joke about a billboard.
  6. A Different Diet, talking about aspiring towards something slightly-closer to veganism, perhaps starting by reducing my dairy consumption.
  7. Live Long and Prosper, in which I commemorate my birthday by talking about the dangers of humans living much longer than they do.
  8. Mission, another silly joke.
King Arthur again, but now he says "I wanna, like, make cool shit on the Internet or whatever."
You and me both, Arthur, King of the Britons.
  1. Attachment, about how I didn’t really have an “attachment object” as a kid.
  2. Paws to Hear my Scents-ible Idea: a silly pitch for a smell-based social network for dogs.
  3. Pizza, a post about the greatest food ever invented.
  4. Road Trip! After ruling out a series of runners-up, perhaps my most-memorable road trip was the one to Kit’s wedding.
  5. Communicate Early, Communicate Often, about the ways I communicate online (spoiler: a lot of it’s right here!).
  6. Magpies are the Best Bird. Nay, the best animal.
  7. Clutter, about the clutter in my physical space but perhaps even more in my head.
  8. Puppy Love: the unconditional love of a dog.
  9. Uninvention, in which I propose uninventing cryptocurrency.
  10. Leadership: I revisited an old post about the qualities I admire in leaders; it’s still true.
  11. Dream Job – am I already doing my dream job? Maybe, though perhaps it isn’t the one that pays me!
  12. What’s in a name? My name today is one I chose for myself, but it’s not the only name I’ve been known by. I revisit the names I’ve been called and what they’ve meant.
  13. New Tricks, about how convenient it’d be to be able to explain to our dog that the builders in our house are not here to steal her toys.
  14. Fun Five: five things I do for fun – code, magic, play, piano, learn. A bit of a parallel to “Playtime” from day 2.
  1. Harcourt Manor, a local attraction I’ve never gotten to see inside.
  2. Landslide, the spectacular song that inspired this post because I didn’t objected to the original prompt.
  3. Traditions my family practices, some of which are pretty unique to us.
  4. Reading List, about how mine is pretty long this time of year, but that doesn’t stop me thinking about what I might re-read next.
  5. Not The Lottery, a game I play that’s… well… not the lottery. And how if I played the actual lottery (and somehow won), how I’d do my “dream job” from day 18.
  6. Sportsball! I don’t really play or follow any sports, but that doesn’t stop me writing a diatribe of what’s wrong with professional soccer.
  7. Toilet Paper is typically mounted on a holder in one of two polarities. One of those orientations is an abomination.
  8. The Fear of expressing vulnerability is real in this final Bloganuary entry.

So yeah: 31 posts in as many days! Actually, it was closer to 40, because on a couple of days I wrote non-Bloganuary posts too:

Generating a chart...
If this message doesn't go away, the JavaScript that makes this magic work probably isn't doing its job right: please tell Dan so he can fix it.

Of course, with the addition of this post, it’s now 32+ posts in 32 days. As I’ve noted before, this is my longest daily streak in over 25 years of blogging… and I’m genuinely a little curious how much longer I can keep it up. There are lots of things I meant to write about last month but simply didn’t have time: if I dusted off a few of those ideas I could push on a few days longer. My longest unstreak or “dry spell” – the longest number of consecutive days I’ve gone without making a post – is 42 days: could I beat that? That’d be a special level of personal best.

Trophy on a desk with the plaque "most pointless blog posts".
Wait, is that “most pointless” in quality, or most “pointless posts” as in quantity?

I initially aimed to fuel and inspire my blogging at the start of this year in a more-interpersonal way, by making some pen pals and writing about the experience of that. Except I ran slightly late with my first (and haven’t written it up yet) and even later with my second (on account of winter blues plus spending any spare “blogging” time doing Bloganuary) so that project’s already way off track. Still aiming to catch-up though.

But I’m pleased to have been able to throw out 20,000 words of prompt-driven blog posts too, even if some of the prompts were weaker than others!

× × ×

[Bloganuary] My Biggest Challenge

This post is part of my attempt at Bloganuary 2024. Today’s prompt is:

What are your biggest challenges?

The Challenge of Winter Motivation

Two years ago, I reflected in February that I’d made almost zero blog posts in the previous month. Last month, I implemented a dynamically-updating Blog Stats page and my “winter/early Spring dip” became more-visible than ever.

Chart showing number of articles on DanQ.me by month of year, with a pronounced dip starting in January and continuing through until a rebound in April.
I find winters are generally bad for my creativity and motivation, usually until I bounce back in the Spring.

In an attempt to keep me writing daily, I’m giving Bloganuary a go this year. It’s sort-of like the NaNoWriMo of blogging1. And for me, Bloganuary’s very purpose is to overcome the challenge of getting disconnected from blogging when the nights are long and inspiration’s hard to find2.

The Challenge of Staying On-Task

But outside of the winter, my biggest challenge is usually… staying on-task!

It’s easy to get my focus to wane and for me to drift into some other activity than whatever it is I should be spending my time on. It’s not even procrastination3 so much as it’s a fluctuating and changing field of interest. I’ll drift off of what I’m supposed to be working on and start on something that interests me more in that moment… and then potentially off that too, in turn. The net result is that both my personal and professional lives are awash with half-finished projects4, all waiting their turn for me to find the motivation to swing back around and pick them up on some subsequent orbit of my brain.

A person wearing a cardboard box on their head, labelled "BRAIN". Above, a hand reaches from out-of-frame to hold a sign labelled "IDEA" above them.
You know how sometimes a stock image says exactly what you need it to? This isn’t one of those times.

It’s the kind of productivity antipattern I’d bring up with my coach, except that I already know exactly how she’d respond. First, she’d challenge the need to change; require that I justify it first. Second, she’d insist that before I can change, I need to accept and come to terms with who I am, intrinsically: if this flitting-about is authentically “me”, who am I to change it?

Finally, after weeks or months of exercises to fulfil these two tasks, she’d point out that I’ve now reached a place where I’m still just as liable to change lanes in the middle of a project as I was to begin with, but now I’m more comfortable with that fact. I won’t have externally changed, I’ll “just” have found some kind of happy-clappy inner peace. And she’ll have been right that that’s what I’d actually needed all along.

Maybe it’s not such a challenge, after all.

Footnotes

1 Except that would be NaBloPoMo, of course. But it’s a similar thing.

2 Also, perhaps, to help me focus on writing more-often, on more-topics, than I might otherwise in the course of my slow, verbose writing.

3 Except when it is.

4 Not to mention countless draft blog posts!

× ×

Blogging Stats

During a conversation with a colleague last week, I claimed that while I blog more-frequently than I did 5-10 years ago, it’s still with a much lower frequency than say 15-20 years ago.

Only later did I stop to think: is that actually true? It’s time for a graph!

I’ve previously graphed my blogging in an ad-hoc way, e.g. in 2016 I did a word-count and in 2021 I graphed posts-by-month-of-year, but I’ve never made an “eternal”, automatically-updating, interactive1 graph. Until now:

Generating a chart...
If this message doesn't go away, the JavaScript that makes this magic work probably isn't doing its job right: please tell Dan so he can fix it.

If you consider just articles (and optionally notes, which some older content might have been better classified-as, in retrospect) it looks like I’m right. Long gone are months like February 2005 when I posted an average of three times every two days! November 2018 was a bit of an anomaly as a I live-tweeted Challenge Robin II: my recent output’s mostly been comparable to the “quiet period” from 2008-20102.

Looking at number of posts by month of the year, it’s interesting to see a pronounced “dip” in all kinds of output roundabout March, less reposts in Summer and Autumn, and – perhaps unsurprisingly – more checkins (which often represent geocaching/geohashing logs) in the warmer months. Even on this scale, you can see the impact of the November “Challenge Robin spike” in the notes:

Generating a chart...
If this message doesn't go away, the JavaScript that makes this magic work probably isn't doing its job right: please tell Dan so he can fix it.

Anyway, now I’ve actually automated these kinds of stats its easier than ever for me to ask questions about how and when I write in my blog. I’ve put living copies of the charts plus additional treats (want to know when my longest “daily streak” was?) on a special page dedicated to that purpose. It’ll be interesting to see how it looks on this blog’s 25th anniversary, in a little under a year!

Footnotes

1 Try clicking on any of the post kinds in the legend to add/remove them, or click-and-drag a range across the chart to zoom in.

2 In hindsight, I was clearly depressed in and around 2009 and this doubtless impacted my ability to engage in “creative” pursuits.

Automattic International

(This is yet another post about Automattic. Seee more posts about my experience of working at Automattic.)

Off the back of my recent post about privileges I enjoy as a result of my location and first language, even at my highly-multinational employer, and inspired by my colleague Atanas‘ data-mining into where Automatticians are located, I decided to do another treemap, this time about which countries Automatticians call home:

Where are the Automatticians?

Treemap showing countries of Automatticians. North America and specifically the USA dominates, the UK has the most in Europe, etc.
If raw data’s your thing (or if you’re just struggling to make out the names of the countries with fewer Automatticians), here’s a CSV file for you.

To get a better picture of that, let’s plot a couple of cartograms. This animation cycles between showing countries at (a) their actual (landmass) size and (b) approximately proportional to the number of Automatticians based in each country:

Animation showing countries "actual size" changing to proportional-to-Automattician-presence.
This animation alternates between showing countries at “actual size” and proportional to the number of Automatticians based there. North America and Europe dominate the map, but there are other quirks too: look at e.g. how South Africa, New Zealand and India balloon.

Another way to consider the data would be be comparing (a) the population of each country to (b) the number of Automatticians there. Let’s try that:

Animation showing countries proportional to population changing to proportional-to-Automattician-presence.
Here we see countries proportional to their relative population change shape to show number of Automatticians, as seen before. Notice how countries with larger populations like China shrink away to nothing while those with comparatively lower population density like Australia blow up.

There’s definitely something to learn from these maps about the cultural impact of our employee diversity, but I can’t say more about that right now… primarily because I’m not smart enough, but also at least in part because I’ve watched the map animations for too long and made myself seasick.

A note on methodology

A few quick notes on methodology, for the nerds out there who’ll want to argue with me:

  1. Country data was extracted directly from Automattic’s internal staff directory today and is based on self-declaration by employees (this is relevant because we employ a relatively high number of “digital nomads”, some of whom might not consider any one country their home).
  2. Countries were mapped to continents using this dataset.
  3. Maps are scaled using Robinson projection. Take your arguments about this over here.
  4. The treemaps were made using Excel. The cartographs were produced based on work by Gastner MT, Seguy V, More P. [Fast flow-based algorithm for creating density-equalizing map projections. Proc Natl Acad Sci USA 115(10):E2156–E2164 (2018)].
  5. Some countries have multiple names or varied name spellings and I tried to detect these and line-up the data right but apologies if I made a mess of it and missed yours.
× × ×

Automattic Privilege

I’ve been thinking recently about three kinds of geographic privilege I enjoy in my work at Automattic. (See more posts about my experience of working at Automattic.)

1. Timezone Privilege

Take a look at the map below. I’m the pink pin here in Oxfordshire. The green pins are my immediate team – the people I work with on a day-to-day basis – and the blue pins are people outside of my immediate team but in its parent team (Automattic’s org chart is a bit like a fractal).

World map showing the locations of Dan, his immediate team, and its parent team. There's a cluster of nine pins Europe, a few pins further East in Russia and Indonesia, one in Cape Town, two in North America, and one in Central America.
I’m the pink pin; my immediate team are the green pins. People elsewhere in our parent team are the blue pins. Some pins represent multiple people.

Thinking about timezones, there are two big benefits to being where I am:

  1. I’m in the median timezone, which makes times that are suitable-for-everybody pretty convenient for me (I have a lot of lunchtime/early-afternoon meetings where I get to watch the sun rise and set, simultaneously, through my teammates’ windows).
  2. I’m West of the mean timezone, which means that most of my immediate coworkers start their day before me so I’m unlikely to start my day blocked by anything I’m waiting on.

(Of course, this privilege is in itself a side-effect of living close to the meridian, whose arbitrary location owes a lot to British naval and political clout in the 19th century: had France and Latin American countries gotten their way the prime median would have probably cut through the Atlantic or Pacific oceans.)

2. Language Privilege

English is Automattic’s first language (followed perhaps by PHP and Javascript!), not one of the 120 other languages spoken by Automatticians. That’s somewhat a consequence of the first language of its founders and the language in which the keywords of most programming languages occur.

It’s also a side-effect of how widely English is spoken, which in comes from (a) British colonialism and (b) the USA using Hollywood etc. to try to score a cultural victory.

Treemap showing languages spoken by Automatticians: English dominates, followed by Spanish, French, German, Italian, Hindi, Portugese, Mandarin, Russian, Japanese, Polish, Afrikaans, Dutch, Green, Catalan, Cantonese, Romanian, and many others.
Languages self-reportedly spoken by Automatticians, sized proportional to the number of speakers. No interpretation/filtering has been done, so you’ll see multiple dialects of the same root language.

I’ve long been a fan of the concept of an international axillary language but I appreciate that’s an idealistic dream whose war has probably already been lost.

For now, then, I benefit from being able to think, speak, and write in my first language all day, every day, and not have the experience of e.g. my two Indonesian colleagues who routinely speak English to one another rather than their shared tongue, just for the benefit of the rest of us in the room!

3. Passport Privilege

Despite the efforts of my government these last few years to isolate us from the world stage, a British passport holds an incredible amount of power, ranking fifth or sixth in the world depending on whose passport index you follow. Compared to many of my colleagues, I can enjoy visa-free and/or low-effort travel to a wider diversity of destinations.

Normally I might show you a map here, but everything’s a bit screwed by COVID-19, which still bars me from travelling to many places around the globe, but as restrictions start to lift my team have begun talking about our next in-person meetup, something we haven’t done since I first started when I met up with my colleagues in Cape Town and got assaulted by a penguin.

But even looking back to that trip, I recall the difficulties faced by colleagues who e.g. had to travel to a different country in order tom find an embassy just to apply for the visa they’d eventually need to travel to the meetup destination. If you’re not a holder of a privileged passport, international travel can be a lot harder, and I’ve definitely taken that for granted in the past.

I’m going to try to be more conscious of these privileges in my industry.

× ×

Automattic Growth

It just passed two years since I started working at Automattic, and I just made a startling discovery: I’ve now been with the company for longer than 50% of the staff.

When you hear that from a 2-year employee at a tech company, it’s easy to assume that they have a high staff turnover, but Automattic’s churn rate is relatively low, especially for our sector: 86% of developers stay longer than 5 years. So what’s happening? Let’s visualise it:

Graph showing all 1.802 current Automatticians as coloured squares, ordered by their start date. Dan Q's square is squarely (LOL) in the middle.
Everything in this graph, in which each current Automattician is a square, explains how I feel right now: still sometimes like a new fish, but in an increasingly big sea.

All that “red” at the bottom of the graph? That’s recent growth. Automattic’s expanding really rapidly right now, taking on new talent at a never-before-seen speed.

Since before I joined it’s been the case that our goals have demanded an influx of new engineers at a faster rate than we’ve been able to recruit, but it looks like things are improving. Recent refinements to our recruitment process (of which I’ve written about my experience) have helped, but I wonder how much we’ve also been aided by pandemic-related changes to working patterns? Many people, and especially in tech fields, have now discovered that working-from-home works for them, and a company like Automattic that’s been built for the last decade and a half on a “distributed” model is an ideal place to see that approach work at it’s best.

We’re rolling out new induction programmes to support this growth. Because I care about our corporate culture, I’ve volunteered myself as a Culture Buddy, so I’m going to spend some of this winter helping Newmatticians integrate into our (sometimes quirky, often chaotic) ways of working. I’m quite excited to be at a point where I’m in the “older 50%” of the organisation and so have a responsibility for supporting the “younger 50%”, even though I’m surprised that it came around so quickly.

Screenshot from officetoday.wordpress.com, where Automatticians share photos of their current work environment.
Automattic… culture? Can’t we just show them Office Today and be done with it?

I wonder how that graph will look in another two years.

×

Note #19508

When do I #blog? A breakdown of my top four #indieweb content types – articles, reposts, checkins and notes – by month.

Blog posts by type and month, a graph showing how articles barely change throughout the year but reposts are highest in the first half of the year, checkins peak in the spring and summer, and notes get a big boost in November.

Unsurprisingly my checkins, which represent #geocaching/#geohashing activity, grow in the spring and peak in the summer when the weather’s better!

At first I assumed the notes peak in November might have been thrown off by a single conference, e.g. musetech, but it turns out I’ve just done more note-friendly things in Novembers, like Challenge Robin II and my Cape Town meetup, which are enough to throw the numbers off.

×

Heatmapping my Movements

As I mentioned last year, for several years I’ve collected pretty complete historic location data from GPSr devices I carry with me everywhere, which I collate in a personal μlogger server.

Going back further, I’ve got somewhat-spotty data going back a decade, thanks mostly to the fact that I didn’t get around to opting-out of Google’s location tracking until only a few years ago (this data is now also housed in μlogger). More-recently, I now also get tracklogs from my smartwatch, so I’m managing to collate more personal location data than ever before.

Inspired perhaps at least a little by Aaron Parecki, I thought I’d try to do something cool with it.

Heatmapping my movements

The last year

Heatmap showing Dan's movements around Oxford since moving house in 2020. There's a strong cluster around Stanton Harcourt with heavy tendrils around Witney and Eynsham and along the A40 to Summertown, and lighter tendrils around North and Central Oxford.
My movements over the last year have been relatively local, but there are some interesting hotspots and common routes.

What you’re looking at is a heatmap showing my location over the last year or so since I moved to The Green. Between the pandemic and switching a few months prior to a job that I do almost-entirely at home there’s not a lot of travel showing, but there’s some. Points of interest include:

  • The blob around my house, plus some of the most common routes I take to e.g. walk or cycle the children to school.
  • A handful of my favourite local walking and cycling routes, some of which stand out very well: e.g. the “loop” just below the big blob represents a walk around the lake at Dix Pit; the blob on its right is the Devils Quoits, a stone circle and henge that I thought were sufficiently interesting that I made a virtual geocache out of them.
  • The most common highways I spend time on: two roads into Witney, the road into and around Eynsham, and routes to places in Woodstock and North Oxford where the kids have often had classes/activities.
  • I’ve unsurprisingly spent very little time in Oxford City Centre, but when I have it’s most often been at the Westgate Shopping Centre, on the roof of which is one of the kids’ favourite restaurants (and which we’ve been able to go to again as Covid restrictions have lifted, not least thanks to their outdoor seating!).

One to eight years ago

Let’s go back to the 7 years prior, when I lived in Kidlington. This paints a different picture:

Heatmap showing Dan's movements around Kidlington, including a lot of time in the village and in Oxford City Centre, as well as hotspots at the hospital, parks, swimming pools, and places that Dan used to volunteer. Individual expeditions can also be identified.
For the seven years I lived in Kidlington I moved around a lot more than I have since: each hotspot tells a story, and some tell a few.

This heatmap highlights some of the ways in which my life was quite different. For example:

  • Most of my time was spent in my village, but it was a lot larger than the hamlet I live in now and this shows in the size of my local “blob”. It’s also possible to pick out common destinations like the kids’ nursery and (later) school, the parks, and the routes to e.g. ballet classes, music classes, and other kid-focussed hotspots.
  • I worked at the Bodleian from early 2011 until late in 2019, and so I spent a lot of time in Oxford City Centre and cycling up and down the roads connecting my home to my workplace: Banbury Road glows the brightest, but I spent some time on Woodstock Road too.
  • For some of this period I still volunteered with Samaritans in Oxford, and their branch – among other volunteering hotspots – show up among my movements. Even without zooming in it’s also possible to make out individual venues I visited: pubs, a cinema, woodland and riverside walks, swimming pools etc.
  • Less-happily, it’s also obvious from the map that I spent a significant amount of time at the John Radcliffe Hospital, an unpleasant reminder of some challenging times from that chapter of our lives.
  • The data’s visibly “spottier” here, mostly because I built the heatmap only out of the spatial data over the time period, and not over the full tracklogs (i.e. the map it doesn’t concern itself with the movement between two sampled points, even where that movement is very-guessable), and some of the data comes from less-frequently-sampled sources like Google.

Eight to ten years ago

Let’s go back further:

Heatmap showing Dan's movements around Oxford during the period he lived in Kennington. Again, it's dominated by time at home, in the city centre, and commuting between the two.
Back when I lived in Kennington I moved around a lot less than I would come to later on (although again, the spottiness of the data makes that look more-significant than it is).

Before 2011, and before we bought our first house, I spent a couple of years living in Kennington, to the South of Oxford. Looking at this heatmap, you’ll see:

  • I travelled a lot less. At the time, I didn’t have easy access to a car and – not having started my counselling qualification yet – I didn’t even rent one to drive around very often. You can see my commute up the cyclepath through Hinksey into the City Centre, and you can even make out the outline of Oxford’s Covered Market (where I’d often take my lunch) and a building in Osney Mead where I’d often deliver training courses.
  • Sometimes I’d commute along Abingdon Road, for a change; it’s a thinner line.
  • My volunteering at Samaritans stands out more-clearly, as do specific venues inside Oxford: bars, theatres, and cinemas – it’s the kind of heatmap that screams “this person doesn’t have kids; they can do whatever they like!”

Every map tells a story

I really love maps, and I love the fact that these heatmaps are capable of painting a picture of me and what my life was like in each of these three distinct chapters of my life over the last decade. I also really love that I’m able to collect and use all of the personal data that makes this possible, because it’s also proven useful in answering questions like “How many times did I visit Preston in 2012?”, “Where was this photo taken?”, or “What was the name of that place we had lunch when we got lost during our holiday in Devon?”.

There’s so much value in personal geodata (that’s why unscrupulous companies will try so hard to steal it from you!), but sometimes all you want to do is use it to draw pretty heatmaps. And that’s cool, too.

Heatmap showing Dan's movements around Great Britain for the last 10 years: with a focus on Oxford, tendrils stretch to hotspots in South Wales, London, Cambridge, York, Birmingham, Preston, Glasgow, Edinburgh, and beyond.

How these maps were generated

I have a μlogger instance with the relevant positional data in. I’ve automated my process, but the essence of it if you’d like to try it yourself is as follows:

First, write some SQL to extract all of the position data you need. I round off the latitude and longitude to 5 decimal places to help “cluster” dots for frequency-summing, and I raise the frequency to the power of 3 to help make a clear gradient in my heatmap by making hotspots exponentially-brighter the more popular they are:

SELECT ROUND(latitude, 5) lat, ROUND(longitude, 5) lng, POWER(COUNT(*), 3) `count`
FROM positions
WHERE `time` BETWEEN '2020-06-22' AND '2021-08-22'
GROUP BY ROUND(latitude, 5), ROUND(longitude, 5)

This data needs converting to JSON. I was using Ruby’s mysql2 gem to fetch the data, so I only needed a .to_json call to do the conversion – like this:

db = Mysql2::Client.new(host: ENV['DB_HOST'], username: ENV['DB_USERNAME'], password: ENV['DB_PASSWORD'], database: ENV['DB_DATABASE'])
db.query(sql).to_a.to_json

Approximately following this guide and leveraging my Mapbox subscription for the base map, I then just needed to include leaflet.js, heatmap.js, and leaflet-heatmap.js before writing some JavaScript code like this:

body.innerHTML = '<div id="map"></div>';
let map = L.map('map').setView([51.76, -1.40], 10);
// add the base layer to the map
L.tileLayer('https://api.mapbox.com/styles/v1/{id}/tiles/{z}/{x}/{y}?access_token={accessToken}', {
  maxZoom: 18,
  id: 'itsdanq/ckslkmiid8q7j17ocziio7t46', // this is the style I defined for my map, using Mapbox
  tileSize: 512,
  zoomOffset: -1,
  accessToken: '...' // put your access token here if you need one!
}).addTo(map);
// fetch the heatmap JSON and render the heatmap
fetch('heat.json').then(r=>r.json()).then(json=>{
  let heatmapLayer = new HeatmapOverlay({
    "radius": parseFloat(document.querySelector('#radius').value),
    "scaleRadius": true,
    "useLocalExtrema": true,
  });
  heatmapLayer.setData({ data: json });
  heatmapLayer.addTo(map);
});

That’s basically all there is to it!

× × × ×

Note #19021

I made a graph to show how the number of large hand tools stored in our garage has changed this last year…

Graph showing, over time, the number of large tools increasing as a rake, midi-spade, post holer, rake and others are acquired. Each acquired tool is labelled with what it is. However: a hatchet, a pickaxe and two log splitting axes are not labelled.

…but I forgot to label the axes.

×

It’s 2020 and you’re in the future

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

West Germany’s 1974 World Cup victory happened closer to the first World Cup in 1930 than to today.

The Wonder Years aired from 1988 and 1993 and depicted the years between 1968 and 1973. When I watched the show, it felt like it was set in a time long ago. If a new Wonder Years premiered today, it would cover the years between 2000 and 2005.

Also, remember when Jurassic Park, The Lion King, and Forrest Gump came out in theaters? Closer to the moon landing than today.

These things come around now and again, but I’m not sure of the universal validity of observing that a memorable event is now closer to another memorable event than it is to the present day. I don’t think that the relevance of events is as linear as that. Instead, perhaps, it looks something like this:

Graph showing that recent events matter a lot, but rapidly tail off for a while before levelling out again as they become long-ago events.
Recent events matter more than ancient events to the popular consciousness, all other things being equal, but relative to one another the ancient ones are less-relevant and there’s a steep drop-off somewhere between the two.

Where the drop-off in relevance occurs is hard to pinpoint and it probably varies a lot by the type of event that’s being remembered: nobody seems to care about what damn terrible thing Trump did last month or the month before when there’s some new terrible thing he did just this morning, for example (I haven’t looked at the news yet this morning, but honestly whenever you read this post he’ll probably have done something awful).

Nonetheless, this post on Wait But Why was a fun distraction, even if it’s been done before. Maybe the last time it happened was so long ago it’s irrelevant now?

XKCD 1393: Timeghost - 'Hello, Ghostbusters?' 'ooOOoooo people born years after that movie came out are having a second chiiiild right now ooOoooOoo'
Of course, there’s a relevant XKCD. And it was published closer to the theatrical releases of Cloudy with a Chance of Meatballs and Paranormal Activity than it was to today. OoooOOoooOOoh.
×

Note #14044

Lots of interesting results from the @bodleianlibs staff survey. Pleased to have my suspicions confirmed about my department’s propensity to be accepting of individuals: it’s the only one where a majority of people strongly agreed with the statement “I feel able to by myself at work” and one of only two where nobody disagreed with it. That feels like an accurate representation of my experience with my team these last 7-8 years!

'I feel able to by myself at work' staff survey results chart showing my department strongly agrees

Going Critical

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

If you’ve spent any time thinking about complex systems, you surely understand the importance of networks.
Networks rule our world. From the chemical reaction pathways inside a cell, to the web of relationships in an ecosystem, to the trade and political networks that shape the course of history.
Or consider this very post you’re reading. You probably found it on a social network, downloaded it from a computer network, and are currently deciphering it with your neural network.
But as much as I’ve thought about networks over the years, I didn’t appreciate (until very recently) the importance of simple diffusion.
This is our topic for today: the way things move and spread, somewhat chaotically, across a network. Some examples to whet the appetite:
  • Infectious diseases jumping from host to host within a population
  • Memes spreading across a follower graph on social media
  • A wildfire breaking out across a landscape
  • Ideas and practices diffusing through a culture
  • Neutrons cascading through a hunk of enriched uranium
A quick note about form.
Unlike all my previous work, this essay is interactive. There will be sliders to pull, buttons to push, and things that dance around on the screen. I’m pretty excited about this, and I hope you are too.
So let’s get to it. Our first order of business is to develop a visual vocabulary for diffusion across networks.

A simple model

I’m sure you all know the basics of a network, i.e., nodes + edges.
To study diffusion, the only thing we need to add is labeling certain nodes as active. Or, as the epidemiologists like to say, infected:
This activation or infection is what will be diffusing across the network. It spreads from node to node according to rules we’ll develop below.
Now, real-world networks are typically far bigger than this simple 7-node network. They’re also far messier. But in order to simplify — we’re building a toy model here — we’re going to look at grid or lattice networks throughout this post.
(What a grid lacks in realism, it makes up for in being easy to draw ;)
Except where otherwise specified, the nodes in our grid will have 4 neighbors, like so:
And we should imagine that these grids extend out infinitely in all directions. In other words, we’re not interested in behavior that happens only at the edges of the network, or as a result of small populations.
Given that grid networks are so regular, we can simplify by drawing them as pixel grids. These two images represent the same network, for example:
Alright, let’s get interactive.

Fabulous (interactive! – click through for the full thing to see for yourself) exploration of network interactions with applications for understanding epidemics, memes, science, fashion, and much more. Plus Kevin’s made the whole thing CC0 so everybody can share and make use of his work. Treat as a longread with some opportunities to play as you go along.