Heatmapping my Movements

As I mentioned last year, for several years I’ve collected pretty complete historic location data from GPSr devices I carry with me everywhere, which I collate in a personal μlogger server.

Going back further, I’ve got somewhat-spotty data going back a decade, thanks mostly to the fact that I didn’t get around to opting-out of Google’s location tracking until only a few years ago (this data is now also housed in μlogger). More-recently, I now also get tracklogs from my smartwatch, so I’m managing to collate more personal location data than ever before.

Inspired perhaps at least a little by Aaron Parecki, I thought I’d try to do something cool with it.

Heatmapping my movements

The last year

Heatmap showing Dan's movements around Oxford since moving house in 2020. There's a strong cluster around Stanton Harcourt with heavy tendrils around Witney and Eynsham and along the A40 to Summertown, and lighter tendrils around North and Central Oxford.
My movements over the last year have been relatively local, but there are some interesting hotspots and common routes.

What you’re looking at is a heatmap showing my location over the last year or so since I moved to The Green. Between the pandemic and switching a few months prior to a job that I do almost-entirely at home there’s not a lot of travel showing, but there’s some. Points of interest include:

  • The blob around my house, plus some of the most common routes I take to e.g. walk or cycle the children to school.
  • A handful of my favourite local walking and cycling routes, some of which stand out very well: e.g. the “loop” just below the big blob represents a walk around the lake at Dix Pit; the blob on its right is the Devils Quoits, a stone circle and henge that I thought were sufficiently interesting that I made a virtual geocache out of them.
  • The most common highways I spend time on: two roads into Witney, the road into and around Eynsham, and routes to places in Woodstock and North Oxford where the kids have often had classes/activities.
  • I’ve unsurprisingly spent very little time in Oxford City Centre, but when I have it’s most often been at the Westgate Shopping Centre, on the roof of which is one of the kids’ favourite restaurants (and which we’ve been able to go to again as Covid restrictions have lifted, not least thanks to their outdoor seating!).

One to eight years ago

Let’s go back to the 7 years prior, when I lived in Kidlington. This paints a different picture:

Heatmap showing Dan's movements around Kidlington, including a lot of time in the village and in Oxford City Centre, as well as hotspots at the hospital, parks, swimming pools, and places that Dan used to volunteer. Individual expeditions can also be identified.
For the seven years I lived in Kidlington I moved around a lot more than I have since: each hotspot tells a story, and some tell a few.

This heatmap highlights some of the ways in which my life was quite different. For example:

  • Most of my time was spent in my village, but it was a lot larger than the hamlet I live in now and this shows in the size of my local “blob”. It’s also possible to pick out common destinations like the kids’ nursery and (later) school, the parks, and the routes to e.g. ballet classes, music classes, and other kid-focussed hotspots.
  • I worked at the Bodleian from early 2011 until late in 2019, and so I spent a lot of time in Oxford City Centre and cycling up and down the roads connecting my home to my workplace: Banbury Road glows the brightest, but I spent some time on Woodstock Road too.
  • For some of this period I still volunteered with Samaritans in Oxford, and their branch – among other volunteering hotspots – show up among my movements. Even without zooming in it’s also possible to make out individual venues I visited: pubs, a cinema, woodland and riverside walks, swimming pools etc.
  • Less-happily, it’s also obvious from the map that I spent a significant amount of time at the John Radcliffe Hospital, an unpleasant reminder of some challenging times from that chapter of our lives.
  • The data’s visibly “spottier” here, mostly because I built the heatmap only out of the spatial data over the time period, and not over the full tracklogs (i.e. the map it doesn’t concern itself with the movement between two sampled points, even where that movement is very-guessable), and some of the data comes from less-frequently-sampled sources like Google.

Eight to ten years ago

Let’s go back further:

Heatmap showing Dan's movements around Oxford during the period he lived in Kennington. Again, it's dominated by time at home, in the city centre, and commuting between the two.
Back when I lived in Kennington I moved around a lot less than I would come to later on (although again, the spottiness of the data makes that look more-significant than it is).

Before 2011, and before we bought our first house, I spent a couple of years living in Kennington, to the South of Oxford. Looking at this heatmap, you’ll see:

  • I travelled a lot less. At the time, I didn’t have easy access to a car and – not having started my counselling qualification yet – I didn’t even rent one to drive around very often. You can see my commute up the cyclepath through Hinksey into the City Centre, and you can even make out the outline of Oxford’s Covered Market (where I’d often take my lunch) and a building in Osney Mead where I’d often deliver training courses.
  • Sometimes I’d commute along Abingdon Road, for a change; it’s a thinner line.
  • My volunteering at Samaritans stands out more-clearly, as do specific venues inside Oxford: bars, theatres, and cinemas – it’s the kind of heatmap that screams “this person doesn’t have kids; they can do whatever they like!”

Every map tells a story

I really love maps, and I love the fact that these heatmaps are capable of painting a picture of me and what my life was like in each of these three distinct chapters of my life over the last decade. I also really love that I’m able to collect and use all of the personal data that makes this possible, because it’s also proven useful in answering questions like “How many times did I visit Preston in 2012?”, “Where was this photo taken?”, or “What was the name of that place we had lunch when we got lost during our holiday in Devon?”.

There’s so much value in personal geodata (that’s why unscrupulous companies will try so hard to steal it from you!), but sometimes all you want to do is use it to draw pretty heatmaps. And that’s cool, too.

Heatmap showing Dan's movements around Great Britain for the last 10 years: with a focus on Oxford, tendrils stretch to hotspots in South Wales, London, Cambridge, York, Birmingham, Preston, Glasgow, Edinburgh, and beyond.

How these maps were generated

I have a μlogger instance with the relevant positional data in. I’ve automated my process, but the essence of it if you’d like to try it yourself is as follows:

First, write some SQL to extract all of the position data you need. I round off the latitude and longitude to 5 decimal places to help “cluster” dots for frequency-summing, and I raise the frequency to the power of 3 to help make a clear gradient in my heatmap by making hotspots exponentially-brighter the more popular they are:

SELECT ROUND(latitude, 5) lat, ROUND(longitude, 5) lng, POWER(COUNT(*), 3) `count`
FROM positions
WHERE `time` BETWEEN '2020-06-22' AND '2021-08-22'
GROUP BY ROUND(latitude, 5), ROUND(longitude, 5)

This data needs converting to JSON. I was using Ruby’s mysql2 gem to fetch the data, so I only needed a .to_json call to do the conversion – like this:

db = Mysql2::Client.new(host: ENV['DB_HOST'], username: ENV['DB_USERNAME'], password: ENV['DB_PASSWORD'], database: ENV['DB_DATABASE'])
db.query(sql).to_a.to_json

Approximately following this guide and leveraging my Mapbox subscription for the base map, I then just needed to include leaflet.js, heatmap.js, and leaflet-heatmap.js before writing some JavaScript code like this:

body.innerHTML = '<div id="map"></div>';
let map = L.map('map').setView([51.76, -1.40], 10);
// add the base layer to the map
L.tileLayer('https://api.mapbox.com/styles/v1/{id}/tiles/{z}/{x}/{y}?access_token={accessToken}', {
  maxZoom: 18,
  id: 'itsdanq/ckslkmiid8q7j17ocziio7t46', // this is the style I defined for my map, using Mapbox
  tileSize: 512,
  zoomOffset: -1,
  accessToken: '...' // put your access token here if you need one!
}).addTo(map);
// fetch the heatmap JSON and render the heatmap
fetch('heat.json').then(r=>r.json()).then(json=>{
  let heatmapLayer = new HeatmapOverlay({
    "radius": parseFloat(document.querySelector('#radius').value),
    "scaleRadius": true,
    "useLocalExtrema": true,
  });
  heatmapLayer.setData({ data: json });
  heatmapLayer.addTo(map);
});

That’s basically all there is to it!

Holidays in the Age of COVID

We’ve missed out on or delayed a number of trips and holidays over the last year and a half for, you know, pandemic-related reasons. So this summer, in addition to our trip to Lichfield, we arranged a series of back-to-back expeditions.

1. Alton Towers

The first leg of our holiday saw us spend a long weekend at Alton Towers, staying over at one of their themed hotels in between days at the water park and theme park:

2. Darwin Forest

The second leg of our holiday took us to a log cabin in the Darwin Forest Country Park for a week:

3. Preston

Kicking off the second week of our holiday, we crossed the Pennines to Preston to hang out with my family (with the exception of JTA, who had work to do back in Oxfordshire that he needed to return to):

4. Forest of Bowland

Ruth and I then left the kids with my mother and sisters for a few days to take an “anniversary mini-break” of glamping in the gorgeous Forest of Bowland:

(If you’re interested in Steve Taylor’s bathtub-carrying virtual-Everest expedition, here’s his Facebook page and JustGiving profile.)

5. Meanwhile, in Preston

The children, back in Preston, were apparently having a whale of a time:

6. Suddenly, A Ping

The plan from this point was simple: Ruth and I would return to Preston for a few days, hang out with my family some more, and eventually make a leisurely return to Oxfordshire. But it wasn’t to be…

Screenshot from the NHS Covid App: "You need to self-isolate."
Well that’s not the kind of message you want to get from your phone.

I got a “ping”. What that means is that my phone was in close proximity to somebody else’s phone on 29 August and that other person subsequently tested positive for COVID-19.

My risk from this contact is exceptionally low. There’s only one place that my phone was in close proximity to the phone of anybody else outside of my immediate family, that day, and it’s when I left it in a locker at the swimming pool near our cabin in the Darwin Forest. Also, of course, I’d been double-jabbed for a month and a half and I’m more-cautious than most about contact, distance, mask usage etc. But my family are, for their own (good) reasons, more-cautious still, so self-isolating at Preston didn’t look like a possibility for us.

Ruth and Dan in a car, in a car park.
Ruth and I went directly to a drive-through PCR testing facility.

As soon as I got the notification we redirected to the nearest testing facility and both got swabs done. 8 days after possible exposure we ought to have a detectable viral load, if we’ve been infected. But, of course, the tests take a day or so to process, so we still needed to do a socially-distanced pickup of the kids and all their stuff from Preston and turn tail for Oxfordshire immediately, cutting our trip short.

The results would turn up negative, and subsequent tests would confirm that the “ping” was a false positive. And in an ironic twist, heading straight home actually put us closer to an actual COVID case as Ruth’s brother Owen turned out to have contracted the bug at almost exactly the same time and had, while we’d been travelling down the motorway, been working on isolating himself in an annex of the “North wing” of our house for the duration of his quarantine.

Barricade with signs reading "Quarantine: Zombie Outbreak"
I set up a “yellow zone” between Owen’s quarantine area and the rest of the house into which we could throw supplies. And I figured I’d have fun with the signage.

7. Ruth & JTA go to Berwick

Thanks to negative tests and quick action in quarantining Owen, Ruth and JTA were still able to undertake the next part of this three-week holiday period and take their anniversary break (which technically should be later in the year, but who knows what the situation will be by then?) to Berwick-upon-Tweed. That’s their story to tell, if they want to, but the kids and I had fun in their absence:

8. Reunited again

Finally, Ruth and JTA returned from their mini-break and we got to do a few things together as a family again before our extended holiday drew to a close:

9. Back to work?

Tomorrow I’m back at work, and after 23 days “off” I’m honestly not sure I remember what I do for a living any more. Something to do with the Internet, right? Maybe ecommerce?

I’m sure it’ll all come right back to me, at least by the time I’ve read through all the messages and notifications that doubtless await me (I’ve been especially good at the discipline, this break, of not looking at work notifications while I’ve been on holiday; I’m pretty proud of myself.)

But looking back, it’s been a hell of a three weeks. After a year and a half of being pretty-well confined to one place, doing a “grand tour” of so many destinations as a family and getting to do so many new and exciting things has made the break feel even longer than it was. It seems like it must have been months since I last had a Zoom meeting with a work colleague!

For now, though, it’s time to try to get the old brain back into work mode and get back to making the Web a better place!

Dan Q found GC1Y6GZ One Giant Step

This checkin to GC1Y6GZ One Giant Step reflects a geocaching.com log entry. See more of Dan's cache logs.

Greetings from Oxfordshire! I’m staying in the holiday park just East of here as part of the first of a three week holiday (taken in an attempt to “make up” for holidays cancelled over the last year and a half owing to The Situation). I figured it’d be an easy and relatively direct hike from there to here, but I’d not counted on the work underway at the moment by the Forestry Commission! Several diverted footpaths later I finally found this “giant step”! Took Paul the Seahorse TB, SL, TFTC.

Map of 53.180133,-1.566933

Lichfield

We took a family trip up to Lichfield this weekend. I don’t know if I can give a “review” of a city-break as a whole, but if I can: I give you five stars, Lichfield.

Dan in front of Lichfield Cathedral, early on Sunday morning.
It’s got a cathedral, which is quite pretty.

Maybe it’s just because we’ve none of us had a night away from The Green… pretty-much since we moved in, last year. But there was something magical about doing things reminiscent of the “old normal”.

Dan and the kids in a bed at a hotel.
“I’m so excited! We get to stay… at a Premier Inn!” At first I rolled my eyes at this joyous line from our 4-year-old (I mean… it’s just a Premier Inn…), but it did feel good to go somewhere and do something.

It’s not that like wasn’t plenty of mask-wearing and social distancing and hand sanitiser and everything that we’ve gotten used to now: there certainly was. The magic, though, came from getting to do an expedition further away from home than we’re used to. And, perhaps, with that happening to coincide with glorious weather and fun times.

A balloon artist wearing a unicorn on her head makes sculptures for children.
Socially-distanced balloon modelling turns out to work, not least because you can hand one of those long balloons to somebody without getting anywhere near them.

We spent an unimaginably hot summer’s day watching an outdoor interpretation of Peter and the Wolf, which each of the little ones has learned about in reasonable depth, at some point or another, as part of the (fantastic) “Monkey Music” classes of which they’re now both graduates.

Ruth and John sit on a picnic blanket in a painted circle; the maquee for the band is behind them.
So long as you weren’t staring at the painted circles on the grass – for corralling families apart from one another – you’d easily forget how unusual things are, right now.

And maybe it’s that they’ve been out-of-action for so long and are only just beginning to once again ramp up… or maybe I’ve just forgotten what the hospitality industry is like?… but man, we felt well-looked after.

From the staff at the hotel who despite the clear challenges of running their establishment under the necessary restrictions still went the extra mile to make the kids feel special to the restaurant we went to that pulled out all the stops to give us all a great evening, I basically came out of the thing with the impression of Lichfield as a really nice place.

Dan in Lichfield city centre, deserted early on a Sunday morning.
Take social distancing to the next level: do your urban geocaching at the crack of dawn.

I’m not saying that it was perfect. A combination of the intolerable heat (or else the desiccating effect of the air conditioner) and a mattress that sagged with two adults on it meant that I didn’t sleep much on Saturday night (although that did mean I could get up at 5am for a geocaching expedition around the city before it got too hot later on). And an hour and a half of driving to get to a place where you’re going to see a one-hour show feels long, especially in this age where I don’t really travel anywhere, ever.

But that’s not the point.

Ruth and the kids eat breakfast
The buffet was closed, of course, but these kids were made for an “all you can eat” breakfast.

The point is that Lichfield made me happy, this weekend. And I don’t know how much of that is that it’s just a nice place and how much is that I’ve missed going anywhere or doing anything, but either way, it lead to a delightful weekend.

Retrohashing expedition 2012 02 19 51 -1

This checkin to geohash 2012-02-19 51 -1 reflects a geohashing expedition. See more of Dan's hash logs.

Location

Field behind Hill Barn, near the Gom’s Hole public footpath, in the valley beneath the hamlet of Clapton-on-the-Hill. About 4km outside the village of Bourton-on-the-Water, Gloucestershire.

(Retro) Participants

  • Dan Q (as a retrohash on the same date but 19 years later, on 2021-02-19)

(Retro) Plans

On the second anniversary of the death of my father, a man who loved to get out into the world and get lost, I undertook my first geohashing expedition. As this seemed to be a good way to remember him I decided to repeat the experience on this, the ninth anniversary of his death, but the actual hashpoints for the day didn’t look interesting… so I opted to make my way to what would have been my nearest hashpoint on the day he died.

(Retro) Expedition

The weather looked horrible and the COVID lockdown (and working from home in general in recent years) has put me out of practice at cycling, so I thought a 40-50 mile round trip through the rolling hills of the Cotswolds was just the thing. This may have been a mistake, as my aching legs were able to testify for several days.

Cycling through Witney, over the hills behind Burford, and then across the Windrush valley and into Gloucestershire was a long, arduous, and damp journey, but what really got me was the wind picking up in the afternoon and giving me a headwind to fight against all the way back home.

Near the hashpoint I was able to lock my bike up at the junction between Sherbourne Street and Bourton Hill – a place shown on my map as “Gom’s Hole” which sounds exactly like what a D&D dungeon master would have a goblin would name his bar. From there I followed the footpath towards Farringdon. As the hashpoint drew closer I began to suspect that it would be unreachable: tall walls, fences, and hedges stood on both sides of the (flooded) footpath, but at the last minute they gave way to wide meadows. I turned off the path and crossed a dyke to the hashpoint, where I had a great view of hares and deer in the valley below. Minutes later, the owner of Hill Barn came over with her dog and asked what I was doing around the back of her land and why I was taking pictures, so I explained that I’d strayed from the footpath (true) because my GPS had told me too (technically true) but I was heading back down to what I could see was the path, now (true, if misleading).

She continued to watch me all the way back to my bike, so I changed my plans (which had been to eat a sandwich lunch and drink a pint of Guinness: my dad’s beer of choice) near the hashpoint and instead I cycled away to a nearby layby to have my lunch.

After a 48.3 mile round trip I got back home aching and exhausted, but pleased to have made it to this damp hashpoint.

(Retro) Tracklog

GPX tracklog: Track 2021-02-19 RETROHASH 2012.gpx

(Retro) Photos

Map of 51.8569418,-1.7795905

The Diamonds, The Dagger and One Classy Dame

On account of the pandemic, I’d expected my fortieth birthday to be a somewhat more-muted affair than I’d hoped. I had a banner, I got trolled by bagels, and I received as a gift a pizza oven with which I immediately set fire to several pieces of cookware, but I hadn’t expected to be able to do anything like the “surprise” party of my thirtieth, and that saddened me a little. So imagine my surprise when I come back from an evening walk the day after my birthday to discover than an actual (remote) surprise party really had been arranged without my knowing!

Matt, Suz, Alec, Jen, Dermot and Doreen on a Google Meet screen.
“Hello, remote guests! What are you doing here?”

Not content with merely getting a few folks together for drinks, though, Ruth and team had gone to great trouble (involving lots of use of the postal service) arranging a “kit” murder mystery party in the Inspector McClue series – The Diamonds, The Dagger, and One Classy Dame – for us all to play. The story is sort-of a spiritual successor to The Brie, The Bullet, and The Black Cat, which we’d played fifteen years earlier. Minor spoilers follow.

JTA (wearing a string of pearls) and Robin (wearing sunglasses)
“Hello, local guests. Wait… why are you all in costumes…?”

Naturally, I immediately felt underdressed, having not been instructed that I might need a costume, and underprepared, having only just heard for the first time that I would be playing the part of German security sidekick Lieutenant Kurt Von Strohm minutes before I had to attempt my most outrageous German accent.

Dan with his tongue out holding a glass of champagne.
Fortunately I was able to quickly imbibe a few glasses of champagne and quickly get into the spirit. Hic.

The plot gave me in particular a certain sense of deja vu. In The Brie, The Bullet, and The Black Cat, I played a French nightclub owner who later turned out to be an English secret agent supplying the French Resistance with information. But in The Diamonds, The Dagger, and One Classy Dame I played a Gestapo officer who… also later turned out to be an English secret agent infiltrating the regime and, you guessed it, supplying the French Resistance.

Jen drinking from the neck of a nearly-empty wine bottle.
As she had previously with Sour Grapes, Ruth had worked to ensure that a “care package” had reached each murder mystery guest. Why yes, it was a boozy care package.

It was not the smoothest nor the most-sophisticated “kit” murder mystery we’ve enjoyed. The technology made communication challenging, the reveal was less-satisfying than some others etc. But the company was excellent. (And the acting way pretty good too, especially by our murderer whose character was exquisitely played.)

JTA downing a Jeroboam of champagne.
The largest bottle, though, was with us: we opened the Jeroboam of champagne Ruth and JTA had been saving from their anniversary (they have a tradition involving increasing sizes of bottle; it’s a whole thing; I’ll leave them to write about it someday).

And of course the whole thing quickly descended into a delightful shouting match with accusations flying left, right, and centre and nobody having a clue what was going on. Like all of our murder mystery parties!

Google Meet transcript with the words "You are the Jewish", which nobody said.
I’m not sure how I feel about Google Meet’s automatic transcription feature. It was generally pretty accurate, but it repeatedly thought that it heard the word “Jewish” being spoken by those of us who were putting on German accents, even though none of us said that.

In summary, the weekend of my fortieth birthday was made immeasurably better by getting to hang out with (and play a stupid game with) some of my friends despite the lockdown, and I’m ever so grateful that those closest to me were able to make such a thing happen (and without me even noticing in advance).

COVID Ipsum

So I made a COVID conspiracy theory-themed lorem ipsum generator:

I blame my friend Bryn, who put the idea into my head while he was coming up with fake COVID conspiracy theories (I realise this sentence makes it sound like there are real COVID conspiracy theories) on a WhatsApp group we’re both in:

WhatsApp conversation: Bryn says that it's easy to come up with COVID conspiracy theories, Dan says somebody should make a Lorem Ipsum generator based on them.
This is about the minimum level of encouragement I need to do just about anything in tech.

It’s implemented using perchance, a platform for creating random text generators that I’ve been playing with – sometimes with the kids – lately. It’s really easy to use and provides a kind of instant-satisfaction that I think is important if you want to inspire the next generation of software engineers. This means, among other things, that you can clone, edit, and mashup my tool: perhaps you can make it better! Or perhaps you’ll use perchance to write some fiction, or poetry, or something else entirely. But regardless, I’d encourage you to have a play.

Mostly my generator comes up with meaningless gibberish, nonsense, and laughable claims. So it’s marginally more-trustworthy than your typical COVID conspiracy theorist.

Staying Sane with GEMSAW

I’ve been having a tough time these last few months. Thanks to COVID, I’m sure I’m not alone in that.

Times are strange, and even when you get a handle on how they’re strange they can still affect you: lockdown stress can quickly magnify anything else you’re already going through.

We’ve all come up with our own coping strategies; here’s part of mine.

JTA, Dan and Ruth shopping for a Christmas tree, wearing face masks
Only people who are highly-allergic to pine needles normally look like this when they’re shopping for a Christmas tree.

These last few months have occasionally seen me as emotionally low as… well, a particularly tough spell a decade ago. But this time around I’ve benefited from the self-awareness and experience to put some solid self-care into practice!

By way partly of self-accountability and partly of sharing what works for me, let me tell you about the silly mnemonic that reminds me what I need to keep track of as part of each day: GEMSAW! (With thanks to Amy Blankson for, among other things, the idea of this kind of acronym.)

Because it’s me, I’ve cited a few relevant academic sources for you in my summary, below:

  • Gratitude
    Taking the time to stop and acknowledge the good things in your life, however small, is associated with lower stress levels (Taylor, Lyubomirsky & Stein, 2017) to a degree that can’t just be explained by the placebo effect (Cregg & Cheavens, 2020).
    Frankly, the placebo effect would be fine, but it’s nice to have my practice of trying to intentionally recognise something good in each day validated by the science too!
  • Exercise
    I don’t even need a citation; I’m sure everybody knows that aerobic exercise is associated with reduced risk and severity of depression: the biggest problem comes from the fact that it’s an exceptionally hard thing to motivate yourself to do if you’re already struggling mentally!
    But it turns out you don’t need much to start to see the benefits (Josefsson, Lindwall & Archer, 2014): I try to do enough to elevate my heart rate each day, but that’s usually nothing more than elevating my desk to standing height, putting some headphones on, and dancing while I work!
Dan dancing at his desk (animated GIF)
Warming up. Things only get nuts when the bass drops, but I’ll spare you having to watch that.
  • Meditation/Mindfulness
    Understandably a bit fuzzier as a concept and tainted by being a “hip” concept. A short meditation break or mindfulness exercise might be verifiably therapeutic, but more (non-terrible) studies are needed (Vonderlin, Biermann, Bohus & Lyssenko 2020). For me, a 2-5 minute meditation break punctuates a day and feels like it contributes towards the goal of staying-sane-in-challenging-times, so it makes it into my wellbeing plan.
    Maybe it’s doing nothing. But I’m not losing much time over it so I’m not worried.
  • Sunlight
    During my 20s I gradually began to suffer more and more from “winter blues”. Nobody’s managed to make an argument for the underlying cause of seasonal affective disorder that hasn’t been equally-well debunked by some other study. Small-scale studies often justify light therapy (e.g. Lam, Levitan & Morehouse 2006) but it’s possibly no-more-effective than a placebo at scale (SBU 2007).
    Since my early 30s, I’ve always felt better to get myself 30 minutes of lightbox on winter mornings (I use one of these bad boys). I admit it’s possible that the benefits are just the result of tricking my brain into waking-up more promptly and therefore feeing like I’m being more-productive with my waking hours! But either way, getting some sunlight – whether natural or artificial – makes me feel better, so it makes it onto my daily self-care checklist.
Bright sunlight in an almost-cloudless blue sky.
10 minutes of overhead, unoccluded sunlight is the minimum therapeutic dose. That translates to about 30 minutes of winter sun at my latitude or 10,000 lux full-spectrum sunlamp.
  • Acts of kindness
    It’s probably not surprising that a person’s overall happiness correlates with their propensity for kindness (Lyubomirsky, King & Diener 2005). But what’s more interesting is that the causal link can be “gamed”. That is: a deliberate effort to engage in acts of kindness results in increased happiness (Buchanan & Bardi 2010)!
    Beneficial acts of kindness can be as little as taking the time to acknowledge somebody’s contribution or compliment somebody’s efforts. The amount of effort it takes is far less-important for happiness than the novelty of the experience, so the type of kindness you show needs to be mixed-up a bit to get the best out of it. But demonstrating kindness helps to make the world a better place for other humans, so it pays off even if you’re coming from a fully utilitarian perspective.
  • Writing
    I write a lot anyway, often right here, and that’s very-definitely for my own benefit first and foremost. But off the back of some valuable “writing therapy” (Baikie & Wilhelm 2005) I undertook earlier this year, I’ve been continuing with the simpler, lighter approach of trying to no more than three sentences about something that’s had an impact on me that day.
    As an approach, it doesn’t help everybody (Zachariae 2015), but writing a little about your day – not even about how you feel about it, just the facts will do (Koschwanez, Robinson, Beban, MacCormick, Hill, Windsor, Booth, Jüllig & Broadbent 2017; fuck me that’s a lot of co-authors) – helps to keep you content, and I’m loving it.

Despite the catchy acronym (Do I need to come up with a GEMSAW logo? I’m pretty sure real gemcutting is actually more of a grinding process…) and stack of references, I’m not actually writing a self-help book; it just sounds like I am.

I don’t claim to be an authority on anything beyond my own head, and I’m not very confident on that subject! I just wanted to share with you something that’s been working pretty well at keeping me sane for the last month or two, just in case it’s of any use to you. These are challenging times; do what you need to find the happiness you can, and hang in there.

Watching Films Together… Apart

This weekend I announced and then hosted Homa Night II, an effort to use technology to help bridge the chasms that’ve formed between my diaspora of friends as a result mostly of COVID. To a lesser extent we’ve been made to feel distant from one another for a while as a result of our very diverse locations and lifestyles, but the resulting isolation was certainly compounded by lockdowns and quarantines.

Mark, Sian, Alec, Paul, Kit, Adam, Dan and Claire at Troma Night V.
Long gone are the days when I could put up a blog post to say “Troma Night tonight?” and expect half a dozen friends to turn up at my house.

Back in the day we used to have a regular weekly film night called Troma Night, named after the studio who dominated our early events and whose… genre… influenced many of our choices thereafter. We had over 300 such film nights, by my count, before I eventually left our shared hometown of Aberystwyth ten years ago. I wasn’t the last one of the Troma Night regulars to leave town, but more left before me than after.

Sour Grapes: participants share "hearts" with Ruth
Observant readers will spot a previous effort I made this year at hosting a party online.

Earlier this year I hosted Sour Grapes, a murder mystery party (an irregular highlight of our Aberystwyth social calendar, with thanks to Ruth) run entirely online using a mixture of video chat and “second screen” technologies. In some ways that could be seen as the predecessor to Homa Night, although I’d come up with most of the underlying technology to make Homa Night possible on a whim much earlier in the year!

WhatsApp chat: Dan proposes "Troma Night Remote"; Matt suggests calling it "Troma at Homa"; Dan settles on "Homa Night".
The idea spun out of a few conversations on WhatsApp but the final name – Homa Night – wasn’t agreed until early in November.

How best to make such a thing happen? When I first started thinking about it, during the first of the UK’s lockdowns, I considered a few options:

  • Streaming video over a telemeeting service (Zoom, Google Meet, etc.)
    Very simple to set up, but the quality – as anybody who’s tried this before will attest – is appalling. Being optimised for speech rather than music and sound effects gives the audio a flat, scratchy sound, video compression artefacts that are tolerable when you’re chatting to your boss are really annoying when they stop you reading a crucial subtitle, audio and video often get desynchronised in a way that’s frankly infuriating, and everybody’s download speed is limited by the upload speed of the host, among other issues. The major benefit of these platforms – full-duplex audio – is destroyed by feedback so everybody needs to stay muted while watching anyway. No thanks!
  • Teleparty or a similar tool
    Teleparty (formerly Netflix Party, but it now supports more services) is a pretty clever way to get almost exactly what I want: synchronised video streaming plus chat alongside. But it only works on Chrome (and some related browsers) and doesn’t work on tablets, web-enabled TVs, etc., which would exclude some of my friends. Everybody requires an account on the service you’re streaming from, potentially further limiting usability, and that also means you’re strictly limited to the media available on those platforms (and further limited again if your party spans multiple geographic distribution regions for that service). There’s definitely things I can learn from Teleparty, but it’s not the right tool for Homa Night.
  • “Press play… now!”
    The relatively low-tech solution might have been to distribute video files in advance, have people download them, and get everybody to press “play” at the same time! That’s at least slightly less-convenient because people can’t just “turn up”, they have to plan their attendance and set up in advance, but it would certainly have worked and I seriously considered it. There are other downsides, though: if anybody has a technical issue and needs to e.g. restart their player then they’re basically doomed in any attempt to get back in-sync again. We can do better…
  • A custom-made synchronised streaming service…?
Homa Night architecture: S3 delivers static content to browsers, browsers exchange real-time information via Firebase.
A custom solution that leveraged existing infrastructure for the “hard bits” proved to be the right answer.

So obviously I ended up implementing my own streaming service. It wasn’t even that hard. In case you want to try your own, here’s how I did it:

Media preparation

First, I used Adobe Premiere to create a video file containing both of the night’s films, bookended and separated by “filler” content to provide an introduction/lobby, an intermission, and a closing “you should have stopped watching by now” message. I made sure that the “intro” was a nice round duration (90s) and suitable for looping because I planned to hold people there until we were all ready to start the film. Thanks to Boris & Oliver for the background music!

Dan uses a green screen to add to the intermission.
Honestly, the intermission was just an excuse to keep my chroma key gear out following its most-recent use.

Next, I ran the output through Handbrake to produce “web optimized” versions in 1080p and 720p output sizes. “Web optimized” in this case means that metadata gets added to the start of the file to allow it to start playing without downloading the entire file (streaming) and to allow the calculation of what-part-of-the-file corresponds to what-part-of-the-timeline: the latter, when coupled with a suitable webserver, allows browsers to “skip” to any point in the video without having to watch the intervening part. Naturally I’m encoding with H.264 for the widest possible compatibility.

Handbrake preparing to transcode Premiere's output.
Even using my multi-GPU computer for the transcoding I had time to get up and walk around a bit.

Real-Time Synchronisation

To keep everybody’s viewing experience in-sync, I set up a Firebase account for the application: Firebase provides an easy-to-use Websockets platform with built-in data synchronisation. Ignoring the authentication and chat features, there wasn’t much shared here: just the currentTime of the video in seconds, whether or not introMode was engaged (i.e. everybody should loop the first 90 seconds, for now), and whether or not the video was paused:

Firebase database showing shared currentTime, introMode, and paused values.
Firebase makes schemaless real-time databases pretty easy.

To reduce development effort, I never got around to implementing an administrative front-end; I just manually went into the Firebase database and acknowledged “my” computer as being an administrator, after I’d connected to it, and then ran a little Javascript in my browser’s debugger to tell it to start pushing my video’s currentTime to the server every few seconds. Anything else I needed to edit I just edited directly from the Firebase interface.

Other web clients’ had Javascript to instruct them to monitor these variables from the Firebase database and, if they were desynchronised by more than 5 seconds, “jump” to the correct point in the video file. The hard part of the code… wasn’t really that hard:

// Rewind if we're passed the end of the intro loop
function introModeLoopCheck() {
  if (!introMode) return;
  if (video.currentTime > introDuration) video.currentTime = 0;
}

function fixPlayStatus() {
  // Handle "intro loop" mode
  if (remotelyControlled && introMode) {
    if (video.paused) video.play(); // always play
    introModeLoopCheck();
    return; // don't look at the rest
  }

  // Fix current time
  const desync = Math.abs(lastCurrentTime - video.currentTime);
  if (
    (video.paused && desync > DESYNC_TOLERANCE_WHEN_PAUSED) ||
    (!video.paused && desync > DESYNC_TOLERANCE_WHEN_PLAYING)
  ) {
    video.currentTime = lastCurrentTime;
  }
  // Fix play status
  if (remotelyControlled) {
    if (lastPaused && !video.paused) {
      video.pause();
    } else if (!lastPaused && video.paused) {
      video.play();
    }
  }
  // Show/hide paused notification
  updatePausedNotification();
}

Web front-end

Finally, there needed to be a web page everybody could go to to get access to this. As I was hosting the video on S3+CloudFront anyway, I put the HTML/CSS/JS there too.

Configuring a Homa Night video player.
I decided to carry the background theme of the video through to the web interface too.

I tested in Firefox, Edge, Chrome, and Safari on desktop, and (slightly less) on Firefox, Chrome and Safari on mobile. There were a few quirks to work around, mostly to do with browsers not letting videos make sound until the page has been interacted with after the video element has been rendered, which I carefully worked-around by putting a popup “over” the video to “enable sync”, but mostly it “just worked”.

Delivery

On the night I shared the web address and we kicked off! There were a few hiccups as some people’s browsers got disconnected early on and tried to start playing the film before it was time, and one of these even when fixed ran about a minute behind the others, leading to minor spoilers leaking via the rest of us riffing about them! But on the whole, it worked. I’ve had lots of useful feedback to improve on it for the next version, and I might even try to tidy up my code a bit and open-source the results if this kind of thing might be useful to anybody else.

COVID-19 and Acedia

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Moving around is what we do as creatures, and for that we need horizons. COVID-19 has erased many of the spatial and temporal horizons we rely on, even if we don’t notice them very often. We don’t know how the economy will look, how social life will go on, how our home routines will be changed, how work will be organized, how universities or the arts or local commerce will survive.

What unsettles us is not only fear of change. It’s that, if we can no longer trust in the future, many things become irrelevant, retrospectively pointless. And by that we mean from the perspective of a future whose basic shape we can no longer take for granted. This fundamentally disrupts how we weigh the value of what we are doing right now. It becomes especially hard under these conditions to hold on to the value in activities that, by their very nature, are future-directed, such as education or institution-building.

That’s what many of us are feeling. That’s today’s acedia.

In a blog post far from his usual topics, Schneier shares a word – albeit an arguably-archaic one! – that captures the feeling of listlessness that many of us are experiencing as the coronavirus pandemic continues to unfold.

Automattic Retrospective (days 207 to 334)

Last year, I accepted a job offer with Automattic and I’ve been writing about it every 128 days. I’ve talked about my recruitment, induction, and experience of lockdown (which in turn inspired a post about the future of work). I’ve even helped enthuse other new Automatticians! Since my last post I’ve moved house so my home office has changed shape, but I’m still plodding along as always… and fast-approaching my first “Automattic birthday”! (This post ran a little late; the 128-day block was three weeks ago!)

Dan in his home office (links to an interactive 360° panoramic photo with info points).
If you missed it the first time around, click through to explore an interactive panoramic view of my workspace. It’s slightly more “unpacked” now.

As I approach my first full year as an Automattician, I find myself looking back on everything I’ve learned… but also looking around at all the things I still don’t understand! I’m not learning something new every day any more… but I’m still learning something new most weeks.

This summer I’ve been getting up-close and personal with Gutenberg components. I’d mostly managed to avoid learning the React (eww; JSX, bad documentation, and an elephantine payload…) necessary to hack Gutenberg, but in helping to implement new tools for WooCommerce.com I’ve discovered that it’s… not quite as painful as I’d thought. There are even some bits I quite like. But I don’t expect to fall in love with React any time soon. This autumn I’ve been mostly working on search and personalisation, integrating customer analytics data with our marketplace to help understand what people look for on our sites and using that to guide their future experience (and that of others “like” them). There’s always something new.

Alpha project planning meeting via Zoom.
I suppose that by now everybody‘s used to meetings that look like this, but when I first started at Automattic a year ago they were less-commonplace.

My team continues to grow, with two newmatticians this month and a third starting in January. In fact, my team’s planning to fork into two closely-linked subteams; one with a focus on customers and vendors, the other geared towards infrastructure. It’s exciting to see my role grow and change, but I worry about the risk of gradually pigeon-holing myself into an increasingly narrow specialisation. Which wouldn’t suit me: I like to keep a finger in all the pies. Still; my manager’s reassuring that this isn’t likely to be the case and our plans are going in the “right” direction.

Kudos to Dan "for resolving a weeks worth of project issues in one day".
Our “Kudos” system can be used to acknowledge other Automatticians going above and beyond. I was particularly proud of this one.

On the side of my various project work, I’ve occasionally found the opportunity for more-creative things. Last month, I did some data-mining over the company’s “kudos” history of the last five years and ran it through vis.js to try to find a new angle on understanding how Automattic’s staff, teams, and divisions interact with one another. It lead to some interesting results: panning through time, for example, you can see the separate island of Tumblr staff who joined us during the acquisition gradually become more-interconnected with the rest of the organisation over the course of the last year.

Automattic Kudos social graph for September 2020
Automattic as a social graph of kudos given/received during September 2020, colour-coded by team. Were you one of us, you’d be able to zoom in and find yourself. The large “branch” in the bottom right is mostly comprised of Tumblr staff.

The biggest disappointment of my time at Automattic so far was that I’ve not managed to go to a GM! The 2019 one – which looked awesome – took place only a couple of weeks before my contract started (despite my best efforts to wrangle my contract dates with the Bodleian and Automattic to try to work around that), but people reassured me that it was okay because I’d make it to the next one. Well.. 2020 makes fools of us all, I guess, because of course there’s no in-person GM this year. Maybe, hopefully, if and when the world goes back to normal I’ll get to spend time in-person with my colleagues once in a while… but for now, we’re having to suffice with Internet-based socialisation only, just like the rest of the world.

I got married and had kids so you don’t have to

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

I’m sure that the graveyard of over-optimism is littered with the corpses of parents who planned to help their children learn self-moderation by showing them the wonders of nature, but who realized too late that fields of wheat don’t stand a chance against Rocket League. I’m hoping that we can agree that computer games are good, but other things are good too, cf fields of wheat. I don’t want to have to sneak in my own gaming time after my son has gone to bed. I also don’t want to be a hypocrite; at least, I don’t want Oscar to know that I’m a hypocrite. Maybe we can play together and use it as father-son bonding time. This might work until he’s ten and after he’s twenty-five.

Robert Heaton, of Programming Projects for Advanced Beginners fame and reverse-engineering device drivers that spy on you (which I’ve talked about before), has also been blogging lately about his experience of Dadding, with the same dry/sarcastic tone you might be used to. This long post is a great example of the meandering thoughts of a (techie) parent in these (interesting) times, and it’s good enough for that alone. But it’s the raw, genuine “honesty and dark thoughts” section towards the end of the article that really makes it stand out.

The Unexpected Solace in Learning to Play Piano

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

…while I practice, I have to simultaneously read, listen, think, translate. Every synapse of my brain is so utterly overwhelmed, there is no capacity left to think about the world out there.

When Christoph Niemann published this piece about learning to play the piano during the most-lockdown-y parts of the Coronavirus lockdown, it rang a chord with me (hah!). I, too, have experimented with learning to play the piano this spring/summer, and found a similar kind of Zen-like focussed calm emerge out of the frustration of staring at a piece of sheet music and wondering why I couldn’t for the life of me get me fingers to remember to do when they got to that point.

I started out with – after following some random links off the back of finishing the last bit of work for my recent masters degree – a free course in music theory by the OU, because I figured that coming in from a theoretical perspective would help with the way my brain thinks about this kind of thing. I supplemented that with a book we got for the kids to use to learn to play, and now I’ve now graduated to very gradually hunt-and-pecking my way through Disney’s back catalogue. I can play Go The Distance, Colors of the Wind and most of Can You Feel The Love Tonight barely well enough that I don’t feel the need to tear my own ears off, so I guess I’m making progress, though I still fall over my own hands every time I try to play any bloody thing from Moana. 20 minutes at a time, here and there, and I’m getting there. I don’t expect to ever be good at it, but I’m enjoying it nonetheless.

But anyway: this piece in the NYT Magazine really spoke to me, and to hear that somebody with far more music experience than me can struggle with all the same things I do when getting started with the piano was really reassuring.