When do I #blog? A breakdown of my top four #indieweb content types – articles, reposts, checkins and notes – by month.

Blog posts by type and month, a graph showing how articles barely change throughout the year but reposts are highest in the first half of the year, checkins peak in the spring and summer, and notes get a big boost in November.

Unsurprisingly my checkins, which represent #geocaching/#geohashing activity, grow in the spring and peak in the summer when the weather’s better!

At first I assumed the notes peak in November might have been thrown off by a single conference, e.g. musetech, but it turns out I’ve just done more note-friendly things in Novembers, like Challenge Robin II and my Cape Town meetup, which are enough to throw the numbers off.

Heatmapping my Movements

As I mentioned last year, for several years I’ve collected pretty complete historic location data from GPSr devices I carry with me everywhere, which I collate in a personal μlogger server.

Going back further, I’ve got somewhat-spotty data going back a decade, thanks mostly to the fact that I didn’t get around to opting-out of Google’s location tracking until only a few years ago (this data is now also housed in μlogger). More-recently, I now also get tracklogs from my smartwatch, so I’m managing to collate more personal location data than ever before.

Inspired perhaps at least a little by Aaron Parecki, I thought I’d try to do something cool with it.

Heatmapping my movements

The last year

Heatmap showing Dan's movements around Oxford since moving house in 2020. There's a strong cluster around Stanton Harcourt with heavy tendrils around Witney and Eynsham and along the A40 to Summertown, and lighter tendrils around North and Central Oxford.
My movements over the last year have been relatively local, but there are some interesting hotspots and common routes.

What you’re looking at is a heatmap showing my location over the last year or so since I moved to The Green. Between the pandemic and switching a few months prior to a job that I do almost-entirely at home there’s not a lot of travel showing, but there’s some. Points of interest include:

  • The blob around my house, plus some of the most common routes I take to e.g. walk or cycle the children to school.
  • A handful of my favourite local walking and cycling routes, some of which stand out very well: e.g. the “loop” just below the big blob represents a walk around the lake at Dix Pit; the blob on its right is the Devils Quoits, a stone circle and henge that I thought were sufficiently interesting that I made a virtual geocache out of them.
  • The most common highways I spend time on: two roads into Witney, the road into and around Eynsham, and routes to places in Woodstock and North Oxford where the kids have often had classes/activities.
  • I’ve unsurprisingly spent very little time in Oxford City Centre, but when I have it’s most often been at the Westgate Shopping Centre, on the roof of which is one of the kids’ favourite restaurants (and which we’ve been able to go to again as Covid restrictions have lifted, not least thanks to their outdoor seating!).

One to eight years ago

Let’s go back to the 7 years prior, when I lived in Kidlington. This paints a different picture:

Heatmap showing Dan's movements around Kidlington, including a lot of time in the village and in Oxford City Centre, as well as hotspots at the hospital, parks, swimming pools, and places that Dan used to volunteer. Individual expeditions can also be identified.
For the seven years I lived in Kidlington I moved around a lot more than I have since: each hotspot tells a story, and some tell a few.

This heatmap highlights some of the ways in which my life was quite different. For example:

  • Most of my time was spent in my village, but it was a lot larger than the hamlet I live in now and this shows in the size of my local “blob”. It’s also possible to pick out common destinations like the kids’ nursery and (later) school, the parks, and the routes to e.g. ballet classes, music classes, and other kid-focussed hotspots.
  • I worked at the Bodleian from early 2011 until late in 2019, and so I spent a lot of time in Oxford City Centre and cycling up and down the roads connecting my home to my workplace: Banbury Road glows the brightest, but I spent some time on Woodstock Road too.
  • For some of this period I still volunteered with Samaritans in Oxford, and their branch – among other volunteering hotspots – show up among my movements. Even without zooming in it’s also possible to make out individual venues I visited: pubs, a cinema, woodland and riverside walks, swimming pools etc.
  • Less-happily, it’s also obvious from the map that I spent a significant amount of time at the John Radcliffe Hospital, an unpleasant reminder of some challenging times from that chapter of our lives.
  • The data’s visibly “spottier” here, mostly because I built the heatmap only out of the spatial data over the time period, and not over the full tracklogs (i.e. the map it doesn’t concern itself with the movement between two sampled points, even where that movement is very-guessable), and some of the data comes from less-frequently-sampled sources like Google.

Eight to ten years ago

Let’s go back further:

Heatmap showing Dan's movements around Oxford during the period he lived in Kennington. Again, it's dominated by time at home, in the city centre, and commuting between the two.
Back when I lived in Kennington I moved around a lot less than I would come to later on (although again, the spottiness of the data makes that look more-significant than it is).

Before 2011, and before we bought our first house, I spent a couple of years living in Kennington, to the South of Oxford. Looking at this heatmap, you’ll see:

  • I travelled a lot less. At the time, I didn’t have easy access to a car and – not having started my counselling qualification yet – I didn’t even rent one to drive around very often. You can see my commute up the cyclepath through Hinksey into the City Centre, and you can even make out the outline of Oxford’s Covered Market (where I’d often take my lunch) and a building in Osney Mead where I’d often deliver training courses.
  • Sometimes I’d commute along Abingdon Road, for a change; it’s a thinner line.
  • My volunteering at Samaritans stands out more-clearly, as do specific venues inside Oxford: bars, theatres, and cinemas – it’s the kind of heatmap that screams “this person doesn’t have kids; they can do whatever they like!”

Every map tells a story

I really love maps, and I love the fact that these heatmaps are capable of painting a picture of me and what my life was like in each of these three distinct chapters of my life over the last decade. I also really love that I’m able to collect and use all of the personal data that makes this possible, because it’s also proven useful in answering questions like “How many times did I visit Preston in 2012?”, “Where was this photo taken?”, or “What was the name of that place we had lunch when we got lost during our holiday in Devon?”.

There’s so much value in personal geodata (that’s why unscrupulous companies will try so hard to steal it from you!), but sometimes all you want to do is use it to draw pretty heatmaps. And that’s cool, too.

Heatmap showing Dan's movements around Great Britain for the last 10 years: with a focus on Oxford, tendrils stretch to hotspots in South Wales, London, Cambridge, York, Birmingham, Preston, Glasgow, Edinburgh, and beyond.

How these maps were generated

I have a μlogger instance with the relevant positional data in. I’ve automated my process, but the essence of it if you’d like to try it yourself is as follows:

First, write some SQL to extract all of the position data you need. I round off the latitude and longitude to 5 decimal places to help “cluster” dots for frequency-summing, and I raise the frequency to the power of 3 to help make a clear gradient in my heatmap by making hotspots exponentially-brighter the more popular they are:

SELECT ROUND(latitude, 5) lat, ROUND(longitude, 5) lng, POWER(COUNT(*), 3) `count`
FROM positions
WHERE `time` BETWEEN '2020-06-22' AND '2021-08-22'
GROUP BY ROUND(latitude, 5), ROUND(longitude, 5)

This data needs converting to JSON. I was using Ruby’s mysql2 gem to fetch the data, so I only needed a .to_json call to do the conversion – like this:

db = Mysql2::Client.new(host: ENV['DB_HOST'], username: ENV['DB_USERNAME'], password: ENV['DB_PASSWORD'], database: ENV['DB_DATABASE'])
db.query(sql).to_a.to_json

Approximately following this guide and leveraging my Mapbox subscription for the base map, I then just needed to include leaflet.js, heatmap.js, and leaflet-heatmap.js before writing some JavaScript code like this:

body.innerHTML = '<div id="map"></div>';
let map = L.map('map').setView([51.76, -1.40], 10);
// add the base layer to the map
L.tileLayer('https://api.mapbox.com/styles/v1/{id}/tiles/{z}/{x}/{y}?access_token={accessToken}', {
  maxZoom: 18,
  id: 'itsdanq/ckslkmiid8q7j17ocziio7t46', // this is the style I defined for my map, using Mapbox
  tileSize: 512,
  zoomOffset: -1,
  accessToken: '...' // put your access token here if you need one!
}).addTo(map);
// fetch the heatmap JSON and render the heatmap
fetch('heat.json').then(r=>r.json()).then(json=>{
  let heatmapLayer = new HeatmapOverlay({
    "radius": parseFloat(document.querySelector('#radius').value),
    "scaleRadius": true,
    "useLocalExtrema": true,
  });
  heatmapLayer.setData({ data: json });
  heatmapLayer.addTo(map);
});

That’s basically all there is to it!

It’s 2020 and you’re in the future

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

West Germany’s 1974 World Cup victory happened closer to the first World Cup in 1930 than to today.

The Wonder Years aired from 1988 and 1993 and depicted the years between 1968 and 1973. When I watched the show, it felt like it was set in a time long ago. If a new Wonder Years premiered today, it would cover the years between 2000 and 2005.

Also, remember when Jurassic Park, The Lion King, and Forrest Gump came out in theaters? Closer to the moon landing than today.

These things come around now and again, but I’m not sure of the universal validity of observing that a memorable event is now closer to another memorable event than it is to the present day. I don’t think that the relevance of events is as linear as that. Instead, perhaps, it looks something like this:

Graph showing that recent events matter a lot, but rapidly tail off for a while before levelling out again as they become long-ago events.
Recent events matter more than ancient events to the popular consciousness, all other things being equal, but relative to one another the ancient ones are less-relevant and there’s a steep drop-off somewhere between the two.

Where the drop-off in relevance occurs is hard to pinpoint and it probably varies a lot by the type of event that’s being remembered: nobody seems to care about what damn terrible thing Trump did last month or the month before when there’s some new terrible thing he did just this morning, for example (I haven’t looked at the news yet this morning, but honestly whenever you read this post he’ll probably have done something awful).

Nonetheless, this post on Wait But Why was a fun distraction, even if it’s been done before. Maybe the last time it happened was so long ago it’s irrelevant now?

XKCD 1393: Timeghost - 'Hello, Ghostbusters?' 'ooOOoooo people born years after that movie came out are having a second chiiiild right now ooOoooOoo'
Of course, there’s a relevant XKCD. And it was published closer to the theatrical releases of Cloudy with a Chance of Meatballs and Paranormal Activity than it was to today. OoooOOoooOOoh.

Lots of interesting results from the @bodleianlibs staff survey. Pleased to have my suspicions confirmed about my department’s propensity to be accepting of individuals: it’s the only one where a majority of people strongly agreed with the statement “I feel able to by myself at work” and one of only two where nobody disagreed with it. That feels like an accurate representation of my experience with my team these last 7-8 years!

'I feel able to by myself at work' staff survey results chart showing my department strongly agrees

Going Critical

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

If you’ve spent any time thinking about complex systems, you surely understand the importance of networks.
Networks rule our world. From the chemical reaction pathways inside a cell, to the web of relationships in an ecosystem, to the trade and political networks that shape the course of history.
Or consider this very post you’re reading. You probably found it on a social network, downloaded it from a computer network, and are currently deciphering it with your neural network.
But as much as I’ve thought about networks over the years, I didn’t appreciate (until very recently) the importance of simple diffusion.
This is our topic for today: the way things move and spread, somewhat chaotically, across a network. Some examples to whet the appetite:
  • Infectious diseases jumping from host to host within a population
  • Memes spreading across a follower graph on social media
  • A wildfire breaking out across a landscape
  • Ideas and practices diffusing through a culture
  • Neutrons cascading through a hunk of enriched uranium
A quick note about form.
Unlike all my previous work, this essay is interactive. There will be sliders to pull, buttons to push, and things that dance around on the screen. I’m pretty excited about this, and I hope you are too.
So let’s get to it. Our first order of business is to develop a visual vocabulary for diffusion across networks.

A simple model

I’m sure you all know the basics of a network, i.e., nodes + edges.
To study diffusion, the only thing we need to add is labeling certain nodes as active. Or, as the epidemiologists like to say, infected:
This activation or infection is what will be diffusing across the network. It spreads from node to node according to rules we’ll develop below.
Now, real-world networks are typically far bigger than this simple 7-node network. They’re also far messier. But in order to simplify — we’re building a toy model here — we’re going to look at grid or lattice networks throughout this post.
(What a grid lacks in realism, it makes up for in being easy to draw ;)
Except where otherwise specified, the nodes in our grid will have 4 neighbors, like so:
And we should imagine that these grids extend out infinitely in all directions. In other words, we’re not interested in behavior that happens only at the edges of the network, or as a result of small populations.
Given that grid networks are so regular, we can simplify by drawing them as pixel grids. These two images represent the same network, for example:
Alright, let’s get interactive.

Fabulous (interactive! – click through for the full thing to see for yourself) exploration of network interactions with applications for understanding epidemics, memes, science, fashion, and much more. Plus Kevin’s made the whole thing CC0 so everybody can share and make use of his work. Treat as a longread with some opportunities to play as you go along.

[x-post /r/MegaMegaMegaLounge] [Graph] Tracking ‘engagement’ across the MegaLounges. Why are some Megas more-active than others? (interpretation in comments)

This link was originally posted to /r/MegaMegaMonitor. See more things from Dan's Reddit account.

Engagement level by MegaLounge

In /u/10_9_‘s recent thread about gilding trends, /u/razerxs made an interesting observation: that the huge ‘drop offs’ in membership of MegaLounges after /r/MegaMegaMegaLounge does not correlate with their expectations, based upon the level of activity in /r/MegaLoungeV. /u/razerxs

observed that e.g. /r/MegaLoungeV is a highly-active sub, which isn’t necessarily what you’d expect if the activity level was based entirely upon the number of people permitted to have access.

I started wondering if there might be a better predictor of engagement levels. I experimented by looking at the ratio of how many people ‘subscribe’ to each MegaLounge to how many people are permitted into there. This isn’t a perfect measure of engagement, of course, but my thinking was that of the people who are invited into a MegaLounge, only some of those will add it to their front page… but that those who add it to their front page are more-likely to actively participate.

The graph shows three things. From the left axis, the blue and red lines show the number of people who are allowed into each MegaLounge and the number of people who are subscribed to each MegaLounge. As you might expect, there’s a gap between the two and the gap narrows in higher lounges, where there are fewer people.

But what I was interested in is whether and where this gap changes proportionally: is the “subscription rate” among eligible people higher in particular lounges, and can this been seen as a predictor of activity levels and engagement rates in each lounge? That’s what the green bars show (against the right-hand axis: note that it doesn’t start at zero). In general, across the MegaLounges up to and including /r/MegaLoungeSol, there does seem to be a slight upward trend: i.e. the higher a lounge you’re in, the more-likely that eligible people are to add that lounge to their front page. Beyond /r/MegaLoungeSol the bars jump around all over the place, probably because of the small number of people ‘up there’, and I suggest that we ignore them: accuracy of this as a predictor would be expected to be better where there were more subscribers: say, up to about /r/MegaLoungeDiamond.

What this would predict would be a “lull” at /r/MegaLoungeVIII. I don’t know if that’s your experience or not. And, interestingly, the ‘subscription ratio’ at /r/MegaLoungeV and /r/MegaLoungeX are also unusually low, bucking the overall trend. What does this mean? I don’t know. But if /r/MegaLoungeV really is to be considered one of the more “active” MegaLounges, then I think that we can safely say that my hypothesis – that we might be able to predict activity hotspots by looking at the subscription rate – is not backed up by the data.

Still: interesting stuff.

Just for fun, I’ve pre-generated ascension graphs for everybody in MegaLoungeIndia

This self-post was originally posted to /r/MegaLoungeIndia. See more things from Dan's Reddit account.

As described over there, I’ve come up with a way to make graphs of the speed of everybody’s ascension through the MegaLounges. But because everybody up here in /r/MegaLoungeIndia is far too important to have to ask for things for themselves, I’ve pre-generated graphs for you all. Here they are:

Hope that provides some amusement and diversion to you all.

MegaLoungeVII is the most-generous of the first 15 MegaLounges! (and other interesting statistics) [x-post /r/MegaMegaMonitor]

This self-post was originally posted to /r/MegaLoungeVII. See more things from Dan's Reddit account.

Thanks to the data that /r/MegaMegaMonitor collates, I’ve been able to throw together some fun statistics about the MegaLounge chain. Let me share some graphs with you:

This is all just something fun I threw together while I’ve been off work sick today. If there’s anything else that anybody would like to see extracted from MMM’s data, let me know: it’s all interesting stuff!

Some graphs about the MegaLounge chain

This self-post was originally posted to /r/MegaMegaMonitor. See more things from Dan's Reddit account.

Thanks to the data that MegaMegaMonitor collates, I’ve been able to throw together some fun statistics about the MegaLounge chain. Let me share some graphs with you:

This is all just something fun I threw together while I’ve been off work sick today. If there’s anything else that anybody would like to see extracted from MMM’s data, let me know: it’s all interesting stuff!