Dan Q

Tag: statistics

The Beer Token

Duration

Podcast Version

This post is also available as a podcast. Listen here, download for later, or subscribe wherever you consume podcasts.

This post is also available as a video. If you'd prefer to watch/listen to me talk about this topic, give it a look.

1979

The novelisation of The Hitch-Hiker’s Guide to the Galaxy came out in 1979, just a smidge before I was born. There’s a well-known scene in the second chapter featuring Ford Prefect, an alien  living on Earth, distracting his human friend Arthur Dent. Arthur is concerned about the imminent demolition of his house by a wrecking crew, and Ford takes him to the pub to get him drunk, in anticipation of the pair attempting to hitch a lift on an orbiting spacecraft that’s about to destroy the planet:

“Six pints of bitter,” said Ford Prefect to the barman of the Horse and Groom. “And quickly please, the world’s about to end.”

The barman of the Horse and Groom didn’t deserve this sort of treatment, he was a dignified old man. He pushed his glasses up his nose and blinked at Ford Prefect. Ford ignored him and stared out of the window, so the barman looked instead at Arthur who shrugged helplessly and said nothing.

So the barman said, “Oh yes sir? Nice weather for it,” and started pulling pints.

He tried again.

“Going to watch the match this afternoon then?”

Ford glanced round at him.

“No, no point,” he said, and looked back out of the window.

“What’s that, foregone conclusion then you reckon sir?” said the barman. “Arsenal without a chance?”

“No, no,” said Ford, “it’s just that the world’s about to end.”

“Oh yes sir, so you said,” said the barman, looking over his glasses this time at Arthur. “Lucky escape for Arsenal if it did.”

Ford looked back at him, genuinely surprised.

“No, not really,” he said. He frowned.

The barman breathed in heavily. “There you are sir, six pints,” he said.

Arthur smiled at him wanly and shrugged again. He turned and smiled wanly at the rest of the pub just in case any of them had heard what was going on.

None of them had, and none of them could understand what he was smiling at them for.

A man sitting next to Ford at the bar looked at the two men, looked at the six pints, did a swift burst of mental arithmetic, arrived at an answer he liked and grinned a stupid hopeful grin at them.

“Get off,” said Ford, “They’re ours,” giving him a look that would have an Algolian Suntiger get on with what it was doing.

Ford slapped a five-pound note on the bar. He said, “Keep the change.”

“What, from a fiver? Thank you sir.”

There’s a few great jokes there, but I’m interested in the final line. Ford buys six pints of bitter, pays with a five-pound note, and says “keep the change”, which surprises the barman. Presumably this is as a result of Ford’s perceived generosity… though of course what’s really happening is that Ford has no use for Earth money any longer; this point is hammered home for the barman and nearby patrons when Ford later buys four packets of peanuts, also asking the barman to keep the change from a fiver.

A pint of light-coloured beer on wooden board, with the sea in the background. An orange towel sits alongside.
Beer’s important, but you also need to know where your towel is.

We’re never told exactly what the barman would have charged Ford. But looking at the history of average UK beer prices and assuming that the story is set in 1979, we can assume that the pints will have been around 34p each1, so around £2.04 for six of them. So… Ford left a 194% tip for the beer2.

1990

By the time I first read Hitch-Hikers, around 1990, this joke was already dated. By then, an average pint of bitter would set you back £1.10. I didn’t have a good awareness of that, being as I was well-underage to be buying myself alcohol! But I clearly had enough of an awareness that my dad took the time to explain the joke… that is, to point out that when the story was written (and is presumably set), six pints would cost less than half of five pounds.

But by the mid-nineties, when I’d found a friend group who were also familiar with the Hitch-Hikers… series, we’d joke about it. Like pointing out that by then if you told the barman to keep the change from £5 after buying six pints, the reason he’d express surprise wouldn’t be because you’d overpaid…

Stock photo of a barman serving two patrons, annotated with speech bubbles. One patron says "Keep the change." The barman, smiling, says "What, from a a fiver. Fuck you sir."
In his defence, Ford’s an alien and might not fully understand human concepts of inflation. Or sarcasm.

1998

Precocious drinker that I was, by the late nineties I was quite aware of the (financial) cost of drinking.

Dan, then with long black hair, sits with another young white man in a garden, in front of a table laden with several bottles of wine and around 80 cans of beer.
Sure, this seems like a responsible amount of alcohol for a party thrown by a couple of tearaway teenagers. Definitely nothing going to go wrong here, no siree.

And so when it was announced that a new denomination of coin – the £2 coin – would enter general circulation3 I was pleased to announce how sporting it was of the government to release a “beer token”.

With the average pint of beer at the time costing around £1.90 and a still cash-dominated economy, the “beer token” was perfect! And in my case, it lasted: the bars I was drinking at in the late 1990s were in the impoverished North, and were soon replaced with studenty bars on the West coast of Wales, both of which allowed the price of a pint to do battle with inflationary forces for longer than might have been expected elsewhere in the country. The “beer token” that was the £2 coin was a joke that kept on giving for some time.

Close-up of a 1998 £2 coin; bimetallic, with an inner ring of cogs expanding outwards into several interlocking spirals.
The one thing I always hated about the initial design for the bimetallic £2 coin was – and this is the nerdiest thing in the world with which to take issue – the fact that it had a ring of 19 cogs to represent British industry. But if you connect a circuit of an odd number of cogs… it won’t function. Great metaphor, there. Photo courtesy of the late Andy Fogg, used under a Creative Commons license.

2023

As the cost of living rapidly increased circa 2023, the average price of a pint of beer in the UK finally got to the point where, rounded to the nearest whole pound, it was closer to £5 than it is to £44.

And while we could moan and complain about how much things cost nowadays, I’d prefer to see this as an opportunity. An opportunity for a new beer token: a general-release of the £5 coin. We already some defined characteristics that fit: a large, heavy coin, about twice the weight of the £2 coin, with a copper/nickel lustre and struck from engravings with thick, clear lines.

And the design basically comes up with itself. I give you… the Beer Token of the 2020s:

Mockup of a silver-coloured £5 coin with the words "FIVE POUNDS" and also "BEER TOKEN" printed around a stylised frothy pint of beer, dripping down the sides of its glass.
Wouldn’t this be much more-satisfying to give to a barman than a plasticky note or a wave of a contactless card or device?

It’s time for the beer token to return, in the form of the £5 coin. Now is the time… now is the last time, probably… before cash becomes such a rarity that little thought is evermore given to the intersection of its design and utility. And compared to a coin that celebrates industry while simultaneously representing a disfunctional machine, this is a coin that Brits could actually be proud of. It’s a coin that tourists would love to take home with them, creating a satisfying new level of demand for the sinking British Pound that might, just might, prop up the economy a little, just as here at home they support those who prop up the bar.

I know there must be a politician out there who’s ready to stand up and call for this new coin. My only fear is that it’s Nigel Fucking Farage… at which point I’d be morally compelled to reject my own proposal.

But for now, I think I’ll have another drink.

Footnotes

1 The recession of the 1970s brought high inflation that caused the price of beer to rocket, pretty much tripling in price over the course of the decade. Probably Douglas Adams didn’t anticipate that it’d more-than-double again over the course of the 1980s before finally slowing down somewhat… at least until tax changes in 2003 and the aftermath of the 2022 inflation rate spike!

2 We do know that the four packets of peanuts Ford bought later were priced at 7p each, so his tip on that transaction was a massive 1,686%: little wonder the barman suddenly started taking more-seriously Ford’s claims about the imminent end of the world!

3 There were commemorative £2 coins of a monometallic design floating around already, of course, but – being collectible – these weren’t usually found in circulation, so I’m ignoring them.

4 Otherwise known as “two beer tokens”, of course. As in “Bloody hell, 2022, why does a pint of draught cost two beer tokens now?”

× × × × ×

Article posted at UTC on .

1 comment

99 Days of Blogging

With this post, for the first time ever1, I’ve blogged for 99 consecutive days!2

Calendar showing Sunday 25 August through Sunday 1 December inclusive, with a coloured "spot" on each day with a size corresponding to the number of posts made on that day, broken into a pie chart showing the proportion of different post kinds.
The dots are sized based on the number of posts and broken-down by post kind: articles are blue, notes are green, checkins are orange, reposts are purple, and replies are red3.
I didn’t set out with the aim of getting to a hundred4, as I might well manage tomorrow, but after a while I began to think it a real possibility. In particular, when a few different factors came together:

Previous long streaks have sometimes been aided by pre-writing posts in bulk and then scheduling them to come out one-a-day6. I mostly don’t do that any more: when a post is “ready”, it gets published.

I didn’t want to make a “this is my 100th day of consecutive blogging” on the 100th day. That attaches too much weight to the nice round number. But I wanted to post to acknowledge that I’m going to make it to 100 days of consecutive blogging… so long as I can think of something worth saying tomorrow. I guess we’ll all have to wait and see.

Footnotes

1 Given that I’ve been blogging for over 26 years, that I’m still finding noteworthy blogging “firsts” is pretty cool, I think

2 My previous record “streak” was only 37 days, so there’s quite a leap there.

3 A massive 219 posts are represented over the last 99 days: that’s an average of over 2 a day!

4 This isn’t an attempt at #100DaysToOffload; I already achieved that this year as it does not require consecutive days. But it’s a cool challenge anyway.

5 My site’s backed by WordPress, but the mobile wp-admin isn’t the best and my site’s so-customised that apps like Jetpack mangle my metadata.

6 As you might now, I consider myself to be the primary audience for my blog: everybody else comes second. That’s why I don’t collect any webstats! When I used to collect webstats, I would sometimes pre-write and “schedule” posts, but without them it just feels pointless to do so!

×

Article posted at UTC on .

No comments

100 Days To Offload

The ever-excellent Kev Quirk in 2020 came up with this challenge: write a blog post on each of 100 consecutive days. He called it #100DaysToOffload, in nominal reference to the “100 days of code” challenge. I was reflecting upon this as I reach this, my 36th consecutive day of blogging and my longest ever “daily streak” (itself a spin-off of my attempt at Bloganuary this year), and my 48th post of the year so far.

Monochrome photograph showing sprinters at the starting line.
I guess I’ve always been more of a sprinter/hurdles blogger than a marathon runner.

Might I meet that challenge? Maybe. But it turns out it’s easier than I thought because Kev revised the rules to require only 100 posts in a calendar year (or any other 365-day period, but I’m not going to start thinking about the maths of that).

That’s not only much more-achievable… I’ve probably already achieved it! Let’s knock out some SQL to check how many posts I made each year: 

SELECT
  YEAR(wp_posts.post_date_gmt) yyyy,
  COUNT(wp_posts.ID) total
FROM
  wp_posts
WHERE
  wp_posts.post_status='publish'
  AND wp_posts.post_type='post'
GROUP BY yyyy
ORDER BY yyyy
My code’s actually a little more-complicated than this, because of some plot, but this covers the essentials.

A big question in some years is what counts as a post. Kev’s definition is quite liberal and includes basically-everything, but I wonder if mine shouldn’t perhaps be stricter. For example:

  • Should I count checkins, even though they’re not always born as blog posts but often start as logs on geocaching websites? (My gut says yes!)
  • Do reposts and bookmarks contribute, a significant minority of which are presented without any further interpretation by me? (My gut says no!)
  • Does a vlog version of a blog post count separately, or is it a continuation of the same content? (My gut says the volume is too low to matter!)
  • Can a retroactive achievement (i.e. from before the challenge was announced) count? Kev writes “there is no specific start date”, but it seems a little counter to the idea of it specifically being a challenge to claim it when you weren’t attempting the challenge at the time.
  • And so on…
Year Posts Success? Notes
1998 7 ❌ No Some posts are lost from 1998/1999. If they were recovered I might have made 100 posts in 1999, but probably not in 1998 as I only started blogging on 27 September 1998.
1999 66 ❌ No
2000 2 ❌ No
2001 11 ❌ No
2002 5 ❌ No
2003 189 🏆 Yes Achieved 1 September, with a post about an article on The Register about timewasting. Or, if we allow reposts, three days earlier with a repost about Claire's car being claimed by the sea.
2004 374 🏆 Yes An early win on 20 April, with a made-up Chez Geek card. Or if we allow reposts, two days earlier with thoughts on a confusing pro-life (???) website.
2005 381 🏆 Yes In a highly-productive year of blogging, achieved on 7 April with a post about enjoy curry and public information films with friends. If we allow bookmarks (I was highly-active on del.icio.us at the time!), achieved even earlier on 18 February with some links to curious websites.
2006 206 🏆 Yes On 21 July, I shared a personality test (which was actually my effort to repeat an experiment in using Barnum-Forer statements) - I didn't initially give away that I was the author of the "test". Non-pedants will agree I achieved the goal earlier, on 19 June, with my thoughts on a programming language for a hypothetical infinitely-fast computer.
2007 166 🏆 Yes Achieved on 2 July with thoughts on films I'd watched and board games I'd played recently. Or arguably 12 days earlier with Claire's birthday trip to Manchester.
2008 86 ❌ No
2009 79 ❌ No
2010 159
(84 for pedants) 		✅ Yes* A heartfelt post about saying goodbye to Aberystwyth as I moved to Oxford on 16 June was my 100th of the year. Pedants might argue that this year shouldn't count, but so long as you're willing to count checkins (and you should) then it would... and my qualifying post would have come only a couple of days later, with a post about the Headington Shark, which I had just moved-in near to.
2011 177 🏆 Yes Reached the goal on 28 October when I wrote about mild successes in my enquiries with the Office of National Statistics about ensuring that information about polyamorous households was accurately recorded. Or if we earlier on 9 June with a visual gag about REM lyrics if you accept all my geocache logs as posts too (and again: you should).
2012 133
(91 for pedants) 		✅ Yes* My 100th post of the year came on 28 August when I wrote about launching a bus named after my recently-deceased father. You have to be willing to accept both checkins and reposts as posts to allow this year to count.
2013 151
(72 for pedants) 		😓 Probably not I'm not convined this low-blogging year should count: a clear majority of the posts were geocaching logs, and they weren't always even that verbose (consider this candidate for 100th post of 2013, from 1 October).
2014 335
(22 for pedants) 		🙁 Not really Another geocache log heavy, conventional blogpost light year that I'm not convinced should count, even if the obvious candidate for 100th post would be 18 May's cool article about geocaching like Batman!
2015 205
(18 for pedants) 		🙁 Not really Still no, for the same reasons as above.
2016 163
(37 for pedants) 		🙁 Not really
2017 301
(42 for pedants) 		🙁 Not really
2018 547
(87 for pedants) 		✅ Yes* I maintain that checkins should count, even when they're PESOS'd from geocaching sites, so long as they don't make up a majority of the qualifying posts in a year. In which case this year should qualify, with the 100th post being my visit to this well-hidden London pub while on my way to a conference.
2019 387
(86 for pedants) 		✅ Yes* Similarly this year, when on 15 August I visited a GNSS calibration point in the San Francisco Bay Area... on the way to another conference!
2020 221
(64 for pedants) 		✅ Yes* Barely made it this year (ignoring reposts, of which I did lots), with my 21 December article about a little-known (and under-supported) way to inject CSS using HTTP headers, which I later used to make a web page for which View Souce showed nothing.
2021 190
(57 for pedants) 		✅ Yes* A cycle to a nearby geocache was the checkin that made the 100th post of this year, on 27 August.
2022 168
(55 for pedants) 		✅ Yes* My efforts to check up on one of my own geocaches on 7 September scored the qualifying spot.
2023 164
(86 for pedants) 		✅ Yes* My blogging ramped up again this year, and on 24 August I shared a motivational poster with a funny twist, plus a pun at the intersection between my sexuality and my preferred mode of transport.
2024 436 🏆 Yes Writing at full-tilt, my hundredth post came when I found a geocache near Regents Canal, but pedants who disregard reposts and checkins might instead count my excitement at the Ladybird Web browser as the record-breaker. This year also saw me write my 5,000th post on this blog! Wowza!
2025 342 🏆 Yes
Total 5,656 Total count of all the posts.
Doesn't add up? Not all posts feature in one of the years above!

* Pedants might claim this year was not a success for the reasons described above. Make your own mind up.

In any case, I’d argue that I clearly achieved the revised version of the challenge on certainly six, probably fourteen, arguably (depending on how you count posts) as many as nineteen different years since I started blogging in 1998. My least-controversial claims would be:

  1. September 2003, with Timewasting
  2. April 2004, with Chez Geek Card of the Day
  3. April 2005, with Curry with Alec and Suz
  4. July 2006, with Coolest Personality Test I’ve Ever Seen
  5. July 2007, with It’s All Fun and Games
  6. June 2010, with Saying Goodbye
  7. October 2011, with Poly and the Census – Success! (almost)
  8. August 2012, with A Bus Called Peter
  9. June 2018, with Dan Q found GLW6CMKQ 16th Century Pub (Central London) 
  10. August 2019, with Dan Q found GC6KR0H Bay Area Calibration Point #4 – New Technology
  11. December 2020, with The Fourth Way to Inject CSS
  12. August 2021, with Dan Q found GC531M9 Walk by the Firehouse #1
  13. October 2022, with Dan Q performed maintenance for GC9Z37H Friar’s Farm – Woodland Walk
  14. August 2023, with Inclusivity

Given all these unanswered questions, I’m not going to just go ahead and raise a PR against the Hall of Fame! Instead, I’ll leave it to Kev to decide whether I’m (a) eligible to claim a 14-time award, (b) merely eligible for a 4-time award for the years following the challenge starting, or (c) ineligible to claim success until I intentionally post 100 times in a year (in, at current rates, another two months…). Over to you, Kev…

Update: Kev’s agreed that I can claim the most-recent four of them, so I raised a PR.

×

Article posted at UTC on .

1 comment

[Bloganuary] My Biggest Challenge

This post is part of my attempt at Bloganuary 2024. Today’s prompt is:

What are your biggest challenges?

The Challenge of Winter Motivation

Two years ago, I reflected in February that I’d made almost zero blog posts in the previous month. Last month, I implemented a dynamically-updating Blog Stats page and my “winter/early Spring dip” became more-visible than ever.

Chart showing number of articles on DanQ.me by month of year, with a pronounced dip starting in January and continuing through until a rebound in April.
I find winters are generally bad for my creativity and motivation, usually until I bounce back in the Spring.

In an attempt to keep me writing daily, I’m giving Bloganuary a go this year. It’s sort-of like the NaNoWriMo of blogging1. And for me, Bloganuary’s very purpose is to overcome the challenge of getting disconnected from blogging when the nights are long and inspiration’s hard to find2.

The Challenge of Staying On-Task

But outside of the winter, my biggest challenge is usually… staying on-task!

It’s easy to get my focus to wane and for me to drift into some other activity than whatever it is I should be spending my time on. It’s not even procrastination3 so much as it’s a fluctuating and changing field of interest. I’ll drift off of what I’m supposed to be working on and start on something that interests me more in that moment… and then potentially off that too, in turn. The net result is that both my personal and professional lives are awash with half-finished projects4, all waiting their turn for me to find the motivation to swing back around and pick them up on some subsequent orbit of my brain.

A person wearing a cardboard box on their head, labelled "BRAIN". Above, a hand reaches from out-of-frame to hold a sign labelled "IDEA" above them.
You know how sometimes a stock image says exactly what you need it to? This isn’t one of those times.

It’s the kind of productivity antipattern I’d bring up with my coach, except that I already know exactly how she’d respond. First, she’d challenge the need to change; require that I justify it first. Second, she’d insist that before I can change, I need to accept and come to terms with who I am, intrinsically: if this flitting-about is authentically “me”, who am I to change it?

Finally, after weeks or months of exercises to fulfil these two tasks, she’d point out that I’ve now reached a place where I’m still just as liable to change lanes in the middle of a project as I was to begin with, but now I’m more comfortable with that fact. I won’t have externally changed, I’ll “just” have found some kind of happy-clappy inner peace. And she’ll have been right that that’s what I’d actually needed all along.

Maybe it’s not such a challenge, after all.

Footnotes

1 Except that would be NaBloPoMo, of course. But it’s a similar thing.

2 Also, perhaps, to help me focus on writing more-often, on more-topics, than I might otherwise in the course of my slow, verbose writing.

3 Except when it is.

4 Not to mention countless draft blog posts!

× ×

Article posted at UTC on .

1 comment

Blogging Stats

During a conversation with a colleague last week, I claimed that while I blog more-frequently than I did 5-10 years ago, it’s still with a much lower frequency than say 15-20 years ago.

Only later did I stop to think: is that actually true? It’s time for a graph!

I’ve previously graphed my blogging in an ad-hoc way, e.g. in 2016 I did a word-count and in 2021 I graphed posts-by-month-of-year, but I’ve never made an “eternal”, automatically-updating, interactive1 graph. Until now:

Generating a chart...
If this message doesn't go away, the JavaScript that makes this magic work probably isn't doing its job right: please tell Dan so he can fix it.

Generating a chart...
If this message doesn't go away, the JavaScript that makes this magic work probably isn't doing its job right: please tell Dan so he can fix it.

If you consider just articles (and optionally notes, which some older content might have been better classified-as, in retrospect) it looks like I’m right. Long gone are months like February 2005 when I posted an average of three times every two days! November 2018 was a bit of an anomaly as a I live-tweeted Challenge Robin II: my recent output’s mostly been comparable to the “quiet period” from 2008-20102.

Looking at number of posts by month of the year, it’s interesting to see a pronounced “dip” in all kinds of output roundabout March, less reposts in Summer and Autumn, and – perhaps unsurprisingly – more checkins (which often represent geocaching/geohashing logs) in the warmer months. Even on this scale, you can see the impact of the November “Challenge Robin spike” in the notes:

Generating a chart...
If this message doesn't go away, the JavaScript that makes this magic work probably isn't doing its job right: please tell Dan so he can fix it.

Anyway, now I’ve actually automated these kinds of stats its easier than ever for me to ask questions about how and when I write in my blog. I’ve put living copies of the charts plus additional treats (want to know when my longest “daily streak” was?) on a special page dedicated to that purpose. It’ll be interesting to see how it looks on this blog’s 25th anniversary, in a little under a year!

Footnotes

1 Try clicking on any of the post kinds in the legend to add/remove them, or click-and-drag a range across the chart to zoom in.

2 In hindsight, I was clearly depressed in and around 2009 and this doubtless impacted my ability to engage in “creative” pursuits.

Article posted at UTC on .

No comments

Hey ONS: This Is Not A Mistake

Hi, ONS! I know we haven’t really spoken since you ghosted me in 2011, but I just wanted to clear something up for you –

This is not a mistake (except for the missing last names):

(Specimen) 2021 census form on which Ruth declares that she cohabits with both a husband AND a partner.
It’s perfectly possible for somebody to live with multiple partners, even if they’re forbidden from marrying more than one.

Back in 2011 you thought it was a mistake, and this prevented my partner, her husband and I from filling out the digital version of the census. I’m sure it’s not common for somebody to have multiple cohabiting romantic relationships (though it’s possibly more common than some other things you track…), but surely an “Are you sure?” would be better than a “No you don’t!”

Clippy says "It looks like you've got a husband AND a partner. Is that right?" with possible answers "Yes, and it's awesome." or "No, but I can dream!"
For all I know, you already fixed it. If not: I mocked-up a UI for you.

We worked around it in 2011 by using the paper forms. Apparently this way you still end up “correcting” our relationship status for us (gee, thanks!) but at least – I gather – the originals are retained. So maybe in a more-enlightened time, future statisticians might be able ask about the demographics of domestic nonmonogamy and have at least some data to work with from the early 21st century.

I know you’re keen for as many people as possible to do the census digitally this year. But unless you’ve fixed your forms then my family and I – and thousands of others like us – will either have to use the paper copies you’re trying to phase out… or else knowingly lie on the digital versions. Which would you prefer?

× ×

MegaLoungeVII is the most-generous of the first 15 MegaLounges! (and other interesting statistics) [x-post /r/MegaMegaMonitor]

This self-post was originally posted to /r/MegaLoungeVII. See more things from Dan's Reddit account.

Thanks to the data that /r/MegaMegaMonitor collates, I’ve been able to throw together some fun statistics about the MegaLounge chain. Let me share some graphs with you:

This is all just something fun I threw together while I’ve been off work sick today. If there’s anything else that anybody would like to see extracted from MMM’s data, let me know: it’s all interesting stuff!

Repost posted at UTC on .

No comments

Some graphs about the MegaLounge chain

This self-post was originally posted to /r/MegaMegaMonitor. See more things from Dan's Reddit account.

Thanks to the data that MegaMegaMonitor collates, I’ve been able to throw together some fun statistics about the MegaLounge chain. Let me share some graphs with you:

This is all just something fun I threw together while I’ve been off work sick today. If there’s anything else that anybody would like to see extracted from MMM’s data, let me know: it’s all interesting stuff!

Repost posted at UTC on .

No comments

Poly and the Census – Success! (almost)

You may remember the long-running story of my letters to the Office of National Statistics, and the more-concentrated effort by another blogger, in regard to the automatic “correction” of supposedly-“erroneous” data in the 2011 census, like somebody having multiple partners or identifying as neither gender. You don’t? Well here’s a reminder: part one, part two, part three, part four.

Well: we’ve finally had some success. A response has been received from the ONS, including – at last – segments of business logic from their “correction” code.

It’s hard to tell for certain what the result of the correction will be, but one thing’s for sure – Ruth, JTA and I’s census data won’t have passed their validation! Their relationship validations BP2, BP2a, and BP2b state that it is logically-impossible for a person to have a spouse and a partner living with them in the same household.

I should invite them around for dinner sometime, and they can see for themselves that this isn’t true.

I also note that they consider it invalid for anybody to tick both or neither of the (two) gender option boxes, although again, it’s not clear from the data they’ve provided how the automatic correction occurs. Increasingly, I’m coming to suspect that this might actually be a manual process, in which case I’m wondering what guidelines there are for their operators?

One good piece of news from this FoI request, though: the ONS has confirmed that the original census data – the filled-in paper forms, which unlike the online version doesn’t enforce its validation upon you – is not adjusted. So in a hundred years time, people will be able to look back at the actual forms filled in by poly, trans, and other non-standard households around the UK, and generate actual statistics on the frequency with which these occur. It’s not much, but it’s something.

Article posted at UTC on .

2 comments

Poly and the Census – Part Four

Following up on my earlier blog posts about how data on polyamorous households is recorded in the census (see parts one, two, and three), as well as subsequent queries by Zoe O’Connell on this and related topics (how the census records data on other relationships, such as marriage between same-gender partners and civil partnerships between opposite-gender partners), there’s finally been some progress!

No; that’s a lie, I’m afraid. We’re still left wading around in the same muddy puddle. Zoe’s Freedom of Information Act request, which basically said “Okay, so you treat this kind of data as erroneous. How often does this happen?” got a response. And that response basically said, “We can’t tell you that, because we don’t have the information and it’d cost too much to work it out.” Back to square one.

Still: it looks like she’s not keen to be beaten, as she’s sent a fresh FoI request to instead ask “So what’s the algorithm you’re using to detect this erroneous data?” I was pleased to see that she went on to add, effectively, “I don’t need an explanation: send me the code if you need to,” which makes it harder for them to fall behind the “It’s too expensive!” excuse yet again.

Anyway: it’s one to watch. And needless to say, I’ll keep you all posted when anything changes…

Article posted at UTC on .

No comments

Poly and the Census – Part Three

Unimpressed with the slow response time that I and others were getting to my query to the Office of National Statistics (to which I still never received a response) the month before last, Zoe O’Connell decided to send a Freedom of Information Act request demanding a response to a couple of similar questions. After some hassling (I suppose they’ve been busy, with the census and all), they finally responded. The original request and the full response is online now, as is Zoe’s blog post about the response. But here’s the short version of the response:

Polygamous marriages are not legally recognised in the UK and therefore any data received from a questionnaire that appeared to show polygamous relationship in the manner that you suggest would be read as an error. It is recognised that the majority of respondents recording themselves as being in a polygamous relationship in a UK census do so erroneously, for example, ticking the wrong box for one household member on the relationships question.

Therefore, the data to be used for statistical purposes would be adjusted by changing one or more of these relationships, so that each respondent is in a relationship with no more than one person. This is consistent with all previous UK censuses, and others around the world.

A copy of the original questionnaire would be retained as part of the historical record which would show such relationships as they were recorded. We do not attempt to amend the original record.

Any mismatches between the indicated sex and marital status of respondents will be resolved using a probabilistic statistical system which will not necessarily deal with each case in the same way. The system will look at other responses for each person, including those for the Household relationships, and will alter one or more variables to make the response consistent. In the example that you propose, it would either change the sex of one individual, or change the marital status to “Same-sex civil partnership”, depending on which is considered statistically more likely to be correct.

Honestly, I’m not particularly impressed. They’ve committed to maintaining a historical record of the original, “uncorrected” data, so that future statisticians can get a true picture of the answers given, but this is about the only positive point in this response. Treating unusual data as erroneous is akin to pretending that a societal change doesn’t exist, and that this approach is “consistent with previous censuses” neglects to entertain the possibility that this data has value that it might not have had previously.

Yes, there will be erroneous data: people who accidentally said that they had two husbands when they only have one, for example. And yes, this can probably (although they don’t state how they know to recognise this) be assumed to be more common that genuine cases where somebody meant to put that on their census (although there will also be an error rate amongst these people, too). But taking the broad brush approach of assuming that every case can be treated as an error reeks of the same narrow-mindedness as the (alleged; almost-certainly an urban legend) statement by Queen Victoria that lesbianism “didn’t exist.”

“Fixing” the data using probabilities just results in a regression towards the mean: “Hmm; this couple of men say they’re married: they could be civil partners, or it could be a mistake… but they’re in a county with statistically-few few gay people, so we’ll assume the latter.” Really: what?

I’m not impressed, ONS.

Update: a second FoI request now aims to determine how many “corrections” have been made on censuses, historically. One to watch.

Poly and the Census – Part Two

No reply yet from the Office of National Statistics after the letter I sent the other week, but I imagine that they’ve been busy, what with the census and everything. Needless to say, I’ll keep you posted.

However, in the meantime somebody’s one-upped me and has put in a Freedom of Information request, which – of course – the law mandates that they respond to. I should’a thought of that. Anyway, you can read the request here, and there’s options to follow it by RSS and/or email if you want updates.

Update (27th April 2011): Still no word in response to the FoI request.

Article posted at UTC on .

1 mention