https://danq.me Personal blog of Dan Q: hacker, magician, geocacher, gamer... Thu, 23 Nov 2017 07:08:37 +0000 en-GB hourly 1 Carry Fire https://danq.me/2017/11/06/carry-fire/ https://danq.me/2017/11/06/carry-fire/#comments Mon, 06 Nov 2017 15:11:31 +0000 https://danq.me/?p=7467 I’ve just listened to Robert Plant’s new album, Carry Fire. It’s pretty good.

A long while after my dad’s death five years ago, I’d meant to write a blog post about the experience of grief in a digital age. As I’ve clearly become increasingly terrible at ever getting draft posts complete, the short of it was this: my dad’s mobile phone was never recovered and soon after its battery went flat any calls to his number would go straight to voicemail. He’d recently switched to a pay-as-you-go phone for his personal mobile, and so the number (and its voicemail) outlived him for many months. I know I’m not the only one that, in those months, called it a few times, just to hear his voice in the outgoing message. I’m fully aware that there are recordings of his voice elsewhere, but I guess there was something ritualistic about “trying to call him”, just as I would have before his accident.

The blog post would have started with this anecdote, perhaps spun out a little better, and then gone on to muse about how we “live on” in our abandoned Inboxes, social media accounts, and other digital footprints in a way we never did before, and what that might mean for the idea of grief in the modern world. (Getting too caught up in thinking about exactly what it does mean is probably why I never finished writing that particular article.) I remember that it took me a year or two until I was able to delete my dad from my phone/email address book, because it like prematurely letting go to do so. See what I mean? New aspects of grief for a new era.

Rob Plant's "Carry Fire"
Thanks, Rob.

Another thing that I used to get, early on, was that moment of forgetting. I’d read something and I’d think “Gotta tell my dad about that!” And then only a second later remember why I couldn’t! I think that’s a pretty common experience of bereavement: certainly for me at least – I remember distinctly experiencing the same thing after my gran’s death, about 11 years ago. I’m pretty sure it’s been almost a year since I last had such a forgetting moment for my father… until today! Half way into the opening track of Carry Fire, a mellow folk-rocky-sounding piece called The May Queen (clearly a nod to Stairway there), I found myself thinking “my dad’d love this…” and took almost a quarter-second before my brain kicked in and added “…damn; shame he missed out on it, then.”

If you came here for a music review, you’re not going to get one. But if you like some Robert Plant and haven’t heard Carry Fire yet, you might like to. It’s like he set out to make a prog rock album but accidentally smoked too much pot and then tripped over his sitar. And if you knew my dad well enough to agree (or disagree) that he would have dug it, let me know.

]]> https://danq.me/2017/11/06/carry-fire/feed/ 1 Jokes That Aged Badly https://danq.me/2017/11/01/badly-aged-jokes/ https://danq.me/2017/11/01/badly-aged-jokes/#comments Wed, 01 Nov 2017 16:43:47 +0000 https://danq.me/?p=7005 I’ve lately taken an interest in collecting jokes that haven’t aged well. By which I mean: jokes that no longer work, or require explanation, because they’re conceptually ‘dated’. Typically, these jokes aren’t funny any more, or are only funny to people who were around at the time that they were first conceived, and I imagine that we, as a civilisation, are necessarily relegating more and more jokes into this particular category as time goes on.

A man used to make his wife go to bed first in the winter-time, caller her his warming-pan; which she not well relishing, one night left something smoaking in his place. Finding himself in a stinking pickle: Wife, said he, I am beshit: No, husband, said she, it is only coal dropt out of your warming-pan!
I don’t mean outdated like this joke, published in 1803, but ones that require explanation because a listener is no longer likely to recognise the concepts or people referenced. For example, if it were the case that warming pans were now such an alien concept that nobody knew what they were any more, then the above would certainly qualify (that’s not true yet… right?).

My favourite joke of this category is the following classic student joke, which was relevant when I first heard it in the 1990s:

What’s pink and takes an hour to drink?

Grant cheque

By way of explanation: the grant cheque was how British students used to receive their government aid to support them during their studies. It had become gradually smaller (relative to the value of the pound) over time by failing to rise in value in line with inflation, and was printed on pink paper, hence the joke. There was an effort to revive it in the late 1990s/early 2000s as follows:

What’s green and takes an hour to drink?

Loan cheque

By this point, the grant had been replaced by the student loan, whose payments came printed on green paper instead. This is, of course, simply an example of adapting an old joke for a new audience, as we’ve all seen time and again with the inevitable string of recycled gags that get rolled-out every time a celebrity is accused of a sex crime. Incidentally, the revised form of the grant cheque/loan cheque joke has itself become dated as students now typically receive all of their loan payments directly to their bank accounts for convenient immediate spending rather than what my generation had to do which was to make the beans-and-rice stretch another few days until the cheque cleared.

An idiot asks a silversmith to make him a lamp. 'How big do you want me to make it?' the silversmith asks. 'Big enough for eight people to see by.'
This 4th/5th century joke (presented with a contemporary translation) doesn’t count, because it’s still flipping hilarious.

Here’s another example:

Bill and Ben the flower pot men are in the garden.

“Flobalobalobalob,” says Bill

Ben replies: “You’re drunk, Bill.”

Now those of you who are about my age, or older, are unlikely to see why this joke has dated badly. But it is dated, because the 2001 reboot of The Flower Pot Men (now called simply Bill and Ben) features the titular characters speaking in reasonably-normal English! The idea that they were only speaking Oddle Poddle because they were too pissed to speak English is no longer a point of humour, and increasingly the population won’t remember the original stilted dialect of the flower pot men. If we assume that anybody under the age of 24 is more-likely to have come across the newer incarnation then that’s a third of the population!

Let’s try another, which became dated at about the same time:

Why are hurricanes names after women?

Because when they come they’re wet and wild and when they go they take your house and your car.

The history of how we’ve named hurricanes over the centuries is really quite interesting, and its certainly true that for the majority of the period during which both meteorologists and the general public have shared the same names for tropical storms they’ve been named after women. Depending on where you are in the world, though, it’s not been true for some time: Australia began using a mixture of masculine and feminine names during the 1970s, but other regions took until the millennium before they followed suit. However, the point still remains that this joke has been dated for a long while.

Eye of a storm.
Why is a topical joke like a tropical storm?
When it comes, you’ve never seen anything like it. By the time it goes, you’re sick of it.

Here’s a very highly-charged joke from the 1960s which I think we can all be glad doesn’t make much sense any more:

What’s all black and comes in an all white box?

Sammy Davis Jr.

For those needing the context: Sammy Davis Jr. was a black American singer, comedian, and variety show host who triggered significant controversy when he married white Swedish actress May Britt. Interracial marriage was at the time still illegal across much of the United States (such prohibition wouldn’t be ruled unconstitutional until the amazingly-named “Loving Day” in 1967) and relationships between whites and “coloureds” were highly taboo even where they weren’t forbidden by law.

Topical jokes like that are often too easy, like this one – even shorter-lived – from the summer of 1995, presented here with no further interpretation:

Q: What’s the difference between O. J. Simpson and Christopher Reeve?

A: O. J.’s gonna walk!

Perhaps my favourite strictly-topical joke of this variety, though, comes from 1989:

Q: Why is Margaret Thatcher like a pound coin?

A: She’s thick, brassy, and she thinks she’s a sovereign.

It’s at least two-thirds funny even if you don’t have the full context, and that’s what’s most-interesting about it: it’ll take until the new £1 becomes ubiquitous and the old one mostly-forgotten before it will lose all of its meaning. But as you’ve probably forgotten why the third part of the punchline – “…and she thinks she’s a sovereign” – comes from, I’ll illuminate you. The joke is wordplay: there are two meanings to “sovereign” in this sentence. The first, of course, is that a sovereign is the bullion coin representing the same value as a conventional pound coin.

To understand the second, we must first remind ourselves of the majestic plural, better known as the “royal ‘we'”. In 1989, following the birth of her grandson Michael, Thatcher made a statement saying “we have become a grandmother”, resulting in much disdain and mockery by the press at the time. The Prime Minister’s relationship with the Queen had always been a frosty one, and Thatcher’s (mis)use of a manner of speech that was typically reserved for the use of royalty did nothing to make her look any more-respectful of the monarch.

1952 advertisement for Prince Albert pipe tobacco.
And you thought it was just something to put in your penis. Turns out it’s something to put in your lungs, too.

The final example I’ve got died out as a joke as a result of changing brand identities, more cost-effective packaging materials, and the gradual decline of tobacco smoking. But for a long while, while Prince Albert Pipe Tobacco was still sold in larger quantities as it always had been, in a can, a popular prank perpetrated by radio stations that went in for such things was to call a tobacconist and ask, “Have you got Prince Albert in can?”. The tobacconist would invariably answer in the affirmative, at which point the prankster would response “Well let him out then!” This joke may well predate the “Is your refrigerator running?” prank call that might be more-familiar to today’s audiences.

If you’ve got any jokes that have aged badly, I’d love to hear them. And then, I suppose, have them explained to me.

]]> https://danq.me/2017/11/01/badly-aged-jokes/feed/ 1 GMail Tip: Use A Plus Sign To Avoid Spam https://danq.me/2017/09/26/gmail-plus/ https://danq.me/2017/09/26/gmail-plus/#comments Tue, 26 Sep 2017 08:26:40 +0000 https://danq.me/?p=7295 This technique’s about a decade old, but a lot of people still aren’t using it, and I can’t help but suspect that can only be because they didn’t know about it yet, so let’s revisit:

You have a GMail account, right? Or else Google for Domains? Suppose your email address is dan@gmail.com… did you know that also means that you own:

You have a practically infinite number of GMail addresses. Just put a plus sign (+) after your name but before the @-sign and then type anything you like there, and the email will still reach you. You can also insert as many full stops (.) as you like, anywhere in the first half of your email address, and they’ll still reach you, too. And that’s really, really useful.

Filling in an Equifax registration form.
Often, you end up having to give your email address to companies that you don’t necessarily trust…

When you’re asked to give your email address to a company, don’t give them your email address. Instead, give them a mutated form of your email address that will still work, but that identifies exactly who you gave it to. So for example you might give the email address dan+amazon@gmail.com to Amazon, the email address dan+twitter@gmail.com to Twitter, and the email address dan+pornhub@gmail.com to… that other website you have an account on.

Why is this a clever idea? Well, there are a few reasons:

The email address dan+equifax@gmail.com being entered into a form.
Certainly, you can have… THIS email address.

I know that some people get some of these benefits by maintaining a ‘throwaway’ email address. But it’s far more-convenient to use the email address you already have (you’re already logged-in to it and you use it every day)! And if you ever do want a true ‘throwaway’, you’re generally better using Mailinator: when you’re asked for your email address, just mash the keyboard and then put @mailinator.com on the end, to get e.g. dsif9tsnev4y8594es87n65y4@mailinator.com. Copy the first half of the email address to the clipboard, and then when you’re done signing up to whatever spammy service it is, just go to mailinator.com and paste into the box to see what they emailed you.

A handful of badly-configured websites won’t accept email addresses with plus signs in them, claiming that they’re invalid (they’re not). Personally, when I come across these I generally just inform the owner of the site of the bug and then take my business elsewhere; that’s how important it is to me to be able to filter my email properly! But another option is to exploit the fact that you can put as many dots in (the first part of) your GMail address as you like. So you could put d…an@gmail.com in and the email will still reach you, and you can later filter-out emails to that address. I’ll leave it as an exercise for the reader to decide how to encode information about the service you’re signing up to into the pattern and number of dots that you use.

Go forth and avoid spam.

]]> https://danq.me/2017/09/26/gmail-plus/feed/ 3 Fnorders https://danq.me/2017/09/13/fnorders/ Wed, 13 Sep 2017 14:53:40 +0000 https://danq.me/?p=7277 I’ve not posted much recently: I’ve had a lot of Complicated Life Stuff going on, sorry.

But I did make a thing: fnorders.com. You’re welcome.

Fnord.

]]> Stones Under Yetzin https://danq.me/2017/06/05/stones-under-yetzin/ https://danq.me/2017/06/05/stones-under-yetzin/#comments Mon, 05 Jun 2017 16:19:33 +0000 https://danq.me/?p=7266 For the last few months, I’ve been GMing a GURPS campaign (that was originally a Warhammer Fantasy Roleplay 1st-edition campaign, in turn built upon a mixture of commercially published and homegrown modules, including, in turn, an AD&D module…) for a few friends.

So far, it’s included such gems as a player-written poem in a fictional language, another player’s drawing of the most-cinematic action sequence they’ve experienced so far… and the opportunity, during a play session that coincided with a player’s birthday, to explain the layout of a ruined tower by presenting them with a cake baked into the shape of the terrain.

Cake shaped sort-of like The Lone Tower.
“So we’re… here?” asked a player, jabbing with his finger at the cream-filled section of the tower at which he was standing.

If you’re interested in what we’ve been up to, the campaign has it’s own blog where you can read about the adventures of Newman, Bret, Lythil, Keru, and (the late) Sir Bea.

But mostly I wanted to make this post so that I had a point of context in case I ever get around to open-sourcing some of the digital tools I’ve been developing to help streamline our play sessions. For example, most of our battle maps and exploration are presented on a ‘board’ comprised of a flat screen monitor stripped of its stand and laid on its back, connected via the web to a tool that allows me to show, hide, or adapt parts of it from my laptop or mobile phone. Player stats, health, and cash, as well as the date, time, position of the sun as well as the phases of the moons are similarly tracked and are available via any player’s mobile phone at any time.

Players fight a basilisk across a paper RPG map.
A rare instance of us using a paper-based battle map. Despite the fact that we play in-person, we’ve used digital tools to save table space!

These kinds of tools have been popular for ‘long-distance’/Internet roleplaying for years, but I think there’s a lot of potential in locally-linked, tabletop-enhancing (rather than replacing) tools that deliver some of the same benefit to the (superior, in my opinion) experience of ‘proper’ face-to-face adventure gaming. Now, at least, when I tell you for example about some software I wrote to help calculate the position of the sun in the sky of a fictional world, you’ll have a clue why I would do such a thing in the first place.

]]> https://danq.me/2017/06/05/stones-under-yetzin/feed/ 2 Playing Dead – Toddlers Gone Weird https://danq.me/2017/05/26/playing-dead/ https://danq.me/2017/05/26/playing-dead/#comments Fri, 26 May 2017 14:05:48 +0000 https://danq.me/?p=7252 No matter how prepared you think you are for the questions your toddler might ask (and the ways in which they might go on to interpret your answer), they’ll always find a way to catch you off guard. The following exchange with our little one began last weekend in the car:

Annabel sitting at a bar in a pub.
I’m sure we’ve all been asked “Why can’t I drink what you’re drinking?”

Her: “I read the Beano Annual at Grandtom’s house.” (Grandtom is what she calls Ruth‘s father – her maternal grandfather.)

Me: “Oh? Did you like it?”

Her: “Yes. Did you have the Beano Annual when you were a little boy?”

Me: “Yes: I would sometimes get one for Christmas when I was little.”

Her: “Who gave it to you?”

Me: My mummy and daddy did.”

Her: “Your mummy is Nanna Doreen.”

Me: “That’s right.”

Her: “Why haven’t I met your daddy?”

Dan sits on his father's knee. 1980s.
Dan and daddy.

That’s a question that I somehow hadn’t expected to come up so soon. I probably ought to have guessed that it was on its way, given her interest in her extended family lately and how they’re all connected to one another, but I’d somehow assumed that it’d have come up organically at some point or another before her curiosity had made the connection that there was somebody clearly missing: somebody whom she’d heard mentioned but, inexplicably, never met.

Me: “My daddy died, a couple of years before you were born. He was climbing a mountain one day when he had a nasty accident and fell off, and he died.”

Her: “…” (a thoughtful pause)

Me: “Are you okay?”

Her: “How many birthdays did he have?”

Me: “Fifty-four. That’s a bigger number than you can count to, I think!”

Her: “How many birthdays will I have?”

Wow, this went further than I expected, very quickly. Obviously, I want to be open about this: the last thing I want is to introduce a taboo, and I’m a big believer in the idea that on I’m suddenly conscious of the fact that she’s clearly close to a minor existential crisis, having for possibly the first time connected the concepts of age and death. And, of course, I’m trying to translate my thoughts into ideas that a toddler can follow every step of the way. While simultaneously trying to focus on driving a car: she knows how to pick her timing! Okay…

Me: “Nobody knows for sure, but you’ll probably get lots and lots: seventy, eighty, ninety… maybe even a hundred birthdays!”

Her: “Then I’ll have a hundred candles.”

Me: “That’s right. Do you think you could blow out a hundred candles?”

Annabel's third birthday party.
Three candles was well within her grasp.

So far, so good. Knowing that, like most toddlers, ours has a tendency to make some new discovery and then sit on it for a day or two before asking a follow-up question, I briefed Ruth and JTA so that they wouldn’t be caught too off-guard when she started telling them, for example, what she’d like for her hundredth birthday or something.

And all was well until yesterday, when we were laying in the garden under the recent glorious sunshine, playing a game that involved rolling along the lawn and back and bumping into one another in the middle, when she stood up and announced that she’d like to play something different.

Her: “Now we’re playing the die game.”

Me: “Oh…kay. How do we play that?”

Her: “We’re going to go up a mountain and then fall off.”

Me: (following her in a stomp around the garden) “Then what do we do?”

Her: “We die.” (mimes falling and then lies very still)

Annabel plays dead after "falling off a mountain"
A ‘dead’ body at the bottom of a ‘mountain’. Erk!

And so that’s how I came to spend an afternoon repeatedly re-enacting the circumstances of my father’s death, complete – later on, after Ruth mentioned the air ambulance that carried his body down from the mountain – with a helicopter recovery portion of the game. I’ve role-played some unusual games over the years, but this one was perhaps the oddest, made stranger by the fact that it was invented by a three year-old.

Toddlers process new information in strange (to adults) ways, sometimes.

]]> https://danq.me/2017/05/26/playing-dead/feed/ 3 AMP Is Poisonous https://danq.me/2017/05/19/amp/ https://danq.me/2017/05/19/amp/#comments Fri, 19 May 2017 21:39:11 +0000 https://danq.me/?p=7243 If you’re a web developer and you haven’t come across the Google AMP project yet… then what stone have you been living under? But just in case you have been living under such a stone – or you’re not a web developer – I’ll fill you in. If you believe Google’s elevator pitch, AMP is “…an open-source initiative aiming to make the web better for all… consistently fast, beautiful and high-performing across devices and distribution platforms.”

I believe that AMP is fucking poisonous and that the people who’ve come out against it by saying it’s “controversial” so far don’t go remotely far enough. Let me tell you about why.

AMP logo in handcuffs

When you configure your website for AMP – like the BBC, The Guardian, Reddit, and Medium already have – you deliver copies of your pages written using AMP HTML and AMP JS rather than the HTML and Javascript that you’re normally would. This provides a subset of the functionality you’re used to, but it’s quite a rich subset and gives you a lot of power with minimal effort, whether you’re trying to make carousels, video players, social sharing features, or whatever. Then when your site is found via Google Search on a mobile device, then instead of delivering the user to your AMP HTML page or its regular-HTML alternative… Google delivers your site for you via an ultra-fast precached copy via their own network. So far, a mixed bag, right? Wrong.

What’s poisonous about Google AMP

Ignoring the facts that you can get locked-in if you try it once, it makes the fake news problem worse than ever, and it breaks the core concepts of a linkable web, the thing that worries me the most is that AMP represents the most-subtle threat to Net Neutrality I’ve ever seen… and it’s from an organisation that is nominally in favour of a free and open Internet but that stands to benefit from a more-closed Internet so long as it’s one that they control.

Google’s stated plan to favour pages that use AMP creates a publisher’s arms race in which content creators are incentivised to produce content in the (open-source but) Google-controlled AMP format to rank higher in the search results, or at least regain parity, versus their competitors. Ultimately, if everybody supported AMP then – ignoring the speed benefits for mobile users (more on that in a moment) – the only winner is Google. Google, who would then have a walled garden of Facebook-beating proportions around the web. Once Google delivers all of your content, there’s no such thing as a free and open Internet any more.

So what about those speed increases? Yes, the mobile web is slower than we’d like and AMP improves that. But with the exception of the precaching – which is something that could be achieved by other means – everything that AMP provides can be done using existing technologies. AMP makes it easy for lazy developers to make their pages faster, quickly, but if speed on mobile devices is the metric for your success: let’s just start making more mobile-friendly pages! We can make the mobile web better and still let it be our Web: we don’t need to give control of it to Google in order to shave a few milliseconds off the load time.

We need to reject AMP, and we need to reject it hard. Right now, it might be sufficient to stand up to your boss and say “no, implementing AMP on our sites is a bad idea.” But one day, it might mean avoiding the use of AMP entirely (there’ll be browser plugins to help you, don’t worry). And if it means putting up with a slightly-slower mobile web while web developers remain lazy, so be it: that’s a sacrifice I’m willing to make to help keep our web free and open. And I hope you will be, too.

Like others, I’m just hoping that Sir Tim will feel the urge to say something about this development soon.

]]> https://danq.me/2017/05/19/amp/feed/ 1 Science! (for toddlers) https://danq.me/2017/05/05/science-for-toddlers/ https://danq.me/2017/05/05/science-for-toddlers/#comments Fri, 05 May 2017 06:19:05 +0000 https://danq.me/?p=7218 I’m not sure that there’s any age that’s too-young at which to try to cultivate an interest in science. Once a child’s old enough to ask why something is the case, every question poses an opportunity for an experiment! Sometimes a thought experiment is sufficient (“Uncle Dan: why do dogs not wear clothes?”) but other times provide the opportunity for some genuine hands-on experimentation (“Why do we put flowers in water?”). All you have to do is take every question and work out what you’d do if you didn’t know the answer either! A willingness to take any problem with a “let’s find out” mentality teaches children two important things: (a) that while grown-ups will generally know more than them, that nobody has all the answers, and (b) that you can use experiments to help find the answers to questions – even ones that have never been asked before!

Annabel sorts jewellery at the Pitt River Museum of Anthropology.
“Why do we make jewellery out of different things?” Thanks to the Pitt Rivers Museum for inspiring this question… and helping us to find an answer.

Sometimes it takes a little more effort. Kids – like all of us, a lot of the time – can often be quite happy to simply accept the world as-it-is and not ask “why”. But because a fun and educational science activity is a good way to occupy a little one (and remember: all it needs to be science is to ask a question and then try to use evidence to answer it!), I’ve been keeping a list of possible future activities so that we’ve got a nice rainy-day list of things to try. And because we are, these days, in an increasingly-large circle of breeders, I thought I’d share some with you.

Annabel observes the filling of a paddling pool with custard.
You don’t strictly need a cement mixer full of custard to demonstrate dilatant (non-Newtonian) fluids, but messiness is engaging all by itself.

Here’s some of the activities we’ve been doing so far (or that I’ve got lined-up for future activities as and when they become appropriate):

Red-spotted black ladybird.
Spotting different subspecies of ladybirds is a great springboard to talking about heritable characteristics and phenotypic variation. Snails are another good candidate.
A demonstration of capillary action using water and tissue paper.
Water, tissue paper, and patience is all you need to demonstrate capillary action and siphoning. Food colouring’s an optional bonus.
Annabel and Ruth cooking.
Cooking provides opportunities for exploration, too. Bake some bread and you’ve got an excuse to talk about yeast!
Paining a colour wheel.
Subtractive colour mixing can be demonstrated by mixing paint. Colour and spin a wheel to demonstrate additive mixing.
Annabel is awed by the size of a tyrannosaurus rex skeleton.
“Why do some animals have sharp teeth and some have flat teeth?” was a question I posed. We found the answer together (and were wowed by the size of the T-rex skeleton behind the camera) at the Natural History Museum.
Annabel hugs a goat.
Get some hands-on biology at your nearest petting zoo. No science in this picture, but plenty of hugging.

So there’s my “now and next” list of science activities that we’ll be playing at over the coming months. I’m always open to more suggestions, though, so if you’re similarly trying to help shape an enquiring and analytical mind, let me know what you’ve been up to!

 

]]> https://danq.me/2017/05/05/science-for-toddlers/feed/ 3 The Long Tail of an Abusive Relationship https://danq.me/2017/04/30/long-tail-of-abuse/ https://danq.me/2017/04/30/long-tail-of-abuse/#comments Sun, 30 Apr 2017 20:50:14 +0000 https://danq.me/?p=7205 I am a survivor of an abusive relationship, and parts of that experience affect the way that I engage in romantic relationships… but I have difficulty quantifying exactly how much. Insert obvious (minor) trigger warning here, and scroll past the kitten if you want to read more.

An adorable long-haired calico kitten. Instant eyebleach.
Mew.

I’m fine, by the way. It took… a long, long time, like in the region of a decade, to be completely fine about it, and I appreciate that compared to many people, I got lucky. Like many victims (and especially among men), my recovery was hampered by the fact that I found it difficult to see the relationship as having been abusive in the first place: that first step took many years all by itself. I’m not kidding when I say I’m fine, by the way: no, I don’t need to talk about it (with many of my circles of friends made up of current and former helpline volunteers of various types, I feel the need to make that doubly-clear: sometimes, one just can’t escape from people who care about you so much that they’ll offer you a cup of tea even if they’ve only got saltwater to make it with, if you catch the drift of my needless in-joke).

But I wanted to share with you something that I’ve gradually realised about how I was changed as a result of that relationship. Something that still affects me today and, for all I know, probably always will: a facet of my personality whose origins I eventually traced back to that dreadful relationship.

A man investigating the inside of his own (mechanical) brain.
Earlier this year, I finally got around to reading the (brilliant) Stories of Your Life and Others by Ted Chiang. As somebody who loves to take apart his own brain to see how it works, I loved the story of an automaton who more-literally does exactly that.

A major factor in my attraction to people, for the last decade and a half, has been whether or not they demonstrate being attracted to me. I’m sure that’s the case for everybody, at least to some extent – there’s a necessary reciprocity for a relationship to work, of course – but in my case there’ve been times in my past when the entirety of my attraction to somebody could be described in terms of their attraction to me… and that’s a level that definitely isn’t healthy! It stems from a lack of belief in my own worth as relationship material, which had grown to such an extent that feeling as if I were even-remotely attractive in somebody else’s eyes has, regardless of whether or not I’d be interested in them under other circumstances, made me feel as though I ought to “give them a shot”. Again: not healthy.

This, in turn, comes from a desperation of considering myself fundamentally unattractive, undateable, and generally unworthy of the attention of anybody else in any relationship capacity… which is highly tied-up in the fact that I had a relationship in which my partner repeatedly and methodically taught me exactly that: that I was lucky to be in a relationship with them or indeed with anybody, etc.

Given enough time, persuasion, and coercive tactics, this is the kind of shit that sinks in and, apparently, sticks.

Dalmatian wrapped in barbed wire.
If this picture makes you sad… then you shouldn’t have scrolled past the kitten, should you?

I don’t mind that I’m a product of my environment. But it bugs me a little that I’m still, to a small (and easily managable, nowadays) extent the product of somebody else’s deliberate and manipulative efforts to control me, a decade and a half after the fact.

Now I’ll stress once again that I’m fine now: I’ve recovered by as much as I need (or at least expect) to. Some years ago, I finally got to the point that if you let me know that you’re attracted to me then that isn’t by itself something that makes me completely infatuated with you. Nowadays, I’m capable of actually engaging my brain and thinking “Hmm: would I be interested in this person if it weren’t for the fact that they’d just validated my worth in some way?” But I’m still aware of the sensation – that nagging feeling that I’m acting according to a manipulative bit of programming – even though I’m pretty confident that it doesn’t influence how I behave any more.

It’s funny how our brains work. At the end of the relationship, I made a reasonably-rapid bounceback/recovery in terms of my general self-worth, but it took far, far longer to get control over this one specific thing. I guess we all react to particular stresses in different ways. For me, somebody who’d spent his childhood and teen years with perhaps, if anything, a little much self-worth, it might have been inevitable that I’d be unable to rebuild the part of that self-image that was most-effectively demolished by somebody else: the bit that is dependent upon somebody else’s validation.

But who knows… as I said, I have difficulty quantifying how much that abusive relationship impacted me. Because it is, of course, true to say that every single thing I’ve ever experienced will have affected me in some way or another – made me the person I subsequently became. How can I justify blaming a single relationship? I know that I wasn’t “like this” back when I first started my dating life, but I can’t conclusively prove that it was the result of any one particular relationship: for all I can claim, perhaps it was something else? Maybe this was always who I’d become? Or maybe, of course, this entire paragraph is simply the result of the fact that my brain still has difficulty with the term “abusive relationship” and is more-than-happy to keep trying to reach for whatever alternative explanations it can find.

Once again though, I’ll stress that I’m okay now and I have been for many years. I just wanted to share with you an observation I’d made about my own psychology… and the long tail that even the “tamest” of abusive relationships can leave.

]]> https://danq.me/2017/04/30/long-tail-of-abuse/feed/ 3 Steer! An Experimental Canvas/Websocket Game https://danq.me/2017/04/28/steer/ Fri, 28 Apr 2017 15:33:42 +0000 https://danq.me/?p=7193 As you may know, I’ve lately found an excuse to play with some new web technologies, and I’ve also taken the opportunity to try to gain a deeper understanding of some less bleeding-edge technologies that I think have some interesting potential. And so it was that, while I was staffing the Three Rings stall at last week’s NCVO conference, I made use of the time that the conference delegates were all off listening to a presentation to throw together a tech demo I call Steer!

Animated GIF from a video, showing a player using their mobile phone to steer a car on a desktop computer screen, all using the web browsers on both devices.
A player uses their mobile phone to steer a car on a desktop computer, using nothing more than a web browser.

As you can see from the GIF above, Steer! is a driving game. The track and your car are displayed in a web browser on a large screen, for example a desktop or laptop computer, television, or tablet, and your mobile phone is used to steer the car by tilting it to swerve around a gradually-narrowing weaving road. It’s pretty fun, but what really makes it interesting to me is the combination of moderately-new technologies I’ve woven together to make it possible, specifically:

Infographic showing how Steer! works. Phone accelerometer determines orientation, pushes to Firebase (up to 60 times/sec), which pushes to browser (via Websocket), which updates screen.

The desktop browser does all of the real work: it takes the orientation of the device and uses that, and the car’s current speed, to determine how it’s position changes over the time that’s elapsed since the screen was last refreshed: we’re aiming for 60 frames a second, of course, but we don’t want the car to travel slower when the game is played on a slower computer, so we use requestAnimationFrame to get the fastest rate possible and calculate the time between renderings to work out how much of a change has occurred this ‘tick’. We leave the car’s sprite close to the bottom of the screen at all times but change how much it rotates from side to side, and we use it’s rotated to decide how much of its motion is lateral versus the amount that’s “along the track”. The latter value determines how much track we move down the screen “behind” it.

The track is generated very simply by the addition of three sine waves of different offset and frequency – a form of very basic procedural generation. Despite the predictability of mathematical curves, this results in a moderately organic-feeling road because the player only sees a fraction of the resulting curve at any given time: the illustration below shows how these three curves combine to make the resulting road. The difficulty is ramped up the further the player has travelled by increasing the amplitude of the resulting wave (i.e. making the curves gradually more-agressive) and by making the road itself gradually narrower. The same mathematics are used to determine whether the car is mostly on the tarmac or mostly on the grass and adjust its maximum speed accordingly.

Sum of sine waves as used to generate the track for Steer!

In order to help provide a visual sense of the player’s speed, I added dashed lines down the road (dividing it into three lanes to begin with and two later on) which zip past the car and provide a sense of acceleration, deceleration, overall speed, and the impact of turning ‘sideways’ (which of course reduces the forward momentum to nothing).

This isn’t meant to be a finished game: it’s an experimental prototype to help explore some technologies that I’d not had time to look seriously at before now. However, you’re welcome to take a copy – it’s all open source – and adapt or expand it. Particular ways in which it’d be fun to improve it might include:

]]> Tomorrow’s Web, Today https://danq.me/2017/04/19/tomorrows-web/ https://danq.me/2017/04/19/tomorrows-web/#comments Wed, 19 Apr 2017 13:31:38 +0000 https://danq.me/?p=7183 Maybe it’s because I was at Render Conf at the end of last month or perhaps it’s because Three Rings DevCamp – which always gets me inspired – was earlier this month, but I’ve been particularly excited lately to get the chance to play with some of the more “cutting edge” (or at least, relatively-new) web technologies that are appearing on the horizon. It feels like the Web is having a bit of a renaissance of development, spearheaded by the fact that it’s no longer Microsoft that are holding development back (but increasingly Apple) and, perhaps for the first time, the fact that the W3C are churning out standards “ahead” of where the browser vendors are managing to implement technical features, rather than simply reflecting what’s already happening in the world.

Ben Foxall at Render Conf 2017 discusses the accompanying JSOxford Hackathon.
Ben Foxall at Render Conf 2017 discusses the accompanying JSOxford Hackathon. Hey, who’s that near the top-right?

It seems to me that HTML5 may well be the final version of HTML. Rather than making grand new releases to the core technology, we’re now – at last! – in a position where it’s possible to iteratively add new techniques in a resilient, progressive manner. We don’t need “HTML6” to deliver us any particular new feature, because the modern web is more-modular and is capable of having additional features bolted on. We’re in a world where browser detection has been replaced with feature detection, to the extent that you can even do non-hacky feature detection in pure CSS, now, and this (thanks to the nature of the Web as a loosely-coupled, resilient platform) means that it’s genuinely possible to progressively-enhance content and get on board with each hot new technology that comes along, if you want, while still delivering content to users on older browsers.

And that’s the dream! A web of progressive-enhancement stays true to Sir Tim’s dream of universal interoperability while still moving forward technologically. I’ve no doubt that there’ll always be people who want to break the Web – even Google do it, sometimes – with single-page Javascript-only web apps, “app shell” websites, mobile-only or desktop-only experiences and “apps” that really ought to have been websites (and perhaps PWAs) to begin with… but the fact that the tools to make a genuinely “progressively-enhanced” web, and those tools are mainstream, is a big deal. If you don’t think we’re at that point yet, I invite you to watch Rachel Andrews‘ fantastic presentation, “Start Using CSS Grid Layout Today”.

Three Rings DevCamp 2017
Three Rings’ developers hard at work at this year’s DevCamp.

Some of the things I’ve been playing with recently include:

Intersection Observers

Only really supported in Chrome, but there’s a great polyfill, the Intersection Observer API is one of those technologies that make you say “why didn’t we have that already?” It’s very simple: all an Intersection Observer does is to provide event hooks for target objects entering or leaving the viewport, without resorting to polling or hacky code on scroll event captures.

Intersection Observer example (animated GIF)

What’s it for? Well the single most-obvious use case is lazy-loading images, a-la Medium or Google Image Search: delivering users a placeholder image or a low-resolution copy until they scroll far enough for the image to come into view (or almost into view) and then downloading the full-resolution version and dynamically replacing it. My first foray into Intersection Observers was to take Medium’s approach and then improve it with a Service Worker in order to make it behave nicely even if the user’s Internet connection was unreliable, but I’ve since applied it to my Reddit browser plugin MegaMegaMonitor: rather than hammering the browser with Javascript the plugin now waits until relevant content enters the viewport before performing resource-intensive tasks.

Web Workers

I’d briefly played with Service Workers before and indeed we’re adding a Service Worker to the next version of Three Rings, which, in conjunction with a manifest.json and the service’s (ongoing) delivery over HTTPS (over H2, where available, since last year), technically makes it a Progressive Web App… and I’ve been looking for opportunities to make use of Service Workers elsewhere in my work, too… but my first dive in to Web Workers was in introducing one to the next upcoming version of MegaMegaMonitor.

MegaMegaMonitor v155a Lists feature
MegaMegaMonitor’s processor-intensive “Lists” feature sees the most benefit from Web Workers

Web Workers add true multithreading to Javascript, and in the case of MegaMegaMonitor this means the possibility of pushing the more-intensive work that the plugin has to do out of the main thread and into the background, allowing the user to enjoy an uninterrupted browsing experience while the heavy-lifting goes on in the background. Because I don’t control the domain on which this Web Worker runs (it’s reddit.com, of course!), I’ve also had the opportunity to play with Blobs, which provided a convenient way for me to inject Worker code onto somebody else’s website from within a userscript. This has also lead me to the discovery that it ought to be possible to implement userscripts that inject Service Workers onto websites, which could be used to mashup additional functionality into websites far in advance of that which is typically possible with a userscript… more on that if I get around to implementing such a thing.

Fetch

The final of the new technologies I’ve been playing with this month is the Fetch API. I’m not pulling any punches when I say that the Fetch API is exactly what XMLHttpRequests should have been from the very beginning. Understanding them properly has finally given me the confidence to stop using jQuery for the one thing for which I always seemed to have had to depend on it for – that is, simplifying Ajax requests! I mean, look at this elegant code:

fetch('posts.json')
.then(function(response) {
  return response.json();
})
.then(function(json) {
  console.log(json.something.otherThing);
});

Whether or not you’re a fan of Javascript, you’ve got to admit that that’s infinitely more readable than XMLHttpRequest hackery (at least, without the help of a heavyweight library like jQuery).

Laser Duck Hunt at Render Conf 2017
Other things I’ve been up to include Laser Duck Hunt, but that’s another story.

So that’s some of the stuff I’ve been playing with lately: Intersection Observers, Web Workers, Blobs, and the Fetch API. And I feel all full of optimism on behalf of the Web.

]]> https://danq.me/2017/04/19/tomorrows-web/feed/ 2 What Does Jack FM Sound Like? https://danq.me/2017/03/13/jack-fm/ https://danq.me/2017/03/13/jack-fm/#comments Mon, 13 Mar 2017 16:37:41 +0000 https://danq.me/?p=7168 Those who know me well know that I’m a bit of a data nerd. Even when I don’t yet know what I’m going to do with some data yet, it feels sensible to start collecting it in a nice machine-readable format from the word go. Because you never know, right? That’s how I’m able to tell you how much gas and electricity our house used on average on any day in the last two and a half years (and how much off that was offset by our solar panels).

Daily energy usage at Dan's house for the last few years. Look at the gas peaks in the winters, when the central heating ramps up!
The red lumps are winters, when the central heating comes on and starts burning a stack of gas.

So it should perhaps come as no huge surprise that for the last six months I’ve been recording the identity of every piece of music played by my favourite local radio station, Jack FM (don’t worry: I didn’t do this by hand – I wrote a program to do it). At the time, I wasn’t sure whether there was any point to the exercise… in fact, I’m still not sure. But hey: I’ve got a log of the last 45,000 songs that the radio station played: I might as well do something with it. The Discogs API proved invaluable in automating the discovery of metadata relating to each song, such as the year of its release (I wasn’t going to do that by hand either!), and that gave me enough data to, for example, do this (click on any image to see a bigger version):

Jack FM: Decade Frequency by Hour
Decade frequency by hour: you’ve got a good chance of 80s music at any time, but lunchtime’s your best bet (or perhaps just after midnight). Note that times are in UTC+2 in this graph.

I almost expected a bigger variance by hour-of-day, but I guess that Jack isn’t in the habit of pandering to its demographics too heavily. I spotted the post-midnight point at which you get almost a plurality of music from 1990 or later, though: perhaps that’s when the young ‘uns who can still stay up that late are mostly listening to the radio? What about by day-of-week, then:

Jack FM: Decade Frequency by Day of Week
Even less in it by day of week… although 70s music fans should consider tuning in on Fridays, apparently, and 80s fans will be happiest on Sundays.

The chunks of “bonus 80s” shouldn’t be surprising, I suppose, given that the radio station advertises that that’s exactly what it does at those times. But still: it’s reassuring to know that when a radio station claims to play 80s music, you don’t just have to take their word for it (so long as their listeners include somebody as geeky as me).

It feels to me like every time I tune in they’re playing an INXS song. That can’t be a coincidence, right? Let’s find out:

Jack FM: Artist Frequency
One in every ten songs are by just ten artists (including INXS). One in every four are by just 34 artists.

Yup, there’s a heavy bias towards Guns ‘n’ Roses, Michael Jackson, Prince, Oasis, Bryan Adams, Madonna, INXS, Bon Jovi, Queen, and U2 (who collectively are responsible for over a tenth of all music played on Jack FM), and – to a lesser extent – towards Robert Palmer, Meatloaf, Blondie, Green Day, Texas, Whitesnake, the Pet Shop Boys, Billy Idol, Madness, Rainbow, Elton John, Bruce Springsteen, Aerosmith, Fleetwood Mac, Phil Collins, ZZ Top, AC/DC, Duran Duran, the Police, Simple Minds, Blur, David Bowie, Def Leppard, and REM: taken together, one in every four songs played on Jack FM is by one of these 34 artists.

Jack FM: Top 20
Amazingly, the most-played song on Jack FM (Alice Cooper’s “Poison”) is not by one of the most-played 34 artists.

I was interested to see that the “top 20 songs” played on Jack FM these last six months include several songs by artists who otherwise aren’t represented at all on the station. The most-played song is Alice Cooper’s Poison, but I’ve never recorded them playing any other Alice Cooper songs (boo!). The fifth-most-played song is Fight For Your Right, by the Beastie Boys, but that’s the only Beastie Boys song I’ve caught them playing. And the seventh-most-played – Roachford’s Cuddly Toy – is similarly the only Roachford song they ever put on.

Next I tried a Markov chain analysis. Markov chains are a mathematical tool that examines a sequence (in this case, a sequence of songs) and builds a map of “chains” of sequential songs, recording the frequency with which they follow one another – here’s a great explanation and playground. The same technique is used by “predictive text” features on your smartphone: it knows what word to suggest you type next based on the patterns of words you most-often type in sequence. And running some Markov chain analysis helped me find some really… interesting patterns in the playlists. For example, look at the similarities between what was played early in the afternoon of Wednesday 19 October and what was played 12 hours later, early in the morning of Thursday 20 October:

19 October 2016 20 October 2016
12:06:33 Kool & The Gang – Fresh Kool & The Gang – Fresh 00:13:56
12:10:35 Bruce Springsteen – Dancing In The Dark Bruce Springsteen – Dancing In The Dark 00:17:57
12:14:36 Maxi Priest – Close To You Maxi Priest – Close To You 00:21:59
12:22:38 Van Halen – Why Can’t This Be Love Van Halen – Why Can’t This Be Love 00:25:00
12:25:39 Beats International / Lindy – Dub Be Good To Me Beats International / Lindy – Dub Be Good To Me 00:29:01
12:29:40 Kasabian – Fire Kasabian – Fire 00:33:02
12:33:42 Talk Talk – It’s My Life Talk Talk – It’s My Life 00:38:04
12:41:44 Lenny Kravitz – Are You Gonna Go My Way Lenny Kravitz – Are You Gonna Go My Way 00:42:05
12:45:45 Shalamar – I Can Make You Feel Good Shalamar – I Can Make You Feel Good 00:45:06
12:49:47 4 Non Blondes – What’s Up 4 Non Blondes – What’s Up 00:50:07
12:55:49 Madness – Baggy Trousers Madness – Baggy Trousers 00:54:09
Eagle Eye Cherry – Save Tonight 00:56:09
Feeling – Love It When You Call 01:04:12
13:02:51 Fine Young Cannibals – Good Thing Fine Young Cannibals – Good Thing 01:10:14
13:06:54 Blur – There’s No Other Way Blur – There’s No Other Way 01:14:15
13:09:55 Pet Shop Boys – It’s A Sin Pet Shop Boys – It’s A Sin 01:17:16
13:14:56 Zutons – Valerie Zutons – Valerie 01:22:18
13:22:59 Cure – The Love Cats Cure – The Love Cats 01:26:19
13:27:01 Bryan Adams / Mel C – When You’re Gone Bryan Adams / Mel C – When You’re Gone 01:30:20
13:30:02 Depeche Mode – Personal Jesus Depeche Mode – Personal Jesus 01:33:21
13:34:03 Queen – Another One Bites The Dust Queen – Another One Bites The Dust 01:38:22
13:42:06 Shania Twain – That Don’t Impress Me Much Shania Twain – That Don’t Impress Me Much 01:42:23
13:45:07 ZZ Top – Gimme All Your Lovin’ ZZ Top – Gimme All Your Lovin’ 01:46:25
13:49:09 Abba – Mamma Mia Abba – Mamma Mia 01:50:26
13:53:10 Survivor – Eye Of The Tiger Survivor – Eye Of The Tiger 01:53:27
Scouting For Girls – Elvis Aint Dead 01:57:28
Verve – Lucky Man 02:00:29
Fleetwood Mac – Say You Love Me 02:05:30
14:03:13 Kiss – Crazy Crazy Nights Kiss – Crazy Crazy Nights 02:10:31
14:07:15 Lightning Seeds – Sense Lightning Seeds – Sense 02:14:33
14:11:16 Pretenders – Brass In Pocket Pretenders – Brass In Pocket 02:18:34
14:14:17 Elvis Presley / JXL – A Little Less Conversation Elvis Presley / JXL – A Little Less Conversation 02:21:35
14:22:19 U2 – Angel Of Harlem U2 – Angel Of Harlem 02:24:36
14:25:20 Trammps – Disco Inferno Trammps – Disco Inferno 02:28:37
14:29:22 Cast – Guiding Star Cast – Guiding Star 02:31:38
14:33:23 New Order – Blue Monday New Order – Blue Monday 02:36:39
14:41:26 Def Leppard – Let’s Get Rocked Def Leppard – Let’s Get Rocked 02:40:41
14:46:28 Phil Collins – Sussudio Phil Collins – Sussudio 02:45:42
14:50:30 Shawn Mullins – Lullaby Shawn Mullins – Lullaby 02:49:43
14:55:31 Stars On 45 – Stars On 45 Stars On 45 – Stars On 45 02:53:45
16:06:35 Dead Or Alive – You Spin Me Round Like A Record Dead Or Alive – You Spin Me Round Like A Record 03:00:47
16:09:36 Dire Straits – Walk Of Life Dire Straits – Walk Of Life 03:03:48
16:13:37 Keane – Everybody’s Changing Keane – Everybody’s Changing 03:07:49
16:17:39 Billy Idol – Rebel Yell Billy Idol – Rebel Yell 03:10:50
16:25:41 Stealers Wheel – Stuck In The Middle Stealers Wheel – Stuck In The Middle 03:14:51
16:28:42 Green Day – American Idiot Green Day – American Idiot 03:18:52
16:33:44 A-Ha – Take On Me A-Ha – Take On Me 03:21:53
16:36:45 Cranberries – Dreams Cranberries – Dreams 03:26:54
Elton John – Philadelphia Freedom 03:30:56
Inxs – Disappear 03:36:57
Kim Wilde – You Keep Me Hanging On 03:40:59
16:44:47 Living In A Box – Living In A Box
16:47:48 Status Quo – Rockin’ All Over The World Status Quo – Rockin’ All Over The World 03:45:00

The similarities between those playlists (which include a 20-songs-in-a-row streak!) surely can’t be coincidence… but they do go some way to explaining why listening to Jack FM sometimes gives me a feeling of déjà vu (along with, perhaps, the no-talk, all-jukebox format). Looking elsewhere in the data I found dozens of other similar occurances, though none that were both such long chains and in such close proximity to one another. What does it mean?

There are several possible explanations, including:

But the question remains: why reuse playlists in close proximity at all? Even when the station operates autonomously, as it clearly does most of the time, it’d surely be easy enough to set up an auto-DJ using “smart random” (because truly random shuffles don’t sound random to humans) to get the same or a better effect.

Jack FM Style Guide
One of the things I love about Jack FM is how little they take seriously. Like their style guide.

Which leads to another interesting observation: Jack FM’s sister stations in Surrey and Hampshire also maintain a similar playlist most of the time… which means that they’re either synchronising their ad breaks (including their duration – I suspect this is the case) or else using filler jingles to line-up content with the beginnings and ends of songs. It’s a clever operation, clearly, but it’s not beyond black-box comprehension. More research is clearly needed. (And yes, I’m sure I could just call up and ask – they call me “Newcastle Dan” on the breakfast show – but that wouldn’t be even half as fun as the data mining is…)

]]> https://danq.me/2017/03/13/jack-fm/feed/ 9 A Suitable Blog https://danq.me/2016/06/21/a-suitable-blog/ https://danq.me/2016/06/21/a-suitable-blog/#comments Tue, 21 Jun 2016 21:01:03 +0000 https://danq.me/?p=7147 At a little over 590 thousand words and spanning 1,349 pages, Vikram Seth’s A Suitable Boy is almost-certainly among the top ten longest single-volume English-language novels. It’s pretty fucking huge.

A Suitable Boy, seen from the edge
I’ll stick with the Kindle edition: I fear that merely holding the paperback would be exhausting.

I only discovered A Suitable Boy this week (and haven’t read it – although there are some good reviews that give me an inclination to) when, on a whim, I decided to try to get a scale of how much I’d ever written on this blog and then decided I needed something tangible to use as a comparison. Because – give or take – that’s how much I’ve written here, too:

Graph showing cumulative words written on this blog, peaking at 593,457.
At 593,457 words, this blog wouldn’t fit into that book unless we printed it on the covers as well.

Of course, there’s some caveats that might make you feel that the total count should be lower:

On the other hand, there are a few reasons that it perhaps ought to be higher:

A delicious-looking BLT. Mmm, bacon.
Another reason for not counting images was that it was harder than you’d think to detect repeat use of images that I’ve used too many times. Like this one.

Of course, my blog doesn’t really have a plot like A Suitable Boy (might compare well to the even wordier Atlas Shrugged, though…): it’s a mixture of mostly autobiographical wittering interspersed with musings on technology and geekery and board games and magic and VR and stuff. I’m pretty sure that if I knew where my life would be now, 18 years ago (which is approximately when I first started blogging), I’d have, y’know, tried to tie it all together with an overarching theme and some character development or something.

Or perhaps throw in the odd plot twist or surprise: something with some drama to keep the reader occupied, rather than just using the web as a stream-of-conciousness diary of whatever it is I’m thinking about that week. I could mention, for example, that there’ll be another addition to our house later this year. You heard it here first (unless you already heard it from somewhere else first, in which case you heard it there first.)

Annabel sitting on her daddy's knee and looking at sonograph pictures of her future baby brother.
Brought up in a world of tiny, bright, UHD colour touchscreens, Annabel seemed slightly underwhelmed by the magic of a sonograph picture of her future baby brother.

Still: by the end of this post I’ll have hit a nice, easy-to-remember 594,000 words.

]]> https://danq.me/2016/06/21/a-suitable-blog/feed/ 3 Anatomy of Cookie XSS https://danq.me/2016/06/16/cookie-xss/ https://danq.me/2016/06/16/cookie-xss/#comments Thu, 16 Jun 2016 10:13:14 +0000 https://danq.me/?p=7138 A cross-site scripting vulnerability (shortened to XSS, because CSS already means other things) occurs when a website can be tricked into showing a visitor unsafe content that came from another site visitor. Typically when we talk about an XSS attack, we’re talking about tricking a website into sending Javascript code to the user: that Javascript code can then be used to steal cookies and credentials, vandalise content, and more.

Good web developers know to sanitise input – making anything given to their pages by a user safe before ever displaying it on a page – but even the best can forget quite how many things really are “user input”.

"Who Am I?" page provided by University of Oxford IT Services.
This page outputs a variety of your inputs right back at you.

Recently, I reported a vulnerability in a the University of Oxford’s IT Services‘ web pages that’s a great example of this.  The page (which isn’t accessible from the public Internet, and now fixed) is designed to help network users diagnose problems. When you connect to it, it tells you a lot of information about your connection: what browser you’re using, your reverse DNS lookup and IP address, etc.. The developer clearly understood that XSS was a risk, because if you pass a query string to the page, it’s escaped before it’s returned back to you. But unfortunately, the developer didn’t consider the fact that virtually anything given to you by the browser can’t be trusted.

My Perl program, injecting XSS code into the user's cookie and then redirecting them.
To demonstrate this vulnerability, I had the option of writing Perl or Javascript. For some reason, I chose Perl.

In this case, I noticed that the page would output any cookies that you had from the .ox.ac.uk domain, without escaping them. .ox.ac.uk cookies can be manipulated by anybody who has access to write pages on the domain, which – thanks to the users.ox.ac.uk webspace – means any staff or students at the University (or, in an escalation attack, anybody’s who’s already compromised the account of any staff member or student). The attacker can then set up a web page that sets up such a “poisoned” cookie and then redirects the user to the affected page and from there, do whatever they want. In my case, I experimented with showing a fake single sign-on login page, almost indistinguishable from the real thing (it even has a legitimate-looking .ox.ac.uk domain name served over a HTTPS connection, padlock and all). At this stage, a real attacker could use a spear phishing scam to trick users into clicking a link to their page and start stealing credentials.

A fake SSO login page, delivered from a legitimate-looking https URL.
The padlock, the HTTPS url, and the convincing form make this page look legitimate. But it’s actually spoofed.

I’m sure that I didn’t need to explain why XSS vulnerabilities are dangerous. But I wanted to remind you all that truly anything that comes from the user’s web browser, even if you think that you probably put it there yourself, can’t be trusted. When you’re defending against XSS attacks, your aim isn’t just to sanitise obvious user input like GET and POST parameters but also anything that comes from a browser header including cookies and referer headers, especially if your domain name carries websites managed by many different people. In an ideal world, Content Security Policy would mitigate all these kinds of attacks: but in our real world – sanitise those inputs!

]]> https://danq.me/2016/06/16/cookie-xss/feed/ 6 Underground and Overground in the City of London https://danq.me/2016/06/04/underground-overground/ https://danq.me/2016/06/04/underground-overground/#comments Sat, 04 Jun 2016 16:34:50 +0000 https://danq.me/?p=7114 Despite being only a short journey away (made even shorter by the new railway station that appeared near by house last year), I rarely find myself in London. But once in a while a week comes along when I feel like I’m there all the time.

British Rail branded poster from an abandoned tunnel under Euston Station, circa 1960s.
Bargain travel to London from the station around the corner! Don’t think this poster is up-to-date, though.

On Friday of last week, Ruth, JTA and I took one of the London Transport Museum‘s Hidden London tours. Back in 2011 we took a tour of Aldwych Tube Station, probably the most well-known of the London Underground’s disused stations, and it was fantastic, so we were very excited to be returning for another of their events. This time around, we were visiting Euston Station.

Our tour group gathers around the corner from Euston Station.
Stylish hi-vis jackets for everybody!

But wait, you might-well say: Euston station isn’t hidden nor disused! And you’d be right. But Euston’s got a long and convoluted history, and it used to consist of not one but three stations: the mainline station and two independent underground stations run by competing operators. The stations all gradually got connected with tunnels, and then with a whole different set of tunnels as part of the redevelopment in advance of the station’s reopening in 1968. But to this day, there’s still a whole network of tunnels underneath Euston station, inaccessible to the public, that are either disused or else used only as storage, air vents, or cable runs.

Disused lift shaft under Euston Station.
This lift shaft used to transport passengers between what are now the Northern and Victoria lines. Now it’s just a big hole.

A particular highlight was getting to walk through the ventilation shaft that draws all of the hot air out of the Victoria Line platforms. When you stand and wait for your train you don’t tend to think about the network of tunnels that snake around the one you’re in, hidden just beyond the grills in the ceiling or through the doors at the end of the platforms. I shot a video (below) from the shaft, periodically looking down on the trains pulling in and out below us.

No sooner were we back than I was away again. Last Saturday, I made my way back to London to visit Twitter’s UK headquarters in Soho to help the fantastic Code First: Girls team to make some improvements to the way they organise and deliver their Javascript, Python and Ruby curricula. I first came across Code First: Girls through Beverley, one of Three Rings‘ volunteers who happens to work for them, and I’ve become a fan of their work. Unfortunatley my calendar’s too packed to be able to volunteer as one of their instructors (which I totally would if it weren’t for work, and study, and existing volunteering, and things), but I thought this would be a good opportunity to be helpful while I had a nominally-“spare” day.

The coffee lounge on the administration/marketing floor of Twitter's offices in Soho.
Twitter’s offices, by the way, are exactly as beautiful as you’d hope that they might be.

Our host tried to win me over on the merits of working for Twitter (they’re recruiting heavily in the UK, right now), and you know what – if I were inclined towards a commute as far as London (and I didn’t love the work I do so much) – I’d totally give that a go. And not just because I enjoyed telling an iPad what I wanted to drink and then having it dispensed minutes later by a magical automated hot-and-cold-running-drinks tap nearby.

Twitter's reception with its "tweet wall" sculpture.
I’m not sure I ‘get’ the idea of a sculpture of tweets, though. Wouldn’t a “live display” have been more-thematic?

And that’s not even all of it. This coming Thursday, I’m back in London again, this time to meet representatives from a couple of charities who’re looking at rolling out Three Rings. In short: having a direct line to London on my doorstep turns out to be pretty useful.

]]> https://danq.me/2016/06/04/underground-overground/feed/ 3 recruit.ox.ac.uk Permalink Generator https://danq.me/2016/05/31/recruit-ox-ac-uk-permalink-generator/ https://danq.me/2016/05/31/recruit-ox-ac-uk-permalink-generator/#comments Tue, 31 May 2016 10:37:42 +0000 https://danq.me/?p=7112 If you’ve ever applied for a job with my employer, the University of Oxford, you’ll have come across recruit.ox.ac.uk, one of the most-frustrating websites in the world. Of its many problems, the biggest (in my mind) is that it makes it really hard to share or save the web address of a particular job listing. That’s because instead of using individual web addresses to correspond to individual jobs, like any sanely-designed system would, it uses Javascript hackery and black magic to undermine the way your web browser was designed to work (which is why, you’ll find, you can’t “open in new tab” properly either), and instead provides its own, inferior, interface.

Some day I might get around to writing e.g. a userscript and/or browser plugin that “fixes” the site – from a user’s perspective, at least. But for the time being, because this morning I needed to share via social media a link to a UX developer post we’ve just advertised, I’ve come up with a little bookmarklet to fix this single problem:

recruit.ox.ac.uk Permalink Generator

Drag the bookmarklet to your bookmarks toolbar, then - when on the recruit.ox.ac.uk site - click it to use it.

This tool makes it easy to get permalinks (web addresses you can save or share) for job listings on recruit.ox.ac.uk. It might be adaptable to make it work with other CoreHR-powered systems, if it turns out that this missing feature comes from the underlying software that powers the site: it could also form the basis of a future userscript that would automatically fix the site “on the fly”. Here’s how to use it:

  1. Drag the link below into your browser’s bookmarks (e.g. the bookmarks toolbar).

    recruit.ox.ac.uk permalink

  2. When you’re on a recruit.ox.ac.uk job page, click on the bookmark. A permalink will appear at the top of the page, for your convenience. If you’re using a modern browser, the permalink will also appear in the address bar.
  3. Copy the permalink and use it wherever you need it, e.g. to share the link to a job listing.

If you have any difficulty with it or want help adapting it for use with other CoreHR systems, give me a shout.

]]> https://danq.me/2016/05/31/recruit-ox-ac-uk-permalink-generator/feed/ 1 DevCamp – have we really been doing this for 7 years? https://danq.me/2016/04/11/devcamp-2016/ Mon, 11 Apr 2016 15:57:50 +0000 https://danq.me/?p=7099 An annual tradition at Three Rings is DevCamp, an event that borrows from the “hackathon” concept and expands it to a week-long code-producing factory for the volunteers of the Three Rings development team. Motivating volunteers is a very different game to motivating paid employees: you can’t offer to pay them more for working harder nor threaten to stop paying them if they don’t work hard enough, so it’s necessary to tap in to whatever it is that drives them to be a volunteer, and help them get more of that out of their volunteering.

Table full of computers at DevCamp 2011.
This photo, from DevCamp 2011, is probably the only instance where I’ve had fewer monitors out than another developer.

At least part of what appeals to all of our developers is a sense of achievement – of producing something that has practical value – as well as of learning new things, applying what they’ve learned, and having a degree of control over the parts of the project they contribute most-directly to. Incidentally, these are the same things that motivate paid developers, too, if a Google search for studies on the subject is to believed. It’s just that employers are rarely able to willing to offer all of those things (and even if they can, you can’t use them to pay your mortgage), so they have to put money on the table too. With my team at Three Rings, I don’t have money to give them, so I have to make up for it with a surplus of those things that developers actually want.

A developer hides inside a handmade camera obscura to watch the solar eclipse at DevCamp 2015.
At the 2015 DevCamp, developers used the solar eclipse as an excuse for an impromptu teambuilding activity: making a camera obscura out of stuff we had lying about.

It seems strange to me in hindsight that for the last seven years I’ve spent a week of my year taking leave from my day job in order to work longer, harder, and unpaid for a voluntary project… but that I haven’t yet blogged about it. Over the same timescale I’ve spent about twice as long at DevCamp than I have, for example, skiing, yet I’ve managed to knock out several blog posts on that subject. Part of that might be borne out of the secretive nature of Three Rings, especially in its early days (when involvement with Three Rings pretty-much fingered you as being a Nightline volunteer, which was frowned upon), but nowadays we’ve got a couple of dozen volunteers with backgrounds in a variety of organisations: and many of those of us that ever were Nightliner volunteers have long since graduated and moved-on to other volunteering work besides.

DevCamp and DocsCamp 2016 volunteers play Betrayal at the House on the Hill
Semi-cooperative horror-themed board games by candlelight are a motivator for everybody, right?

Part of the motivation – one of the perks of being a Three Rings developer – for me at least, is DevCamp itself. Because it’s an opportunity to drop all of my “day job” stuff for a week, go to some beatiful far-flung corner of the country, and (between early-morning geocaching/hiking expeditions and late night drinking tomfoolery) get to spend long days contributing to something awesome. And hanging out with like-minded people while I do so. I like I good hackathon of any variety, but I love me some Three Rings DevCamp!

Geocache GC4EE6C, with accompanying caterpillar and mushroom
The geocaches near DevCamp 2016 were particularly fabulous, though. Like this one – GC4EE6C – part of an Alice In Wonderland-themed series.

So yeah: DevCamp is awesome. It’s more than a little different than those days back in 2003 when I wrote all the code and Kit worked hard at distracting me with facts about the laws of Hawaii – for the majority of DevCamp 2016 we had half a dozen developers plus two documentation writers in attendance! – but it’s still fundamentally about the same thing: producing a piece of software that helps about 25,000 volunteers do amazing things and make the world a better place. We’ve collectively given tens, maybe hundreds of thousands of hours of time in developing and supporting it, but that in turn has helped to streamline the organisation of about 16 million person-hours of other volunteering.

So that’s nice.

Developers marvel at one another's code, etc.
An end-of-day “Show & Tell” session at DevCamp 2016.

Oh, and I was delighted that one of my contributions this DevCamp was that I’ve finally gotten around to expanding the functionality of the “gender” property so that there are now more than three options. That’s almost more-exciting than the geocaches. Almost.

Edit: added a missing word in the sentence about how much time our volunteers had given, making it both more-believable and more-impressive.

]]> Immersive Storytelling – Thoughts on Virtual Reality, part 3 https://danq.me/2016/03/27/immersive-storytelling-3/ Sun, 27 Mar 2016 21:21:50 +0000 https://danq.me/?p=7075 This is the (long-overdue) last in a three-part blog post about telling stories using virtual reality. Read all of the parts here.

For the first time in two decades, I’ve been playing with virtual reality. This time around, I’ve been using new and upcoming technologies like Google Cardboard and the Oculus Rift. I’m particularly interested in how these new experiences can be used as a storytelling medium by content creators, and the lessons we’ll learn about immersive storytelling by experimenting with them.

Annabel plays with a Google Cardboard with a Samsung Galaxy S6 Edge attached.
There are few user interfaces as simple as moving your own head. Even Annabel – who struggles with the idea that some screens aren’t touchscreens – got to grips with it in seconds.

It seems to me that the biggest questions that VR content creators will need to start thinking about as we collectively begin to explore this new (or newly-accessible) medium are:

How do we make intuitive user interfaces?

This question mostly relates to creators making “interactive” experiences. Superficially, VR gives user experience designers a running start because there’s little that’s as intuitive as “turning your head to look around” (and, in fact, trying the technology out on a toddler convinced me that it’s adults – who already have an anticipation of what a computer interface ought to be – who are the only ones who’ll find this challenging). On the other hand, most interactive experiences demand more user interaction than simply looking around, and therein lies the challenge. Using a keyboard while you’re wearing a headset is close to impossible (trust me, I’ve tried), although the augmented-reality approach of the Hololens and potentially even the front-facing webcam that’s been added to the HTC Vive PRE might be used to mitigate this. A gamepad is workable, but it’s slightly immersion-breaking in some experiences to hold your hands in a conventional “gamer pose”, as I discovered while playing my Gone Home hackalong: this was the major reason I switched to using a Wiimote.

A pair of Oculus Rift "touch" controllers.
All of the major VR manufacturers are working on single-handed controllers with spatial awareness and accessible buttons. Some also support haptic feedback so that you can “feel” UI components.

So far, I’ve seen a few attempts that don’t seem to work, though. The (otherwise) excellent educational solar system exploration tool Titans of Space makes players stare at on-screen buttons for a few seconds to “press” them, which is clunky and unintuitive: in the real world, we don’t press buttons with our eyes! I understand why they’ve done this: they’re ensuring that their software has the absolute minimum interface requirement that’s shared between the platforms that it supports, but that’s a concern too! If content creators plan to target two or more of the competing systems that will launch this year alone, will they have to make usability compromises?

There’s also the question of how we provide ancillary information to players: the long-established paradigms of “health in the bottom left, ammo in the bottom right” don’t work so obviously when they’re hidden in your peripheral vision. Games like Elite Dangerous have tackled this problem from their inception by making a virtualised “real” user interface comprised of the “screens” in the spaceship around you, but it’s an ongoing challenge for titles that target both VR and conventional platforms in future. Wareable made some great observations about these kinds of concerns, too.

How do we tell stories without forced visual framing?

In my previous blog post, I talked about a documentary that used 360° cameras to “place” the viewer among the protesters that formed the subject of the documentary. In order to provide some context and to reduce the disorientation experienced by “jumping” from location to location, the creator opted to insert “title slides” between scenes with text explaining what would be seen next. But title slides necessitate that the viewer is looking in a particular direction! In the case of this documentary and several other similar projects I’ve seen, the solution was to put the title in four places – at each of the four cardinal directions – so that no matter which way you were looking you’ll probably be able to find one. But title slides are only a small part of the picture.

Two gamers wearing Oculus Rift headsets.
Does anybody else see photos like this and get reminded of the pictures of hooded captives at interrogation camps?

Directors producing content – whether interactive or not – for virtual reality will have to think hard about the implications of the fact that their camera (whether a physical camera or – slightly easier and indeed more-controllable – a simulated camera in a 3D-rendered world) can look in any direction. Sets must be designed to be all-encompassing, which poses huge challenges for the traditional methods of producing film and television programmes. Characters’ exits and entrances must be through believable portals: they can’t simply walk off to the left and stop. And, of course, the content creator must find a way to get the audience’s attention when they need it: watching the first few minutes of Backstage with an Elite Ballerina, for example, puts you in a spacious dance studio with a spritely ballerina to follow… but there’s nothing to stop you looking the other way (perhaps by accident), and – if you do – you might miss some of the action or find it difficult to work out where you’re supposed to be looking. Expand that to a complex, busy scene like, say… the ballroom scene in Labyrinth… and you might find yourself feeling completely lost within a matter of minutes (of course, a feeling of being lost might be the emotional response that the director intends, and hey – VR is great for that!).

Sarah and Jareth dance in the ballroom scene of Labyrinth.
You’re looking the wrong way. Turn around, and you’ll see the best part of the movie.

The potential for VR in some kinds of stories is immense, though. How about a murder mystery story played out in front of you in a dollhouse (showing VR content “in minature” can help with the motion sickness some people feel if they’re “dragged” from scene to scene): you can move your head to peep in to any room and witness the conversations going on, but the murder itself happens during a power cut or otherwise out-of-sight and the surviving characters are left to deduce the clues. In such a (non-interactive) experience the spectator has the option to follow the action in whatever way they like, and perhaps even differently on different playthroughs, putting the focus on the rooms and characters and clues that interest them most… which might affect whether or not they agree with the detective’s assertions at the end…

What new storytelling mechanisms can this medium provide?

As I mentioned in the previous blog post, we’ve already seen the evolution of storytelling media on several ocassions, such as the jump from theatre to cinema and the opportunities that this change eventually provided. Early screenwriters couldn’t have conceived of some of the tools used in modern films, like the use of long flowing takes for establishing shots or the use of fragmented hand-held shots to add an excited energy to fight scenes. It wasn’t for lack of imagination (Georges Méliès realised back in the nineteenth century that timelapse photography could be used to produce special effects not possible in theatre) but rather a lack of the technology and more-importantly a lack of the maturity of the field. There’s an ongoing artistic process whereby storytellers find new ways to manage their medium from one another: Romeo Must Die may have made clever use of a “zoom-to-X-ray” when a combatant’s bones were broken, but it wouldn’t have been possible if The Matrix hadn’t shown the potential for “bullet time” the previous year. And if we’re going down that road: have you seen the bullet time scene in Zotz!, a film that’s older than the Wachowskis themselves?

Clearly, we’re going to discover new ways of telling stories that aren’t possible with traditional “flat screen” media nor with more-immersive traditional theatre: that’s what makes VR as a storytelling tool so exciting.

Perhaps the original cinematic use of bullet time, in Zotz!
The original use of bullet time still wasn’t entirely new, as the original bullet predates it by hundreds of years.

Of course, we don’t yet know what storytelling tools we’ll find in this medium, but some ideas I’ve been thinking about are:

"Fly" motion simulator
There’s no need to build a rollercoaster at all: a good motion simulator plus a VR headset can probably provide a similar experience.
GIF showing a variety people watching VR porn. [SFW]
Nothing in this GIF reflects how people will genuinely watch VR porn. There’ll be a lot more lube and a lot fewer clothes, I guarantee it.

As I’m sure I’ve given away these last three blog posts, I’m really interested in the storytelling potential of VR, and you can bet I’ll be bothering you all again with updates of the things I get to play with later this year (and, in fact, some of the cool technologies I’ve managed to get access to just while I’ve been writing up these blog posts).

If you haven’t had a chance to play with contemprary VR, get yourself a cardboard. It’s dirt-cheap and it’s (relatively) low-tech and it’s nowhere near as awesome as “real” hardware solutions… but it’s still a great introduction to what I’m talking about and it’s absolutely worth doing. And if you have, I’d love to hear your thoughts on storytelling using virtual reality, too.

]]> Immersive Storytelling – Thoughts on Virtual Reality, part 2 https://danq.me/2016/02/05/immersive-storytelling-2/ https://danq.me/2016/02/05/immersive-storytelling-2/#comments Fri, 05 Feb 2016 12:02:50 +0000 https://danq.me/?p=7059 This is the second in a three-part blog post about telling stories using virtual reality. Read all of the parts here.

I’m still waiting to get in on the Oculus Rift and HTC Vive magic when they’re made generally-available, later this year. But for the meantime, I’m enjoying quite how hackable VR technologies are. I chucked my Samsung Galaxy S6 edge into an I Am Cardboard DSCVR, paired it with a gaming PC using TrinusVR, used GlovePIE to hook up a Wii remote (playing games with a keyboard or even a gamepad is challenging if your headset doesn’t have a headstrap, so a one-handed control is needed), and played a game of Gone Home. It’s a cheap and simple way to jump into VR gaming, especially if – like me – you already own the electronic components: the phone, PC, and Wiimote.

An I Am Cardboard, a Samsung Galaxy S6, and a Nintendo Wii Remote
My VR system is more-ghetto than yours.

While the media seems to mostly fixate on the value of VR in “action” gaming – shoot-’em-ups, flight simulators, etc. – I actually think there’s possibly greater value in it more story-driven genres. I chose Gone Home for my experiment, above, because it’s an adventure that you play at your own pace, where the amount you get out of it as a story depends on your level of attention to detail, not how quickly you can pull a trigger. Especially on this kind of highly-affordable VR gear, “twitchy” experiences that require rapid head turning are particularly unsatisfying, not-least because the response time of even the fastest screens is always going to be significantly slower than that of real life. But as a storytelling medium (especially in an affordable form) it’s got incredible potential.

Screengrab from Hong Kong Unrest - a 360° Virtual Reality Documentary.
Nothing quite gives you a feel of the human scale of the Hong Kong protests like being able to look around you, as if you’re stood in the middle of them.

I was really pleased to discover that some content creators are already experimenting with the storytelling potential of immersive VR experiences. An example would be the video Hong Kong Unrest – a 360° Virtual Reality Documentary, freely-available on YouTube. Standing his camera (presumably a Jump camera rig, or something similar) amongst the crowds of the 2014 Hong Kong protests, the creator of this documentary gives us a great opportunity to feel as though we’re standing right there with the protesters. The sense of immersion of being “with” the protesters is, in itself, a storytelling statement that shows the filmmaker’s bias: you’re encouraged to empathise with the disenfranchised Hong Kong voters, to feel like you’re not only with them in a virtual sense, but emotionally with them in support of their situation. I’m afraid that watching the click-and-drag version of the video doesn’t do it justice: strap a Cardboard to your head to get the full experience.

An augmented reality shoot-'em-up game via Microsoft Hololens
Don’t go thinking that I’m not paying attention to the development of the Hololens, too: I am, because it looks amazing. I just don’t know… what it’s for. And, I suspect, neither does Microsoft.

But aside from the opportunities it presents, Virtual Reality brings huge new challenges for content creators, too. Consider that iconic spaghetti western The Good, The Bad, And The Ugly. The opening scene drops us right into one of the artistic themes of the film – the balance of wide and close-up shots – when it initially shows us a wide open expanse but then quickly fills the frame with the face of Tuco (“The Ugly”), giving us the experience of feeling suddenly cornered and trapped by this dangerous man. That’s a hugely valuable shot (and a director’s wet dream), but it represents something that we simply don’t have a way of translating into an immersive VR setting! Aside from the obvious fact that the viewer could simply turn their head and ruin the surprise of the shot, it’s just not possible to fill the frame with the actor’s face in this kind of way without forcing the focal depth to shift uncomfortably.

Opening scene of The Good, The Bad, And The Ugly.
Sergio Leone’s masterpiece makes strategic use of alternating close and wide shots (and shots like the opening, which initially feels open but rapidly becomes claustrophobic).

That’s not to say that there exist stories that we can’t tell using virtual reality… just that we’re only just beginning to find out feet with this new medium. When stage directors took their first steps into filmography in the early years of the 20th century, they originally tried to shoot films “as if” they were theatre (albeit, initially, silent theatre): static cameras shooting an entire production from a single angle. Later, they discovered ways in which this new medium could provide new ways to tell stories: using title cards to set the scene, close-ups to show actors’ faces more-clearly, panning shots, and so on.

Similarly: so long as we treat the current generation of VR as something different from the faltering steps we took two and a half decades ago, we’re in frontier territory and feeling our way in VR, too. Do you remember when smartphone gaming first became a thing and nobody knew how to make proper user interfaces for it? Often your tiny mobile screen would simply try to emulate classic controllers, with a “d-pad” and “buttons” in the corners of the screen, and it was awful… but nowadays, we better-understand the relationship that people have with their phones and have adapted accordingly (perhaps the ultimate example of this, in my opinion, is the addictive One More Line, a minimalist game with a single-action “press anywhere” interface).

Dan plays Back To Dinosaur Island on an Oculus Rift.
A few seconds after this photograph was taken, a T-rex came bounding out from the treeline and I quite-literally jumped out of my seat.

I borrowed an Oculus Rift DK2 from a co-worker’s partner (have I mentioned lately that I have the most awesome co-workers?) to get a little experience with it, and it’s honestly one of the coolest bits of technology I’ve ever had the priviledge of playing with: the graphics, comfort, and responsiveness blows Cardboard out of the water. One of my first adventures – Crytek’s tech demo Back to Dinosaur Island – was a visual spectacle even despite my apparently-underpowered computer (I’d hooked the kit up to Gina, my two-month old 4K-capable media centre/gaming PC: I suspect that Cosmo, my multi-GPU watercooled beast might have fared better). But I’ll have more to say about that – and the lessons I’ve learned – in the final part of this blog post.

]]> https://danq.me/2016/02/05/immersive-storytelling-2/feed/ 3 Immersive Storytelling – Thoughts on Virtual Reality, part 1 https://danq.me/2016/01/31/immersive-storytelling-1/ https://danq.me/2016/01/31/immersive-storytelling-1/#comments Sun, 31 Jan 2016 10:45:36 +0000 https://danq.me/?p=7046 This is the first in a three-part blog post about telling stories using virtual reality. Read all of the parts here.

As part of my work at the Bodleian… but to a greater extent “just for fun”… I’ve spent the last few weeks playing with virtual reality. But first, a history lesson.

Dan stomps around his office wearing a Google Cardboard.
Virtual Reality’s biggest failing is that it’s sheer coolness is equally offset by what an idiot you look like when you’re using it.

This isn’t the first time I’ve used virtual reality. The first time, for me, was in the early 1990s, at the Future Entertainment Show, where I queued for a shot at Grid Busters on a Virtuality 1000-CS. The Virtuality 1000 was powered by an “Expality”: functionally an Amiga 3000 with specially-written software for reading the (electromagnetically-sensed) facing of the headset and the accompanying “space joystick”… and providing output via a pair of graphics cards (one for each eye) to LCD screens. The screens were embedded in chunky bits on the sides of the helmet and projected towards mirrors and lenses at the far end – this apparently being an effort to reduce how “front-heavy” it felt, but I can tell you that in practice a  Virtuality headset felt weighty on your neck, even for its era!

Nonetheless, the experience stuck with me: I returned to school and became the envy of my friends (the nerdy ones, at least) when I told them about my VR adventure, and – not least thanks to programs like Tomorrow’s World and, of course, the episode of Bad Influence that reminded me quite how badly I wanted to get myself down to Nottingham for a go at Legend Quest – I was genuinely filled with optimism that within the decade, playing a VR game would have gone from the fringes of science fiction to being something where everybody-knew-somebody who did it routinely.

A Virtuality 1000 CS system.
A modern computer and VR headset combined probably weighs less than this reconditioned Virtuality 1000 headset.

I never managed to get to play Legend Quest, and that first “VR revolution” swiftly fell flat. My generation was promised all of the hi-tech science, immersion, and magical experience of The Lawnmower Man, but all we were left with was the overblown promises, expensive effects, and ill-considered user experience of, well… The Lawnmower Man. I discovered Virtuality machines in arcades once or twice, but they seemed to be out-of-order more often than not, and they quickly disappeared. You can’t really blame the owners of arcades: if a machine costs you in the region of £40,000 to buy and you can charge, say, £1 for a 3-minute go on it (bear in mind that even the most-expensive digital arcade machines tended to charge only around 30p, at this time, and most were 10p or 20p), and it needs supervision, and it can’t be maintained by your regular guy… well, that swiftly begins to feel like a bad investment.

Jobe's first experience of virtual reality, in 1992's The Lawnmower Man.
The Lawnmower Man has a lot to answer for.

Plus, the fifth generation of games consoles came along: the (original) Sony PlayStation, the Nintendo N64, and – if you really wanted the highest-technology system (with the absolute least imaginative developers) – the Sega Saturn. These consoles came at price points that made them suitable Christmas gifts for the good boys and girls of middle-class parents and sported 3D polygon graphics of the type that had previously only been seen in arcades, and the slow decline of the video arcade accelerated dramatically. But home buyers couldn’t afford five-figure (still moderately-experimental) VR systems, and the market for VR dried up in a matter of years. Nowadays, if you want to play on a Virtuality machine like the one I did, you need to find a collector (you might start with this guy from Leicester, whose website was so useful in jogging my memory while I wrote this blog post).

The Dean's VR machine, in Season 6 of Community, was clearly inspired by Virtuality.
And Jesus wept, for there were no more VR machines anywhere for, like, two decades.

2016 is the year in which this might change. The need for ubiquitous cheap computing has made RAM and even processors so economical that we throw them away when we’re done with them. The demands of modern gaming computers and consoles has given us fast but affordable graphics rendering hardware. And the battle for the hottest new smartphones each year has helped to produce light, bright, high-resolution screens no bigger than the palm of your hand.

In fact, smartphones are now the simplest and cheapest way to play with VR. Under the assumption that you’ve already got a smartphone, you’re only a couple of cheap plastic lenses and a bit of cardboard away from doing it for yourself. So that’s how my team and I started out playing: with the wonderfully-named Google Cardboard. I know that Google Cardboard is old-hat now and all the early adopters have even got their grandmothers using it now, but it’s still a beautiful example of how economical VR threatens to become if this second “VR revolution” takes hold. Even if you didn’t already own a compatible smartphone, you could buy a second-hand one on eBay for as little as £30: that’s an enormous difference from the £40K Virtuality machines of my youth, which had only a fraction of the power.

Liz plays with a Google Cardboard.
An original-style Google Cardboard makes you look as much of a fool as any VR headset does. But more-specifically like a fool with a cardboard box on their head.

I’m going somewhere with this, I promise: but I wanted to have a jumping-off point from which to talk about virtual reality more-broadly first and it felt like I’d be overstretching if I jumped right in at the middle. Y’know, like the second act of The Lawnmower Man. In the next part of this series, I’d like to talk about the storytelling opportunities that modern VR offers us, and some of the challenges that come with it, and share my experience of playing with some “proper” modern hardware – an Oculus Rift.

]]> https://danq.me/2016/01/31/immersive-storytelling-1/feed/ 1 Highlights of 2016 so far https://danq.me/2016/01/06/highlights-of-2016-so-far/ https://danq.me/2016/01/06/highlights-of-2016-so-far/#comments Wed, 06 Jan 2016 20:35:14 +0000 https://danq.me/?p=7041 Despite a full workload and a backlog of both work, personal, volunteering and study emails to deal with, 2016 is off to a pretty good start so far. Here’s some highlights:

 

 

Annabel sporting a full beard.
This one’s the third design of beard she’s had this week – this one’s “like daddy”.

I hope everybody else’s year is kicking off just as well.

* With one possible exception: the other year, an overenthusiastic bouncer insisted that I join a queue of one in turn to show him my ID before he let me into a nightclub at 9:30pm on a Wednesday night. Like I said, overenthusiastic.

]]> https://danq.me/2016/01/06/highlights-of-2016-so-far/feed/ 1 Raspberry Pi VPN Hotspot (or How To Infuriate Theresa May For Under £40) https://danq.me/2015/11/25/raspberry-pi-vpn-hotspot/ https://danq.me/2015/11/25/raspberry-pi-vpn-hotspot/#comments Wed, 25 Nov 2015 23:12:56 +0000 https://danq.me/?p=7030 As you’re no-doubt aware, Home Secretary Theresa May is probably going to get her way with her “snooper’s charter” by capitalising on events in Paris (even though that makes no sense), and before long, people working for law enforcement will be able to read your Internet usage history without so much as a warrant (or, to put it as the UN’s privacy chief put it, it’s “worse than scary”).

John Oliver on Last Week Tonight discusses the bill.
Or as John Oliver put it, “This bill could write into law a huge invasion of privacy.” Click to see a clip.

In a revelation that we should be thankful of as much as we’re terrified by, our government does not understand how the Internet works. And that’s why it’s really easy for somebody with only a modicum of geekery to almost-completely hide their online activities from observation by their government and simultaneously from hackers. Here’s a device that I built the other weekend, and below I’ll tell you how to do it yourself (and how it keeps you safe online from a variety of threats, as well as potentially giving you certain other advantages online):

"Iceland", one of my Raspberry Pi VPN hotspots
It’s small, it’s cute, and it goes a long way to protecting my privacy online.

I call it “Iceland”, for reasons that will become clear later. But a more-descriptive name would be a “Raspberry Pi VPN Hotspot”. Here’s what you’ll need if you want to build one:

From here on, this post gets pretty geeky. Unless you plan on building your own little box to encrypt all of your home’s WiFi traffic until it’s well out of the UK and close-to-impossible to link to you personally (which you should!), then you probably ought to come back to it another time.

Here’s how it’s done:

1. Plug in, boot, and install some prerequisites

Plug the WiFi dongle into a USB port and connect the Ethernet port to your Internet router.  Boot your Raspberry Pi into Raspbian (as described in the helpsheet that comes with it), and run:

sudo apt-get install bridge-utils hostapd udhcpd bind9 openvpn

2. Make HostAPD support your Edimax dongle

If, like me, you’re using an Edimax dongle, you need to do an extra couple of steps to make it work as an access point. Skip this bit if you’re using one of the other dongles I listed or if you know better.

wget http://dl.dropbox.com/u/1663660/hostapd/hostapd.zip
unzip hostapd.zip
sudo mv /usr/sbin/hostapd /usr/sbin/hostapd.original
sudo mv hostapd /usr/sbin/hostapd.edimax
sudo ln -sf /usr/sbin/hostapd.edimax /usr/sbin/hostapd
sudo chown root.root /usr/sbin/hostapd
sudo chmod 755 /usr/sbin/hostapd

3. Set up OpenVPN

Get OpenVPN configuration files from your VPN provider: often these will be available under the iOS downloads. There’ll probably be one for each available endpoint. I chose the one for Reyjkavik, because Iceland’s got moderately sensible privacy laws and I’m pretty confident that it would take judicial oversight for British law enforcement to collaborate with Icelandic authorities on getting a wiretap in place, which is the kind of level of privacy I’m happy with. Copy your file to /etc/openvpn/openvpn.conf and edit it: you may find that you need to put your VPN username and password into it to make it work.

sudo service openvpn start

You can now test your VPN’s working, if you like. I suggest connecting to the awesome icanhazip.com and asking it where you are (you can use your favourite GeoIP website to tell you what country it thinks you’re in, based on that):

curl -4 icanhazip.com

Another option would be to check with a GeoIP service directly:

curl freegeoip.net/json/

4. Set up your firewall and restart the VPN connection

Unless your VPN provider gives you DNAT (and even if they do, if you’re paranoid), you should set up a firewall to allow only outgoing connections to be established, and then restart your VPN connection:

sudo iptables -A INPUT -i tun0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A INPUT -i tun0 -j DROP
sudo sh -c "iptables-save > /etc/iptables.nat.vpn.secure"
sudo sh -c "echo 'up iptables-restore < /etc/iptables.nat.vpn.secure' >> /etc/network/interfaces"
sudo service openvpn restart

5. Configure your WiFi hotspot

Configure bind as your DNS server, caching responses on behalf of Google’s DNS servers, or another DNS server that you trust. Alternatively, you can just configure your DHCP clients to use Google’s DNS servers directly, but caching will probably improve your performance overall. To do this, add a forwarder to /etc/bind/named.conf.options:

forwarders {
  8.8.8.8;
  8.8.4.4;
};

Restart bind, and make sure it loads on boot:

sudo service bind9 restart
sudo update-rc.d bind9 enable

Edit /etc/udhcpd.conf. As a minimum, you should have a configuration along these lines (you might need to tweak your IP address assignments to fit with your local network – the “router” and “dns” settings should be set to the IP address you’ll give to your Raspberry Pi):

start 192.168.0.2
end 192.168.0.254
interface wlan0
remaining yes
opt dns 192.168.0.1
option subnet 255.255.255.0
opt router 192.168.0.1
option lease 864000 # 10 days

Enable DHCP by uncommenting (remove the hash!) the following line in /etc/default/udhcpd:

#DHCPD_ENABLED="yes"

Set a static IP address on your Raspberry Pi in the same subnet as you configured above (but not between the start and end of the DHCP list):

sudo ifconfig wlan0 192.168.0.1

And edit your /etc/network/interfaces file to configure it to retain this on reboot (you’ll need to use tabs, not spaces, for indentation):

iface wlan0 inet static
  address 192.168.0.1
  netmask 255.255.255.0

And comment out the lines relating to hot-plugging of WiFi adapters/network hopping:

#allow-hotplug wlan0
#wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
#iface default inet manual

Right – onto hostapd, the fiddliest of the tools you’ll have to configure. Create or edit /etc/hostapd/hostapd.conf as follows, but substitute in your own SSID, hotspot password, and channel (to minimise interference, which can slow your network down, I recommend using WiFi scanner tool on your mobile to find which channels your neighbours aren’t using, and use one of those – you should probably avoid the channel your normal WiFi uses, too, so you don’t slow your own connection down with crosstalk):

interface=wlan0
driver=nl80211
ssid=your network name
hw_mode=g
channel=6
macaddr_acl=0
auth_algs=1
ignore_broadcast_ssid=0
wpa=2
wpa_passphrase=your network password
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP
rsn_pairwise=CCMP

Hook up this configuration by editing /etc/default/hostapd:

DAEMON_CONF="/etc/hostapd/hostapd.conf"

Fire up the hotspot, and make sure it runs on reboot:

sudo service hostapd start
sudo service udhcpd start
sudo update-rc.d hostapd enable
sudo update-rc.d udhcpd enable

Finally, set up NAT so that people connecting to your new hotspot are fowarded through the IP tunnel of your VPN connection:

sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
sudo sh -c "echo net.ipv4.ip_forward=1 >> /etc/sysctl.conf"
sudo iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE
sudo sh -c "iptables-save > /etc/iptables.nat.vpn.secure"

6. Give it a go!

Connect to your new WiFi hotspot, and go to your favourite GeoIP service. Or, if your VPN endpoint gives you access to geographically-limited services, give those a go (you’d be amazed how different the Netflix catalogues are in different parts of the world). And give me a shout if you need any help or if you have any clever ideas about how this magic little box can be improved.

Further reading:

]]> https://danq.me/2015/11/25/raspberry-pi-vpn-hotspot/feed/ 4 Twee2 – Interactive Fiction Authoring for Geeks https://danq.me/2015/10/22/twee2/ https://danq.me/2015/10/22/twee2/#comments Thu, 22 Oct 2015 11:55:27 +0000 https://danq.me/?p=7000 There’s a wonderful tool for making web-based “choose your own adventure”-type games, called Twine. One of the best things about it is that it’s so accessible: if you wanted to, you could be underway writing your first ever story with it in about 5 minutes from now, without installing anything at all, and when it was done you could publish it on the web and it would just work.

Screenshot of a Twine 2 story map
A “story map” in Twine 2. Easy interactive fiction writing for normal people.

But the problem with Twine is that, in its latest and best versions, you’re trapped into using the Twine IDE. The Twine IDE is an easy-to-use, highly visual, ‘drag-and-drop’ interface for making interactive stories. Which is probably great if you’re into IDEs or if you don’t “know better”… but for those of us who prefer to do our writing in a nice clean, empty text editor like Sublime or TextMate or to script/automate our builds, it’s just frustrating to lose access to the tools we love. Plus, highly-visual IDEs make it notoriously hard to collaborate with other authors on the same work without simply passing it back and forwards between you: unless they’ve been built with this goal in mind, you generally can’t have two people working in the same file at the same time.

Sublime Text demonstrating multi-line-selection.
Now THIS is what code editing should look like.

Earlier versions of Twine had a command-line tool called Twee that perfectly filled this gap. But the shiny new versions don’t. That’s where I came in.

In that way that people who know me are probably used to by now, I was very-slightly unsatisfied with one aspect of an otherwise fantastic product and decided that the correct course of action was to reimplement it myself. So that’s how, a few weeks ago, I came to release Twee2.

Twee2 logo
Twee2’s logo integrates the ‘branching’ design of Twine adventures with the ‘double-colon’ syntax of Twee.

If you’re interested in writing your own “Choose Your Own Adventure”-type interactive fiction, whether for the world or just for friends, but you find user-friendly IDEs like Twine limiting (or you just prefer a good old-fashioned text editor), then give Twee2 a go. I’ve written a simple 2-minute tutorial to get you started, it works on Windows, MacOS, Linux, and just-about everything else, and it’s completely open-source if you’d like to expand or change it yourself.

(there are further discussions about the concept and my tool on Reddit here, here, here and here, and on the Twinery forums herehere and here)

Get Twee2

]]> https://danq.me/2015/10/22/twee2/feed/ 2 Into the Lair of the Bladder Monster https://danq.me/2015/09/18/the-bladder-monster-2/ https://danq.me/2015/09/18/the-bladder-monster-2/#comments Fri, 18 Sep 2015 18:00:57 +0000 https://danq.me/?p=6987 Warning: this blog post contains pictures of urine, invasive equipment, and the inside of a bladder. It’s probably safe for all audiences, but you might like to put your glass of apple juice down for a minute or two. The short of it all is that I’m probably healthy.

Since my hospitalisation the other month with a renal system infection, I’ve undergone a series of investigations to try to determine if there’s an underlying reason that I fell ill. As my doctor explained to me, it’s quite possible that what I’d experienced was a random opportunistic infection (perhaps aided by a course of unrelated antibiotics I’d been on earlier this year or by certain lifestyle habits), but if that wasn’t the case – if there were some deeper explanation for my health problems – it was important to find out sooner, rather than later.

A sterile pot full of Dan Q's urine.
I’ve peed in so many little pots over the last few months! If you laid them end-to-end across your kitchen counter, then people would think that you were some kind of weirdo.

Early on I had several ultrasound scans of my bladder (at a number of different times and at a variety of levels of fullness) and one of my kidneys, the latter of which revealed some “minor scarring” of one of them which apparently isn’t something I should be worried about… although I wish they’d started the two-page letter I got with that rather than opening with, effectively, “Contrary to what we told you at the hospital, we did later see something wrong with you…” But still, good to be reassured that this is probably not an issue.

Ultrasound scan of one of Dan Q's kidneys.
An ultrasound scan of one of my kidneys. Can you tell the sex yet?

More recently, I went to the hospital to have a “flow rate test” and a cystoscopy. The flow rate test involved the most-ghetto looking piece of NHS equipment I’ve ever seen: functionally, it seemed to be little more than a funnel on top of a large measuring beaker, in turn on top of a pressure-sensitive digital scale. The scale was connected up to the only fancy-looking bit of equipment in the room, a graphing printer that output the calculated volume (based on their weight) of the same and, more-importantly, the rate of change: the “flow rate” of the stream of urine.

A stream of urine pours down into a funnel.
I’m right, aren’t I? That’s basically a kitchen funnel, isn’t it?

I suppose one advantage of using equipment like this is that it basically operates itself. Which meant that the nurse was able to give me five seconds worth of instruction and then leave the room, which saved us from our own Britishness forcing us to make small-talk while I urinated in front of her or something. Ultimately, I turned out to be within the range of normalcy here, too, although I was a little disappointed to find that the ward didn’t maintain a daily “score board” of flow rates, as sort-of a science-backed literal pissing contest.

A graphing printer describes Dan Q's urine flow. The 'flow rate' graph shows an initial peak, then a trough, then continues to a higher sustained peak.
Apparently not all men experience that ‘spurt-and-then-full-pressure’ thing you’ll see on the upper graph, when they start to pee, but some of us do, and I’m told it’s perfectly normal. I’m learning so much!

Finally came the cystoscopy, and this was the bit that I’d been most-nervous about. This procedure involves the insertion of a long flexible tube into the urethra at the tip of the penis, under local anasthetic, and pushing it all the way down, through the sphincter, down through the prostate and then back up into the bladder. It’s then used as a channel to pump water into the bladder, filling it to capacity and stretching out the sides, after which the fibreoptic cord (and light) that runs along its length is used to look around inside the bladder to inspect for any of a plethora of different problems.

Cystoscopy equipment, ready for insertion.
You’re going to put that WHERE?

The doctor invited me to watch with him on the monitor, which I initially assumed was because I was clearly interested in everything and kept asking questions, but in hindsight I wonder if it’s just that he – quite rightly – assumed that I might have panicked if I’d have been looking in the direction of the piece of equipment he brought in and jabbed at my penis with. I only looked at it while it was on its way out, and my god its a scary-looking thing: sort of like a cross between a tyre pressure gauge and a blowtorch. The first few inches were painless – the local anasthetic had made me completely numb right up to and including the external sphincter, which is at the base of the penis. However, what I can only assume was the second sphincter complained of the discomfort, and it stung pretty sharply any time the doctor would twist the cystoscope to change the angle of the picture.

View up a urethra, from a cystoscope.
The view as you ‘travel’ up the urethra looks pretty much like I expected. With a motion simulator, it would make a pretty cool ride!

Seeing the inside of your own body is an amazing experience. I mean: it’s not amazing enough to even be worth the experience of a cystoscopy, never mind the illness that in my case preceeded it… but it’s still pretty cool. The ultrasounds were interesting, but there’s nothing quite so immersive as seeing a picture of the inside of your own bladder, gritting your teeth while the doctor points to an indentation and explains that it’s the opening to the ureter that connects to your own left kidney!

Unfortunately I neglected to take my phone into the operating room, having put it into a locker when I changed into a gown, and so I wasn’t able to (as I’d hoped) take photos of the inside of my own bladder. So you’ll have to make do with this video I found, which approximates the experience pretty well. The good news is that there’s probably nothing wrong with me, now that the infection from earlier this year has passed: nothing to suggest that there’s any deeper underlying issue that caused me to get sick, anyway!

The bad news is that while the procedure itself was shorter and more-bearable than I’d expected, the recovery’s been a real drag. A week later, it still hurts a lot to urinate (although I’ve stopped yelping out loud when I do so) and my crotch is still too sore for me to be able to cycle. I’ve also discovered that an errection can be painful enough to wake me up, which is definitely not the most-pleasant way I’ve been roused by a penis. But it’s getting better, day by day, and at least I know for sure that I’m more-or-less “right” in the renal system, now.

]]> https://danq.me/2015/09/18/the-bladder-monster-2/feed/ 11 Post-It Minesweeper https://danq.me/2015/09/15/post-it-minesweeper/ https://danq.me/2015/09/15/post-it-minesweeper/#comments Tue, 15 Sep 2015 19:40:30 +0000 https://danq.me/?p=6976 Remember Minesweeper? It’s probably been forever since you played, so go have a game online now. And there went your afternoon.

A game of Microsoft Minesweeper in progress.
This is actually a pretty tough move.

My geek-crush Ben Foxall posted on Twitter on Monday morning to share that he’d had a moment of fun nostalgia when he’d come into the office to discover that somebody in his team had covered his monitor with two layers of Post-It notes. The bottom layer contained numbers – and bombs! – to represent the result of a Minesweeper board, and the upper layer ‘covered’ them so that individual Post-Its could be removed to reveal what lay beneath. Awesome.

Ben Foxall discovers Post-It Minesweeper
Unlike most computerised implementations of Minesweeper, the first move isn’t guaranteed to be safe. Tread carefully…

Not to be outdone, I hunted around my office and found some mini-Post-Its. Being smaller meant that I could fit more of them onto a monitor and thus make a more-sophisticated (and more-challenging!) play space. But how to generate the board? Sure: I could do it by hand, but that doesn’t seem very elegant at all – plus, humans make really bad random number generators! I didn’t need quantum-tunnelling-seeded Minesweeper (yes, that’s a thing) levels of entropy, sure, but it’d still be nice to outsource the heavy lifting to a computer, right?

Screenshot of my Post-It Minesweeper board generator.
Yes, I’m quite aware of the irony of using a computer to generate a paper-based version of a computer game, why do you ask?

So naturally, I wrote a program to do it for me. Want to see? It’s at danq.me/minesweeper. Just line up some Post-Its on a co-worker’s monitor to work out how many you can fit across it in each dimension (I found that I could get 6 × 4 standard-sized Post-Its but 7 × 5 or even 8 × 5 mini-sized Post-Its very comfortably onto one of the typical widescreen monitors in my office), decide how many mines you want, and click Generate. Don’t like the board you get? Click it again!

Liz McCarthy tweets about her experience of being given a Post-It Minesweeper game to play.
I set up the first game on my colleague Liz’s computer, before she came in this morning.

And because I was looking for a fresh excuse to play with Periscope, I broadcast the first game I set up live to the Internet. In the end, 66 people ended up watching some or all of a paper-based game of Minesweeper played by my colleague Liz, including moments of cheering her on and, in one weird moment, dispair at the revelation that she was married. The internet’s strange, yo.

Anyway: in case you missed the Periscope broadcast, I’ve put it on YouTube. Sorry about the portrait-orientation filming: I think it’s awful, too, but it’s a Periscope thing and I haven’t installed the new update that fixes it yet.

Now go set up a game of Post-It Minesweeper for a friend or co-worker.

]]> https://danq.me/2015/09/15/post-it-minesweeper/feed/ 3 Anniversary at Wriggles Brook https://danq.me/2015/08/29/anniversary-at-wriggles-brook/ https://danq.me/2015/08/29/anniversary-at-wriggles-brook/#comments Sat, 29 Aug 2015 08:28:20 +0000 https://danq.me/?p=6960 Three weeks ago was (give or take a few weeks because we’ve never bothered with accuracy) the end of Ruth and I’s 8th year together, and we marked the ocassion with a mini-break away for a few nights. We spent the first two nights in a ‘showman’-style gypsy caravan in Herefordshire, and it was amazing enough that I wanted to share it with you:

'Showman' caravan at Wriggles Brook
It wasn’t quite dusk yet, but we couldn’t resist the urge to light the fire (and the dozens of tiny lanterns).

The place we went was Wriggles Brook, a ‘glamping’-style site in the shadow of the Forest of Dean. In a long field that twists its way alongside a babbling brook, the owners have set up a trio of traditional horse-drawn caravans, each in a wooded clearing that isolates it from the others. Two of the caravans are smaller, designed just for couples (who are clearly the target market for this romantic getaway spot), but we took the third, larger, (centenarian!) one, which sported a separate living room and bedroom.

Annabel in wellies stomps through the orchard at Wriggles Brook.
Between our caravan and the others the owners grew a varied orchard, which Annabel found particularly interesting. By which I mean delicious.

The bedroom was set up so that children could be accomodated in a bunk under the adults (with their own string of fairy lights and teeny-tiny windows, but after she bumped her head on the underside of the beams Annabel decided that she didn’t want to sleep there, so we set up her travel cot in the living room.

Dan and Annabel on the hammock.
Annabel and I swinging on a hammock near the serpentine stream. She clearly misinterpreted the word roots, and spent the entire trip calling it a “hat-cot”.

So yeah: a beautiful setting, imaginative and ecologically-friendly accomodation, and about a billion activities on your doorstep. Even the almost-complete lack of phone signal into the valley was pretty delightful, although it did make consulting Google Maps difficult when we got lost about 20 minutes out from the place! But if there’s one thing that really does deserve extra-special mention, it’s the food!

Steam train in the Forest of Dean.
Nearby activities include steam trains. That’s all I needed to hear, really.

Our hosts were able to put on a spectacular breakfast and evening meal for us each night, including a variety of freshly-grown produce from their own land. We generally ate in their mini dining room – itself a greenhouse for their grapevines – but it was equally-nice to have pancakes delivered to the picnic table right outside our caravan. And speaking as somebody who’s had their fair share of second-rate veggie breakfasts over the last… what, four and a half years?… it was a great relief to enjoy a quite-brilliant variety of vegetarian cuisine from a clearly-talented chef.

A speed bump sign in heavy undergrowth.
I’m not sure why the Wriggles Brook site has ocassional signs like this sticking out of the undergrowth, but they sort-of fit the eccentricity of the place.

So yeah – five stars for Wriggles Brook in Herefordshire if you’re looking for an awesome romantic getaway, with or without an accompanying toddler. Ruth and I later palmed the little one off on JTA so that we could have a night away without her, too, which – while fun (even if we didn’t get to try all 280+ gins at the restaurant we ate at) – wasn’t quite so worthy of mention as the unusual gypsy-caravan-escape that had preceeded it. I’m hoping that we’ll get out to Wriggles Brook again.

]]> https://danq.me/2015/08/29/anniversary-at-wriggles-brook/feed/ 5 The Bladder Monster https://danq.me/2015/06/24/the-bladder-monster/ https://danq.me/2015/06/24/the-bladder-monster/#comments Wed, 24 Jun 2015 11:41:09 +0000 https://danq.me/?p=6942 As I mentioned last week, I’ve been ill. For those who wanted the grisly details, well: here you go.

Warning: this blog post contains frank subjective descriptions of the symptoms of slightly-icky medical conditions including photographs. It’s probably safe for all audiences, but you might not want to be drinking anything while you read it.

The trouble began, I suppose, about a month and a half ago, when a contracted what seemed to be a minor urinary tract infection. If you’re a woman then, statistically-speaking, I probably don’t need to tell you what that is, but for the benefit of the men: it’s what happens when bacteria (or, sometimes, a fungus or virus) infects the renal system: the kidneys, bladder, ureters, and especially the urethra. It’s not pleasant: it gives you the feeling of needing to pee the whole time, makes it harder to pee, and – when you do – it feels a little bit like you’re piss is made of lava.

Microscope view of infected urine.
This urine contains white blood cells (the big circles) and bacteria (the tiny ‘wormlike’ things). Healthy urine contains little to none of these. Anybody else feel like they’re playing Spore?

Despite it not being common for men (more on that later), I’ve had mild UTIs on a couple of ocassions in my life, and I’d always found that ensuring that I got plenty of water and a full RDA of vitamin C was more than enough to make it clear up all by itself within a couple of days. So that’s what I started doing. But then things took a turn for the worse: I started getting a stabbing pain in my left kidney. Recognising this as being pyelonephritis, I went to the doctor who prescribed me a course of the antibiotic ciprofloxacin. Within a couple of days I was feeling right as rain (of course I continued to finish the course of drugs, although I was interested to see that that advice is starting to become controversial).

Dan and Annabel have a picnic in the park.
Happy, healthy, and out for a picnic. I guess everything turned out okay, right?

Naturally I was a little disappointed when, the week before last, I started getting UTI-like pain again, followed very swiftly this time by pain in my bladder that constantly felt a little like I was recovering from being punched. Back to the doctor I went, where (after the usual tests to work out what the most-likely best-antibiotic to use was) I was prescribed a course of nitrofurantoin. I’d never had this particular drug before, and it wasn’t initially clear which of the escalating ill-effects I was experiencing were symptoms of the infection and which were side-effects of the medication: it started with joint pain, then nausea, then diarrhoea, then a full-on fever. It was at the point that I was fully-clothed in bed, running a temperature and soaked in sweat but still feeling cold and shivering that Ruth called 111, who told her to take me to A&E.

(Which, like her care for me in general, she did fabulously well, except for a little bit where she sort-of ran me over in the car park of the hospital: thankfully some friendly paramedics were standing around and were able to drag me into the building. Anyway, I don’t have much memory of that bit and I certainly don’t have any amusing photos, so I’ll skip over it.)

Dan, shortly before inpatient admission but already recovering from the worst parts of his hospital visit, last week.
Feverish to the point of delirium, I don’t have much recollection of the first few hours(?) in the hospital. But I was alert enough to request that a photo was taken for the inevitable blog post. Ruth, however, insisted upon waiting until I apparently looked a lot less like I was about to die: so here it is!

A few tests later, the medical staff seemed confident that what I was experiencing was not an allergic reaction to the antibiotic (however, I see that they still made a note of it as a risk on my notes!) but was a progression of the infection, which seemed to have crossed over from the tissues of my renal system and into my bloodstream and was now generally causing havoc by attacking the rest of my body. They hooked me up to a drip of an intravenous antibiotic and kept me stocked with painkillers, then sent me up to the urology ward and set me up with a “bed” (urology mostly deals with outpatients, and so my ‘bed’ was actually a trolley, but they wanted to keep me close to the urologists in case of any complications).

The view from my hospital ward window: Oxford city is visible in the distance.
It was only the following morning, with the delirium passed, that I realised that I was on the sixth floor. Looking out of my window, I could just make out the spires of the Bodleian Library in the distance, so I dropped my coworkers an email to apologise for not being on my way there.

A consultant switched me to a week’s course of yet-another different antibiotic – co-amoxiclav – and recommended keeping me in for another night. Now, I think that co-amoxiclav is a really interesting drug, so I’m going to be a bit of a nerd and tell you about that for a bit (I promise we’ll get back to my health in a moment: if you don’t want the science bit, just scroll past the diagrams to the next photo).

Amoxicillin, with lactam ring highlighted.
This is amoxicillin. I’ve highlighted in blue the lactam ring, which is the important bit.

Co-amoxiclav is a mixture of two drugs. The first is the antibiotic amoxicillin. Amoxicillin belongs to a class of antibiotics (which includes penicllin) called β-lactams, which is the most-commonly used family of antibiotics. These antibiotics contain a four-point lactam ‘ring’ (highlighted in blue above), and the way that they work is that this part of the molecule bonds with a particular protein common to all gram-positive bacteria. Normally this protein is responsible for producing peptidoglycan, which is an essential ingredient in the cell walls of these kinds of bacteria, but when it gets locked to a β-lactam ring it stops working. As a result, when the bacterium reproduces the new child doesn’t have a proper cell wall, and can’t survive long in even the least-hostile environments.

Of course, we’re in a medical arms race right now because some of the bacteria which we’re targetting with antibiotics are becoming resistant. And here’s one what that they’re doing so: some of these bacteria have evolved to produce beta-lactamase, also bonds with beta-lactam rings, adding an OH to them and making them useless. Bummer, eh?

Clavulanic acid, with beta-lactam ring highlighted in blue.
And this is clavulanic acid. Recognise that shape on the left-hand side of the molecule (highlighted in blue)? Yup: it’s another lactam ring.

The second drug in co-amoxiclav, then, is clavulanic acid, which was discovered in the 1970s and started being added to drugs in the 1980s. Despite having a β-lactam ring (as you’ll see in blue above), clavulanic acid by itself it isn’t an effective antibiotic (for reasons I can’t quite get my head around – anyone want to help me?). But what it’s great at is bonding that lactam ring to beta-lactamase, thereby deactivating the bacterial counter-offensive and allowing the amoxicillin to carry on working, combating resistance.

So what you’ve got in co-amoxiclav is a an antibiotic and a chemical that counteracts the effects of a chemical that deactivates that antibiotic. Wow! It’s things like this that really make me wish I had a brain for biology!

A cannula in the back of Dan's hand.
These things aren’t terribly comfortable when you’re trying to sleep.

I was eventually discharged from hospital and released to go home for lots of bed rest and water, along with a further week’s course of co-amoxiclav. Unfortunately it turns out that I’m one of the unlucky folks for whom amoxicillin makes me dizzy, so I spent most of that week lying down in-between wobbly vertigo-filled trips to and from the bathroom. But it worked! Within a few days I was feeling much better and by the end of last week I was able to work from home (and actually feel like I was useful again!).

Free of symptoms and off the drugs, I returned to work properly on Monday morning and everything seemed fine. Until, late in the morning, I went to the bathroom and started pissing blood.

A kitten lying on its side.
I took a picture, but it’s too grim even for this blog post. Here, have a kitten instead. Do an image search for ‘gross hematuria’ if you want a clue: the kitten will still be waiting here when you need it.

Now apparently blood in your urine, while horrifying when it happens to you unexpectedly, isn’t actually a sign of a medical emergency. I was starting to get bladder pain again, quite intensely, so I excused myself from work and called the urology ward, who decided that I wasn’t in bad enough a condition to go and see them but sent me straight to my GP, who gave me another fortnight’s worth of co-amoxiclav. They’re monitoring my progress with urine and blood samples and if by Friday it’s not having an impact, they’re going to want to send me back to hospital (hopefully only as an outpatient) and pump me full of the intraveneous stuff again. So… fingers crossed for a good result out of these drugs.

Co-amoxiclav tablet in packaging.
My co-amoxiclav tablets each come individually wrapped in a nitrogen-filled foil bag. I’ve no idea what it is that they’re concerned that they’ll react with, but I’m eating three of them a day anyway.

I was hoping that by this point I’d be writing this blog post and telling you all about how I’d fought the bladder monster and won. But it looks like I won’t be able to claim that victory for another week or two, yet. All I know is that I searched for “bladder monster” and found this. Yeah: that feels about right.

]]> https://danq.me/2015/06/24/the-bladder-monster/feed/ 16 Calculating Pi (when you’re ill) https://danq.me/2015/06/14/calculating-pi/ https://danq.me/2015/06/14/calculating-pi/#comments Sun, 14 Jun 2015 19:32:56 +0000 https://danq.me/?p=6927 So, I’ve not been well lately. And because a few days lying on my back with insufficient mental stimulation is a quick route to insanity for me, I’ve been trying to spend my most-conscious moment doing things that keep my brain ticking over. And that’s how I ended up calculating pi.

Dan, shortly before inpatient admission but already recovering from the worst parts of his hospital visit, last week.
When I say I’ve been unwell, that might be an understatement. But we’ll get to that another time.

Pi (or π) is, of course, the ratio of the circumference of a circle to its diameter, for every circle. You’ll probably have learned it in school as 3.14, 3.142, or 3.14159, unless you were one of those creepy kids who tried to memorise a lot more digits. Over the years, we’ve been able to calculate it to increasing precision, and although there’s no practical or theoretical reason that we need to know it beyond the 32 digits worked out by Ludolph van Ceulen in the 16th Century, it’s still a fascinating topic that attracts research and debate.

Graph illustrating the calculation of digits of pi over the millenia. Note the logarithmic scale on the left and the staggered scale on the bottom axis.
Our calculation of pi has rocketed since the development of the digital computer.

Most of the computer-based systems we use today are hard to explain, but there’s a really fun computer-based experimental method that can be used to estimate the value of pi that I’m going to share with you. As I’ve been stuck in bed (and often asleep) for the last few days, I’ve not been able to do much productive work, but I have found myself able to implement an example of how to calculate pi. Recovery like a nerd, am I right?

A "pi pie", from a Pi Day celebration.
Pi goes on forever. Pie, sadly, comes to an end.

Remember in school, when you’ll have learned that the formula to describe a circle (of radius 1) on a cartesian coordinate system is x2 + y2 = 1? Well you can work this backwards, too: if you have a point on a grid, (x,y), then you can tell whether it’s inside or outside that circle. If x2 + y2 < 1, it’s inside, and if x2 + y2 > 1, it’s outside. Meanwhile, the difference between the area of a circle and the area of a square that exactly contains it is π/4.

A circle of radius 1 at the intersection of the axes of a cartesian coordinate system.
Think back to your school days. Ever draw a circle like this? Do the words “cartesian coordinates” ring any bells?

Take those two facts together and you can develop an experimental way to determine pi, called a Monte Carlo method. Take a circle of radius 1 inside a square that exactly contains it. Then randomly choose points within the square. Statistically speaking, these random points have a π/4 chance of occuring within the circle (rather than outside it). So if we take the number of points that lie within the circle, divide that by the total number of points, and then multiply by 4, we should get something that approaches the value of pi. You could even do it by hand!

Output of Dan's demonstration of the Monte Carlo method as used to approximate the value of pi.
I wrote some software to do exactly that. Here’s what it looks like – the red points are inside the circle, and the black points are outside.

The software illustration I’ve written is raw Javascript, HTML, and SVG, and should work in any modern web browser (though it can get a little slow once it’s drawn a few thousand points!). Give it a go, here! When you go to that page, your browser will start drawing dots at random points, colouring them red if the sum of the squares of their coordinates is less than 1, which is the radius of the circle (and the width of the square that encompasses it). As it goes along, it uses the formula I described above to approximate the value of pi. You’ll probably get as far as 3.14 before you get bored, but there’s no reason that this method couldn’t be used to go as far as you like: it’s not the best tool for the job, but it’s super-easy to understand and explain.

Oh, and it’s all completely open-source, so you’re welcome to take it and do with it what you wish. Turn off the graphical output to make it run faster, and see if you can get an accurate approximation to 5 digits of pi! Or slow it down so you can see how the appearance of each and every point affects the calculation. Or adapt it into a teaching tool and show your maths students one way that pi can be derived experimentally. It’s all yours: have fun.

And I’ll update you on my health at some other point.

]]> https://danq.me/2015/06/14/calculating-pi/feed/ 5 Supermarket “price match” deals are marketing genius https://danq.me/2015/06/09/supermarket-brand-match/ https://danq.me/2015/06/09/supermarket-brand-match/#comments Tue, 09 Jun 2015 18:34:30 +0000 https://danq.me/?p=6916 It’s been almost five years since Sainsbury’s supermarkets pioneered the “brand match” idea, which rivals Tesco and Asda later adopted into their own schemes, and I maintain that it’s one of the cleverest pieces of marketing that I’ve ever seen. In case you’ve not come across it before, the principle is this: if your shopping would have been cheaper at one of their major competitors, these supermarkets will give you a voucher for the difference right there at the checkout. Properly advertised (e.g. not in ways that get banned for being misleading), these schemes are an incredibly-compelling tool: no consumer should say no to getting the best possible prices without having to shop around, right?

A Sainsbury's Brand Match voucher worth 62p.
This voucher implies that I’d have saved 62p by shopping elsewhere. But that’s not the whole story.

But it’s nowhere near as simple as that. For a start, the terms and conditions (Asda, Sainsbury’s, Tesco) put significant limitations on how the schemes work. You need to buy at least a certain number of items (8 at Asda, 10 at the other two). Those items must be directly-comparable to competitors’ items: which basically means that only branded products count, but even among them, the competitor must stock the exact same size or else it doesn’t count, even if it would have been cheaper to buy two half-sized products there. There are upper limits to the value of the vouchers (usually £10) and the number that you can use per transaction or per month. “Buy X get Y free” offers are excluded. And there’s a huge list of not-compared products which may include batteries, toys, DVDs, some alcoholic drinks, cosmetics, homeware, flowers, baby formula, light bulbs, books, and anything (even non-medicines) from the pharmacy aisle.

Tesco Price Promise voucher worth 11p off my next shop.
A whole 11p off my next shop? That’s absolutely worth me carrying this piece of paper around in my wallet and trying not to lose it for a week.

But even if it only applies to some of your shopping – the stuff that’s easy to directly compare – it’s still a good deal, right? You’re getting money back towards what you would have saved if you’d have gone up the road? Not necessarily. Let us assume that on average the prices of these three supermarket giants are pretty much the same. Individual products might each be a little more expensive here and a little cheaper there, but if you buy a large enough trolley-load you’re not going to notice the difference. Following me so far? What does this mean for the voucher: it means that it no longer remotely represents what you would have saved if you’d actually been “shopping around”. Let’s take a concrete example:

Photo courtesy Johnathan Harford, used under a Creative Commons license.
A typical basket at any one of these supermarkets will, on average, come to about the same price… even if individual items vary wildly.

Suppose that this is my somewhat-eccentric shopping list (I wanted to select a variety of comparable branded products), and I’m considering shopping at either Sainsbury’s or Tesco:

Not too unreasonable, right? I’ve made a spreadsheet showing my working, where you’ll see today’s prices for each of these items (along with the actual brands and package sizes I’ve selected), if you’d like to check my maths, because here comes the clever bit.

Sainsbury's Basics range baked beans and coleslaw
These products don’t count as comparable. And personally, I’m not sure that “basics”-range coleslaw is likely to even count as ‘food’.

Based on my calculation, taking my imaginary shopping list to Sainsbury’s will ultimately cost me £52.85. Taking it to Tesco will cost me £54.13. Pretty close, right, and I’m not likely to care about the difference because Tesco would give me a £1.28 voucher off my next shop which makes up for the difference (note that Sainsbury’s wouldn’t reciprocate in kind if it were the other way around, after a policy change they made late last year). But that’s not actually a true representation of the value of ‘shopping around’. As my spreadsheet shows, if I were to buy each item on my list at the supermarket that was cheapest, it’d only cost me a total of £43.75: that’s a saving of £9.10 (or about 17% off my entire shop) compared to the cheapest of these supermarkets. These schemes don’t give you a real “best of all worlds”. Instead, they give you, at most, a “best of all worlds, assuming that you’re still going to be lazy enough to only shop in one place”.

Ruth and JTA shopping in advance of Murder At The Magic College.
When you’re buying this much shopping, you’re unlikely to want to go to two different supermarkets to do it, however much money it might save you.

If you’re particularly devious of mind, you can exploit this. For example, suppose I went to Tesco but when I reached the checkout I split my shopping into two transactions. The first transaction contains the frozen goods, milk, wine, dough balls, flapjacks, and mini rolls. This comes out at £33.73, which is £10.38 more than Sainsbury’s would charge me for the same goods. Tesco therefore gives me a £10 voucher, which I immediately use on the second batch of shopping: the one which contains goods that are cheaper than their Sainsbury’s equivalents. The total price of my shopping: £44.13 – only 38p more than if I’d gone to both supermarkets and bought only the best-value goods from each (the 38p discrepancy comes from the fact that Tesco won’t ever give you a voucher worth more than £10, no matter how much you’re losing out).

Photo by 'alisdair' on Flickr. Used under a Creative Commons license.
“I’d like to run these through as two transactions, please.”

It’s not even that hard to do. Obviously, somebody’s probably written an app for it, but even if you’re just doing it by guesswork you can get a better result than just piling all of your shopping onto the conveyor belt together. Simply put the things which seem like a good deal (all of the discounted products, plus anything that feels like it’s good value) at one end of your trolley, and unload those things last. Making sure that you’ve got at least ten items on the conveyor, ‘split’ your shopping somewhere towards the beginning of these items. Then take any voucher you get from your first load, and apply it to the second.

It’s pretty easy, so long as you don’t mind looking like a bit of a tool at the checkout.

A Sainsbury's Brand Match voucher advising that my shopping was 1p cheaper than the competition. In total. Photo with thanks to Brett Jordan, used under a Creative Commons license.
Well that makes it all worthwhile then, doesn’t it?

But to most people, most of the time, this is nothing more than a strong and compelling piece of marketing. Either you get reminded that you allegedly “saved money”, on a piece of paper that probably goes into your wallet and helps to combat buyer’s remorse, or else we get told that we paid a particular amount more than we needed to, and are offered the difference back so long as we return to the same store within the next fortnight. Either way, the supermarket wins your loyalty, which – for a couple of pence on each transaction (assuming that the customer doesn’t lose the voucher or otherwise fail to get an opportunity to use it) – is a miniscule price to pay.

]]> https://danq.me/2015/06/09/supermarket-brand-match/feed/ 2 Solar Power, part 2 https://danq.me/2015/05/12/solar-power-2/ https://danq.me/2015/05/12/solar-power-2/#comments Tue, 12 May 2015 12:55:16 +0000 https://danq.me/?p=6892 At the very end of last year, right before the subsidy rate dropped in January, I had solar panels installed: you may remember that I blogged about it at the time. I thought you might be interested to know how that’s working out for us.

Solar panels on our roof.
A power plant, right on top of our house. It’s very small – like, a “13” on Power Grid – but it’s ours.

Because I’m a data nerd, I decided to monitor our energy usage, production, and total cost in order to fully understand the economic impact of our tiny power station. I appreciate that many of you might not be able to appreciate how cool this kind of data is, but that’s because you don’t have as good an appreciation of how fun statistics can be… it is cool, damn it!

This chart, for example, shows our energy usage in KWh of each of gas and electricity for the last 8 months.
This stacked area chart, for example, shows our energy usage in KWh of each of gas and electricity for the last 8 months.

If you look at the chart above, for example (click for a bigger version), you’ll notice a few things:

Solar panels pay for themselves by (1) powering your appliances, thus meaning you buy less electricity from the grid, (2) selling electricity that is generated but not used back to the grid, and (3) through a subsidy scheme that rewards the generation of green electricity.
Solar panels (slowly) pay for themselves in three different ways. People often find it surprising that there aren’t only one or two.

What got me sold on the idea of installing solar panels, though, was their long-term investment potential. I had the money sitting around anyway, and by my calculations we’ll get a significantly better return-on-investment out of our little roof-mounted power station than I would out of a high-interest savings account or bond. And that’s because of the oft-forgotten “third way” in which solar panelling pays for itself. Allow me to explain:

  1. Powering appliances: the first and most-obvious way in which solar power makes economic sense is that it powers your appliances. Right now, we generate almost as much electricity as we use (although because we use significantly more power in the evenings, only about a third of what which we generate goes directly into making our plethora of computers hum away).
  2. Selling back to the grid (export tariff): as you’re probably aware, it’s possible for a household solar array to feed power back into the National Grid: so the daylight that we’re collecting at times when we don’t need the electricity is being sold back to our energy company (who in turn is selling it, most-likely, to our neighbours). Because they’re of an inclination to make a profit, though (and more-importantly, because we can’t commit to making electricity for them when they need it: only during the day, and dependent upon sunlight), they only buy units from us at about a third of the rate that they sell them to consumers. As a result, it’s worth our while trying to use the power we generate (e.g. to charge batteries and to run things that can be run “at any point” during the day like the dishwasher, etc.) rather than to sell it only to have to buy it back.
  3. From a government subsidy (feed-in tariff): here’s the pleasant surprise – as part of government efforts to increase the proportion of the country’s energy that is produced from renewable sources, they subsidise renewable microgeneration. So if you install a wind turbine in your garden or a solar array on your roof, you’ll get a kickback for each unit of electricity that you generate. And that’s true whether you use it to power appliances or sell it back to the grid – in the latter case, you’re basically being paid twice for it! The rate that you get paid as a subsidy gets locked-in for ~20 years after you build your array, but it’s gradually decreasing. We’re getting paid a little over 14.5p per unit of electricity generated, per day.
A graph showing the number of units per day we've generated, peaking during that  sunny spell in late April.
Late April was bright and sunny and we were able to generate up to 19 units per day (for contrast, we use around 12 units per day), but May has so-far been rainy and grey and we’ve made only about 13 units per day.

As the seasons have changed from Winter through Spring we’ve steadily seen our generation levels climbing. On a typical day, we now make more electricity than we use. We’re still having to buy power from the grid, of course, because we use more electricity in the evening than we’re able to generate when the sun is low in the sky: however, if (one day) technology like Tesla’s PowerWall becomes widely-available at reasonable prices, there’s no reason that a house like ours couldn’t be totally independent of the grid for 6-8 months of the year.

Two SSE engineers head back to their van.
These guys came and replaced our electricity meter, because it was… umm… running backwards.

So: what are we saving/making? Well, looking at the last week of April and the first week of May, and comparing them to the same period last year:

  1. Powering appliances: we’re saving about 60p per day on electricity costs (down to about £1.30 per day).
  2. Selling back to the grid: we’re earning about 50p per day in exports.
  3. From a government subsidy: we’re earning about £2.37 per day in subsidies.

As I’m sure you can see: this isn’t peanuts. When you include the subsidy then it’s possible to consider our energy as being functionally “free”, even after you compensate for the shorter days of the winter. Of course, there’s a significant up-front cost in installing solar panels! It’s hard to say exactly when, at this point, I expect them to have paid for themselves (from which point I’ll be able to use the expected life of the equipment to more-accurately predict the total return-on-investment): I’m planning to monitor the situation for at least a year, to cover the variance of the seasons, but I will of course report back when I have more data.

Electricity meter with red light showing.
Our new electricity meter, which replaced the old one – one of those with a “wheel”. The red light indicates that fraud has been detected. Yeah, about that…

I mentioned that the first graph wasn’t accurate? Yeah: so it turns out that our house’s original electricity meter was of an older design that would run backwards when electricity was being exported to the grid. Which was great to see, but not something that our electricity company approved of, on account of the fact that they were then paying us for the electricity we sold back to the grid, twice: for a couple of days of April sunshine, our electricity meter consistently ran backwards throughout the day. So they sent a couple of engineers out to replace it with a more-modern one, pictured above (which has a different problem: its “fraud light” comes on whenever we’re sending power back to the grid, but apparently that’s “to be expected”).

In any case, this quirk of our old meter has made some of my numbers from earlier this year more-optimistic than they might otherwise be, and while I’ve tried to compensate for this it’s hard to be certain that my estimates prior to its replacement are accurate. So it’s probably going to take me a little longer than I’d planned to have an accurate baseline of exactly how much money solar is making for us.

But making money, it certainly is.

]]> https://danq.me/2015/05/12/solar-power-2/feed/ 5