How cool are Silo’s credits?

I’ve long been a fan of Hugh Howey‘s Wool series of books (especially the first and third; the second’s a bit weaker); in fact I’ve been enjoying re-reading them as a bedtime story for our eldest!1

Naturally, when I heard that it would become a TV series I was really excited! I’m enjoying the series so far, especially thanks to its epic casting. It diverges a lot from the books – sometimes in ways I love, sometimes in ways that confuse me – but that’s not what I wanted to talk about. I wanted to share how cool the opening credits sequence is!

Spoiler warning: even if you’re following the TV series there are likely to be major spoilers below based on my recollection of the books!

Sun shining through yellow clouds.

We open on the sun shining above a thick layer of all-obscuring clouds, tinted sickly yellow like poison gas, then descend into the darkness below. This hints at the uninhabitability of the world above, foreshadows Lukas stargazing through gaps in the clouds2, and foreshadows revelations about the argon gas used to flush the airlocks. The descent feels representative of humanity’s migration from the sunlit surface to the underground silos.

A star-shaped structure seen from above, through thick clouds.

Looking down, we see the silo from above in a desolate landscape, introducing the world and its setting. The area around it is shrouded and hostile, reflecting the residents’ view of the outside world as unsurvivable, but also masking our view of the other nearby silos that we might otherwise be able to see.

Dark circle on a yellow background with random radial spokes.

Descending “into” this representation of the silo, we get a view for only a split second that looks distinctly like the platter and spindle of a magnetic hard disk drive, broken-up as if to represent corruption. This reflects a number of major plot points in the first season relating to the destruction and recovery of secret information from ancient storage devices.

Sepia view "down" into a deep silo, with landings and skywalks visible.

Truly within the silo now, we see the spokes of landings radiating out from the great stairwell. The shape is reminiscient of a cog: a motif we’ll return to later. Humanoid shapes made of light, like you get in a long exposure, move around, giving both the idea of a surveillance state, and setting us up to think of all such “glowing spots” as people (relevant later in the credits).

Spiral, shaking, with illuminated gaseos shapes whipping around it.

A representation of the stairwell itself appears, with a lit gaseous substance whipping up and down it. Given that we’ve just been shown that this kind of “light” represents people, it’s easy to see this as showing us the traffic that grinds up and down the silo, but it also feels like looking at part of a great machine, pumping gas through a condenser: notice that there’s no landings any more: this is all about the never-ending traffic.

Skewed camera angle showing a dark bridge with ghostly gaseous figures crossing.

A landing appears, and the gaseous forms are now more-clearly humanoid, almost as if they’re ghosts (perhaps pointing to the number of generations who’ve lived before, in this place, or else a reference to Juliette’s investigation into the lives of those who lived before her).

A swirling mass of luminescent gas around a central pillar, with a fenced balcony visible.

More swirling gas-people, this time below an empty balconette: perhaps a nod to the source of Juliette’s uncommon name (in the books, it’s taken from Romeo & Juliet, a possibly-illicit copy of which is retained by the silo and performed prior to Juliette’s birth and for at least a short while afterwards: she writes mechanical notes on the back of a playscript), or perhaps a reference to George’s death after “falling” from a balcony.

Blue-white gas wirls around a pillar.

Seen from a different angle, the colour shifts, and the gas/ghosts become white like the argon spray of the airlock. The people are all part of a machine: a machine that sends people outside to clean and die. But more than that, the blue comes to represent a clean/perfect view of what a silo can be: a blueprint representation of the goals of its creators to shape the inhabitants into their vision of the future:

Partial X-ray of a human spine?

We refocus on the shape of the silo itself, but just for a split second the view looks more like an x-ray… of a human spine? As if to remind us that it’s people who upload the system of the silo, just as its concrtete uploads its physical structure. Also a reminder that the silo is treated (by those who manage it, both within and beyond it) as an organic thing that can be nurtured, grown, or if necessary killed.

Bridges seen semi-transparently.

This becomes the structure of the silo, but it almost looks architectural: a “clean” look, devoid of people or signs of life, like a blueprint, perhaps foreshadowing Donald’s role in designing the structures that will eventually become the silos. The “space” between the arms is emphasised, showing how the social system that this structure imposes serves to separate and segregate people: classism is a recurring theme in both the books and the TV series, and it eventually becomes apparent that the silos are specifically organised to reduce communication between interdependent groups.

Swirling spiral of gaseous light with a nautillus shell shape at its centre.

Returning to the “populated” silo – swirls of gas spiralling away down (or up: it’s no longer clear!), we catch a glimpse of a nautilus shell at the centre. The nautilus is a “living fossil”, a creature from a bygone era that continues to survive in our modern world, which is an excellent metaphor for the population of the dead world who go on living beneath its surface. The nautilus shell is a recurring image within the TV series: Gloria’s visions of the world that came before see her clutching one and tracing its shape, for example.

A seed in a yellow cloud.

We cut to what appears to be a seed, representing both the eventual conclusion of the story (Juliette, Charlotte and the Silo 18 survivors’ discovery of the cache of supplies that will allow them to begin rebuilding the world) and also the nature of the silo3. The seed we see initially appears to fail and degrade, becoming nothing at all, before eventually growing into the beginnings of a strong new plant. This could represent the eventual and inevitable collapse of silo 18, among others, but the eventual flourishing of those that survive, or on a broader scale the collapse of modern civilization to be replaced by the silos, or even of the silo system to be replaced with that which follows it after the conclusion of the story. Lots of options!

A seedling grows in harsh conditions.

It’s also possibly a reflection of the harsh and opaque eugenics/population control mechanism imposed by the “lottery”, which becomes a major plot point in the TV series much earlier than in the books.

Trees in a yellow fog.

We cut to trees, thriving despite a yellow fog. The sky can’t be seen, which is a reminder that all of humanity’s resources must now be produced underground (trees are especially rare and prized, leading to a shortage of paper4. It seems to be deliberately left unclear whether the trees we see are on the surface before the fall of humanity, on the surface after the fall, or grown underground.

Fruit falling from a tree.

A fruit falls from the tree, which links back to the seed we saw geminate earlier but also seems likely to be a representation of the concept of original sin. The grand idea of the silos was to create a better world on the other side of a man-made catastrophe, but this idea is inherently flawed because the systems that are constructed by the same people who are complicit in the destruction of the world that came before. The structure that’s put in place through the Pact carries the weight of the sins of its creators: even though the inhabitants of silo 1 ultimately intend to destroy themselves, they’re unable to create a new world that is both better than the one that came before and free from their influence: it’s an impossibility.

It’s also possibly a representation of the religious beliefs of some inhabitants that the creators of the silo should be revered as gods. This was a recurring plot point in the books but has been somewhat muted in the TV series so far.

Rotting fruit falls.

The metaphor continues when we see that this falling fruit is already beginning to rot, degrading as it tumbles towards the earth. We don’t see it strike the ground: it almost seems to hover in the air, uncertain and undecided, and reflective of the eventual end when the inhabitants of the silos break free from the shackles of the system that’s been constructed for them and can choose their own destiny. Or perhaps we don’t see the collision simply because the camera continues to fall down into the earth and below the surface again?

Heavy industrial machinery, smoke, and fumes.

This time, wer’e very deep: all the way down in the depths of Mechanical, at the bottom of the silo: home to our heroine and source of many aspects of the story. In the centre, a shaft descends, connecting us back to the “spine” of the silo – the great staircase – but it’s harder to see as a wealth of machinery appears to support it, occluding our view. From down here in Mechanical it appears that the machines keep the silo running, whereas further up it looked like humans pumped through it like blood, which reflects Juliette’s disagreements with many of those up-top about their priorities during her time as Sheriff and, later, as Mayor.

A cloud of steam, some of it with humanoid shapes.

We see a cloud of steam, like that used to drive the generator that brings life to the silo, and for a moment it’s impossible to differentiate it from the cloud of people we saw earlier, rushing up and down the stairs. Look closely at the steam, though, and you’ll see that it too contains the ghosts of people.

Many cogs whirl in a firey light.

Deeper still, the cog motif returns and we’re buried in an impossible number of interconnected gears. The machine that they support is impossible to comprehend from within: How big is it? What is it for? Who made it and why?

A cog crossfades into a top-down view of a spiral staircase.

The final cog mutates into the staircase again, winding away from us and hammering the point home.

A spiral staircase becomes encased in outlines of blue-white light.

The staircase changes again, first becoming an outline of itself (a callback to the “blueprint” design we saw earlier, reminding us that this thing was designed to be like this)…

A double-helix shape cast in light, being made from the alternating-steps of a staircase.

…but this becomes a double-helix, representing the chaos of life. Again, the metaphor is of a perfect idea constructed to achieve a goal, but the unpredictability of humans leads to a different outcome.

The ends of spiraling lights appear to spin around.

Seen from above, the staircase now looks like an enormous clock, a machine of cogs each turning slower than the one beneath, counting down until the end of the silo experiment in accordance with the whim of its creators. Except, of course, if something were to break this machine.

Cross-section of different-sized circular floors with ghostly people moving around.

Seen from the side, the silo is a hive of activity, but the shape the levels form in this depiction are exactly like the rotors of a steam turbine, and this is reflected by an image of steam, almost in the shape of a growing tree – passing behind it in the background. The generator and its rotor blades is a significant early plot point in both the books and the TV series, and the books in particular use engine metaphors to explain Juliette’s interpretation of different situations she finds herself in, even those which are distinctly interpersonal rather than mechanical.

Cog-like shape.

Looking back up the silo, towards the light, we can now see its shape and structure for what it is: just another cog – a part of an even bigger machine that is the whole Operation Fifty silo network. The people are the lifeblood of this machine, but they’re as replaceable and interchangable as any other part.

The word "SILO" in a stab-serif font, black on a yellow background.

Finally, we crossfade to the title, looking like a stencil. Each letter is more-degraded than the one before it, representing the impossibility of building a perfect system.

The credits sequence is less than 90 seconds long, but so much is packed into it. It’s just great.


1 We’re into the final act of Dust now and it’s been amazing to experience the characters – loveable and hateable – of the series.

2 Curiously, in the TV series Lukas is only ever seen stargazing on clear nights, which is one of those confusing choices I mentioned. I suspect it’s for aesthetic reasons and to help add some romance to Juliette and Lukas’s courtship.

3 A silo is, of course, a place to store something valuable through the hard times. This is exactly what the silos in this story are for.

4 The shortage of paper shows up many times in the books but is somewhat glossed-over in the TV series. I’m not sure how they’ll reconcile that with the impact of the discovery of the Legacy, later.

Sun shining through yellow clouds.× A star-shaped structure seen from above, through thick clouds.× Dark circle on a yellow background with random radial spokes.× Sepia view "down" into a deep silo, with landings and skywalks visible.× Spiral, shaking, with illuminated gaseos shapes whipping around it.× Skewed camera angle showing a dark bridge with ghostly gaseous figures crossing.× A swirling mass of luminescent gas around a central pillar, with a fenced balcony visible.× Blue-white gas wirls around a pillar.× Partial X-ray of a human spine?× Bridges seen semi-transparently.× Swirling spiral of gaseous light with a nautillus shell shape at its centre.× A seed in a yellow cloud.× A seedling grows in harsh conditions.× Trees in a yellow fog.× Fruit falling from a tree.× Rotting fruit falls.× Heavy industrial machinery, smoke, and fumes.× A cloud of steam, some of it with humanoid shapes.× Many cogs whirl in a firey light.× A cog crossfades into a top-down view of a spiral staircase.× A spiral staircase becomes encased in outlines of blue-white light.× A double-helix shape cast in light, being made from the alternating-steps of a staircase.× The ends of spiraling lights appear to spin around.× Cross-section of different-sized circular floors with ghostly people moving around.× Cog-like shape.× The word "SILO" in a stab-serif font, black on a yellow background.×

Note #21196

I bought Zach Weinersmith‘s Bea Wolf for my kids (9 and 6, the elder of them already a fan of Beowulf). It arrived today, but neither of them have had a chance to because I wouldn’t put it down.

My favourite bit is when Bea and her entourage arrive near Treeheart and the shield-bearer who greets them says “Your leader sparkles with power and also with sparkles.” The line’s brilliant, clever, and accompanies the most badass illustration.

Inked pencil sketch showing a resolute-faced young girl - Bea Wolf - in a teddy-bear hat, flowerprint dress, and star-spangled boots, standing cross-armed amongst four of her friends. Above, a line of text ends a quote, reading "Your leader sparkles with power and also with sparkles."
I’ll give it to my kids… eventually. But if you’re looking for a book recommendation in the meantime, this is it.

Inked pencil sketch showing a resolute-faced young girl - Bea Wolf - in a teddy-bear hat, flowerprint dress, and star-spangled boots, standing cross-armed amongst four of her friends. Above, a line of text ends a quote, reading "Your leader sparkles with power and also with sparkles."×

Spy’s Guidebook Reborn

When I was a kid of about 10, one of my favourite books was Usborne’s Spy’s Guidebook. (I also liked its sister the Detective’s Handbook, but the Spy’s Guidebook always seemed a smidge cooler to me).

Detective's Handbook andSpy's Guidebook on a child's bookshelf.
I imagine that a younger version of me would approve of our 7-year-old’s bookshelf, too.

So I was pleased when our eldest, now 7, took an interest in the book too. This morning, for example, she came to breakfast with an encrypted message for me (along with the relevant page in the book that contained the cipher I’d need to decode it).

Usborne Spy's Guidebook showing the "Pocket code card" page and a coded message
Decryption efforts were hampered by sender’s inability to get her letter “Z”s the right damn way around.

Later, as we used the experience to talk about some of the easier practical attacks against this simple substitution cipher (letter frequency analysis, and known-plaintext attacks… I haven’t gotten on to the issue of its miniscule keyspace yet!), she asked me to make a pocket version of the code card as described in the book.

Three printed pocket code cards
A three-bit key doesn’t make a simple substitution cipher significantly safer, but it does serve as a vehicle to teach elementary cryptanalysis!

While I was eating leftover curry for lunch with one hand and producing a nice printable, foldable pocket card for her (which you can download here if you like) with the other, I realised something. There are likely to be a lot more messages in my future that are protected by this substitution cipher, so I might as well preempt them by implementing a computerised encoder/decoder right away.

So naturally, I did. It’s at and all the source code is available to do with as you please.

Key 4-1 being used to decode the message: UOMF0 7PU9V MMFKG EH8GE 59MLL GFG00 8A90P 5EMFL
Uh-oh: my cover is blown!

If you’ve got kids of the right kind of age, I highly recommend picking up a copy of the Spy’s Guidebook (and possibly the Detective’s Handbook). Either use it as a vehicle to talk about codes and maths, like I have… or let them believe it’s secure while you know you can break it, like we did with Enigma machines after WWII. Either way, they eventually learn a valuable lesson about cryptography.

Detective's Handbook andSpy's Guidebook on a child's bookshelf.× Usborne Spy's Guidebook showing the "Pocket code card" page and a coded message× Three printed pocket code cards×

Twinebook – Printable Interactive Fiction

Twine 2 is a popular tool for making hypertext interactive fiction, but there’s something about physical printed “choose your own adventure”-style gamebooks that isn’t quite replicated when you’re playing on the Web. Maybe it’s the experience of keeping your finger in a page break to facilitate a “save point” for when you inevitably have to backtrack and try again?

Annabe enjoying Choose Your Own (Minecraft) Story books.
These are the first branching novels I’ve introduced her to for which she’s felt the need to take notes.

As a medium for interactive adventures, paper isn’t dead! Our 7-year-old is currently tackling the second part of a series of books by John Diary, the latest part of which was only published in December! But I worry that authors of printed interactive fiction might have a harder time than those producing hypertext versions. Keeping track of all of your cross-references and routes is harder than writing linear fiction, and in the hypertext

Hand-written note showing branching path story plan, from John Diary's Twitter.
John Diary tweeted about his process back in 2017 and it looks… more manual than I’d want.


So I’ve thrown together Twinebook, an experimental/prototype tool which aims to bring the feature-rich toolset of Twine to authors of paper-based interactive fiction. Simply: you upload your compiled Twine HTML to Twinebook and it gives you a printable PDF file, replacing the hyperlinks with references in the style of “turn to 27” to instruct the player where to go next. By default, the passages are all scrambled to keep it interesting, but with the starting passage in position 1… but it’s possible to override this for specific passages to facilitate puzzles that require flipping to specific numbered passages.

Thumb in the page of a Sorcery choose-your-own-adventure gamebook.
In some adventure games, keeping your thumb in the page feels like it’s essential.

Obviously, it doesn’t work with any kind of “advanced” Twine game – anything that makes use of variables, Javascript, etc., for example! – unless you can think of a way to translate these into the written word… which is certainly possible – see Fighting Fantasy‘s skill, stamina, luck and dice-rolling mechanics, for example! – but whether it’s desirable is up to individual authors.

If this tool is valuable to anybody, that’s great! Naturally I’ve open-sourced the whole thing so others can expand on it if they like. If you find it useful, let me know.

Twine screenshot showing many branching paths of the game "Inpatient".
Mapping a complex piece of interactive fiction is a job for a computer, not a human.

If you’re interested in the possibility of using Twine to streamline the production of printable interactive fiction, give my Twinebook prototype a try and let me know what you think.

Annabe enjoying Choose Your Own (Minecraft) Story books.× Thumb in the page of a Sorcery choose-your-own-adventure gamebook.× Twine screenshot showing many branching paths of the game "Inpatient".×

Ted Chiang Explains the Disaster Novel We All Suddenly Live In

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

While there has been plenty of fiction written about pandemics, I think the biggest difference between those scenarios and our reality is how poorly our government has handled it. If your goal is to dramatize the threat posed by an unknown virus, there’s no advantage in depicting the officials responding as incompetent, because that minimizes the threat; it leads the reader to conclude that the virus wouldn’t be dangerous if competent people were on the job. A pandemic story like that would be similar to what’s known as an “idiot plot,” a plot that would be resolved very quickly if your protagonist weren’t an idiot. What we’re living through is only partly a disaster novel; it’s also—and perhaps mostly—a grotesque political satire.

What will “normal” look like after the coronavirus crisis has passed? Will it be the same normal as we’re used to? Or could we actually learn some lessons from this and progress towards something better?

I love Ted Chiang’s writing; enough to reshare this interview even though I’m only lukewarm about it!

Why do book spines have the titles printed the way they do?

Have you noticed how the titles printed on the spines of your books are all, for the most part, oriented the same way? That’s not a coincidence.

Illustration of a chained library
If you can’t see the spines of your books at all, perhaps you’re in a library and it’s the 17th century. Wait: how are you on the Internet?

ISO 6357 defines the standard positioning of titles on the spines of printed books (it’s also codified as British Standard BS6738). If you assume that your book is stood “upright”, the question is one of which way you tilt your head to read the title printed on the spine. If you tilt your head to the right, that’s a descending title (as you read left-to-right, your gaze moves down, towards the surface on which the book stands). If you tilt your head to the left, that’s an ascending title. If you don’t need to tilt your head in either direction, that’s a transverse title.

ISO 6357:1985 page illustrating different standard spine title alignments.
Not every page in ISO 6357:1985 is as exciting as this one.

The standard goes on to dictate that descending titles are to be favoured: this places the title in a readable orientation when the book lays flat on a surface with the cover face-up. Grab the nearest book to you right now and you’ll probably find that it has a descending title.

Books on a shelf.
This eclectic shelf includes a transverse title (the Holy Bible), a semi-transverse title (The Book of English Magic) and a rare ascending title (A First Dictionary) among a multitude of descending titles.

But if the book is lying on a surface, I can usually read the cover of the book. Only if a book is in a stack am I unable to do so, and stacks are usually relatively short and so it’s easy enough to lift one or more books from the top to see what lies beneath. What really matters when considering the orientation of a spine title is, in my mind, how it appears when it’s shelved.

It feels to me like this standard’s got things backwards. If a shelf of anglophone books is organised into any kind of order (e.g. alphabetically) then it’ll usually be from left to right. If I’m reading the titles from left to right, and the spines are printed descending, then – from the perspective of my eyes – I’m reading from bottom to top: i.e. backwards!

It’s possible that this is one of those things that I overthink.

Illustration of a chained library× Books on a shelf.×

How Do You Move A Bookstore? With A Human Chain, Book By Book

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Human chain moving books

When October Books, a small radical bookshop in Southampton, England, was moving to a new location down the street, it faced a problem. How could it move its entire stock to the new spot, without spending a lot of money or closing down for long?

The shop came up with a clever solution: They put out a call for volunteers to act as a human conveyor belt.

Delightful application of volunteer effort.

Note #11557

Today, @bodleianlibs releases Shadows Out of Time, a Choose-Your-Own-Destiny story. It’s amazing – go read it: #halloween #InteractiveFiction

Oat the Goat

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Oat the Goat (

Oh my Goat! We just finished reading this awesome pick-a-path story that helps children learn the power of kindness. Have a go… #OatTheGoat

Oat the Goat

Discovered this fun interactive storybook; it tells the tale of a goat called Oat who endeavours to climb a mountain (making friends along the way). At a few points, it presents as a “choose your own adventure”-style book (although the forks are artificial and making the “wrong” choice immediately returns you the previous page), but it still does a reasonable job at looking at issues of bullying and diversity.

The Better Bundo Book

Today, I received my long-awaited copy of A Day in the Life of Marlon Bundo, a book inspired by the US Vice President’s family pet not to be confused with Marlin Bundo’s A Day in the Life of the Vice President, which it satirises. In case you’ve been living under a rock: the family of US Vice President Mike Pence have a pet rabbit called Marlon Bundo (and who doesn’t appreciate some punmanship in their pet’s name) and they wrote the latter book that attempts to explain, through the eyes of Marlon Bundo, what the Vice President does. And then John Oliver, who’s become a bit of a master of doing nice things in a dickish way, released the former a few hours earlier and subsequently thoroughly outsold the Pence book.

A Day in the Life of Marlon Bundo
I wasn’t fast enough to get an order in on the first (hugely-oversubscribed) print run and had to wait on both the reprint plus international shipping.

This self-proclaimed “better Bundo book” tells a different (educational and relevant) story: in it, Marlon Bundo falls in love with another boy rabbit but their desire to get married is hampered by the animals’ leader, the Stink Bug, who proclaims that “boy rabbits can’t marry boy rabbits; boy rabbits have to marry girl rabbits!” With the help of the other animals, the rabbits vote-out the Stink Bug, get married, and go on a lovely bunnymoon… a cheery and uplifting story and, of course, a distinctly trollish way to piss off the (clearly anti-LGBT) Mike Pence. This evening, I decided to offer it as a bedtime story to our little bookwork. At four years old, she’s of an age at which the highly-hetronormative narratives of the media to which she’s exposed might be only-just beginning to sink in, so I figured this was a perfect vehicle to talk about difference, diversity, and discrimination. Starting school later this year means that she’s getting closer to the point where she may go from realising that her family is somewhat unusually-shaped to discovering that some people might think that “unusual” means “wrong”, so this is also a possible step towards thinking about her own place in the world and what other people make of it.

Ruth reading with Annabel and John.
Our little bookworm, along with bookworm-junior and their mother.

Her initial verdict was that it was “sweet”, and that she was glad that the Stink Bug was vanquished and that Marlon and Wesley got to live together happily-ever-after. I explained that while the story was made-up, a lot of what it was talking about was something that really happens in this world: that some people think that boys should not marry boys and that girls should not marry girls, even if they love them, and that sometimes, if those people get to be In Charge then they can stop those people marrying who they love. I mentioned that in our country we were fortunate enough that boys can marry boys and girls can marry girls, if they want to, but that there are places where that’s not allowed (and there are even some people who think it shouldn’t be allowed here!). And then I asked her what she thought.

“They’re like the stinky Stink Bug.”

That’ll do.

A Day in the Life of Marlon Bundo× Ruth reading with Annabel and John.×

21 Books You Don’t Have to Read (and 21 you should read instead)

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

The 21 Most Overrated Books Ever (and 21 Books to Read Instead) (GQ)

GQ asked its favorite new authors to dunk on the classics.

We’ve been told all our lives that we can only call ourselves well-read once we’ve read the Great Books. We tried. We got halfway through Infinite Jest and halfway through the SparkNotes on Finnegans Wake. But a few pages into Bleak House, we realized that not all the Great Books have aged well. Some are racist and some are sexist, but most are just really, really boring. So we—and a group of un-boring writers—give you permission to strike these books from the canon. Here’s what you should read instead.

Personally, I quite enjoyed at least two of the books on the “books you don’t have to read” list… but this list has inspired me to look into some of the 21 “you should read instead”.

EEBO-TCP Hackathon

Last month I got the opportunity to attend the EEBO-TCP Hackfest, hosted in the (then still very-much under construction) Weston Library at my workplace. I’ve done a couple of hackathons and similar get-togethers before, but this one was somewhat different in that it was unmistakably geared towards a different kind of geek than the technology-minded folks that I usually see at these things. People like me, with a computer science background, were remarkably in the minority.

Dan Q in Blackwell Hall, at the Weston Library.
Me in the Weston Library (still under construction, as evidenced by the scaffolding in the background).

Instead, this particular hack event attracted a great number of folks from the humanities end of the spectrum. Which is understandable, given its theme: the Early English Books Online Text Creation Partnership (EEBO-TCP) is an effort to digitise and make available in marked-up, machine-readable text formats a huge corpus of English-language books printed between 1475 and 1700. So: a little over three centuries of work including both household names (like Shakespeare, Galileo, Chaucer, Newton, Locke, and Hobbes) and an enormous number of others that you’ll never have heard of.

Dan Q talks to academic Stephen Gregg
After an introduction to the concept and the material, attendees engaged in a speed-networking event to share their thoughts prior to pitching their ideas.

The hackday event was scheduled to coincide with and celebrate the release of the first 25,000 texts into the public domain, and attendees were challenged to come up with ways to use the newly-available data in any way they liked. As is common with any kind of hackathon, many of the attendees had come with their own ideas half-baked already, but as for me: I had no idea what I’d end up doing! I’m not particularly familiar with the books of the 15th through 17th centuries and I’d never looked at the way in which the digitised texts had been encoded. In short: I knew nothing.

Dan Q and Liz McCarthy listen as other attendees pitch their ideas.
The ideas pitch session quickly showed some overlap between different project ideas, and teams were split and reformed a few times as people found the best places for themselves.

Instead, I’d thought: there’ll be people here who need a geek. A major part of a lot of the freelance work I end up doing (and a lesser part of my work at the Bodleian, from time to time) involves manipulating and mining data from disparate sources, and it seemed to me that these kinds of skills would be useful for a variety of different conceivable projects.

Dan Q explains what the spreadsheet he's produced 'means'.
XML may have been our interchange format, but everything fell into Excel in the end for speedy management even by less-technical team members.

I paired up with a chap called Stephen Gregg, a lecturer in 18th century literature from Bath Spa University. His idea was to use this newly-open data to explore the frequency (and the change in frequency over the centuries) of particular structural features in early printed fiction: features like chapters, illustrations, dedications, notes to the reader, encomia, and so on). This proved to be a perfect task for us to pair-up on, because he had the domain knowledge to ask meaningful questions, and I had the the technical knowledge to write software that could extract the answers from the data. We shared our table with another pair, who had technically-similar goals – looking at the change in the use of features like lists and tables (spoiler: lists were going out of fashion, tables were coming in, during the 17th century) in alchemical textbooks – and ultimately I was able to pass on the software tools I’d written to them to adapt for their purposes, too.

Dan Q with two academic-minded humanities folks, talking about their hackathon projects.
A quick meeting on the relative importance of ‘chapters’ as a concept in 16th century literature. Half of the words that the academics are saying go over my head, but I’m formulating XPath queries in my head while I wait.

And here’s where I made a discovery: the folks I was working with (and presumably academics of the humanities in general) have no idea quite how powerful data mining tools could be in giving them new opportunities for research and analysis. Within two hours we were getting real results from our queries and were making amendments and refinements in our questions and trying again. Within a further two hours we’d exhausted our original questions and, while the others were writing-up their findings in an attractive way, I was beginning to look at how the structural differences between fiction and non-fiction might be usable as a training data set for an artificial intelligence that could learn to differentiate between the two, providing yet more value from the dataset. And all the while, my teammates – who’d been used to looking at a single book at a time – were amazed by the possibilities we’d uncovered for training computers to do simple tasks while reading thousands at once.

Laptop showing a map of the area around Old St. Paul's Cathedral.
The area around Old St. Paul’s Cathedral was the place to be if you were a 16th century hipster looking for a new book.

Elsewhere at the hackathon, one group was trying to simulate the view of the shelves of booksellers around the old St. Paul’s Cathedral, another looked at the change in the popularity of colour and fashion-related words over the period (especially challenging towards the beginning of the timeline, where spelling of colours was less-standardised than towards the end), and a third came up with ways to make old playscripts accessible to modern performers.

A graph showing the frequency of colour-related words in English-language books printed over three centuries.
Aside from an increase in the relative frequency of the use of colour words to describe yellow things, there’s not much to say about this graph.

At the end of the session we presented our findings – by which I mean, Stephen explained what they meant – and talked about the technology and its potential future impact – by which I mean, I said what we’d like to allow others to do with it, if they’re so-inclined. And I explained how I’d come to learn over the course of the day what the word encomium meant.

Dan Q presents findings in Excel.
Presenting our findings in amazing technicolour Excel.

My personal favourite contribution from the event was by Sarah Cole, who adapted the text of a story about a witch trial into a piece of interactive fiction, powered by Twine/Twee, and then allowed us as an audience to collectively “play” her game. I love the idea of making old artefacts more-accessible to modern audiences through new media, and this was a fun and innovative way to achieve this. You can even play her game online!

(by the way: for those of you who enjoy my IF recommendations: have a look at Detritus; it’s a delightful little experimental/experiential game)

Output from the interactive fiction version of a story about a witch trial.
Things are about to go very badly for Joan Buts.

But while that was clearly my favourite, the judges were far more impressed by the work of my teammate and I, as well as the team who’d adapted my software and used it to investigate different features of the corpus, and decided to divide the cash price between the four of us. Which was especially awesome, because I hadn’t even realised that there was a prize to be had, and I made the most of it at the Drinking About Museums event I attended later in the day.

Members of the other team, who adapted my software, were particularly excited to receive their award.
Cold hard cash! This’ll be useful at the bar, later!

If there’s a moral to take from all of this, it’s that you shouldn’t let your background limit your involvement in “hackathon”-like events. This event was geared towards literature, history, linguistics, and the study of the book… but clearly there was value in me – a computer geek, first and foremost – being there. Similarly, a hack event I attended last year, while clearly tech-focussed, wouldn’t have been as good as it was were it not for the diversity of the attendees, who included a good number of artists and entrepreneurs as well as the obligatory hackers.

Stephen and Dan give a congratulatory nod to one another.
“Nice work, Stephen.” “Nice work, Dan.”

But for me, I think the greatest lesson is that humanities researchers can benefit from thinking a little bit like computer scientists, once in a while. The code I wrote (which uses Ruby and Nokogiri) is freely available for use and adaptation, and while I’ve no idea whether or not it’ll ever be useful to anybody again, what it represents is the research benefits of inter-disciplinary collaboration. It pleases me to see things like the “Library Carpentry” (software for research, with a library slant) seeming to take off.

And yeah, I love a good hackathon.

Update 2015-04-22 11:59: with thanks to Sarah for pointing me in the right direction, you can play the witch trial game in your browser.

Dan Q in Blackwell Hall, at the Weston Library.× Dan Q talks to academic Stephen Gregg× Dan Q and Liz McCarthy listen as other attendees pitch their ideas.× Dan Q explains what the spreadsheet he's produced 'means'.× Dan Q with two academic-minded humanities folks, talking about their hackathon projects.× Laptop showing a map of the area around Old St. Paul's Cathedral.× A graph showing the frequency of colour-related words in English-language books printed over three centuries.× Dan Q presents findings in Excel.× Output from the interactive fiction version of a story about a witch trial.× Members of the other team, who adapted my software, were particularly excited to receive their award.× Stephen and Dan give a congratulatory nod to one another.×