Just shut up and let your devs concentrate, advises Stack Overflow CEO Joel Spolsky

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

If you want to attract and keep developers, don’t emphasize ping-pong tables, lounges, fire pits and chocolate fountains. Give them private offices or let them work from home, because uninterrupted time to concentrate is the most important and scarcest commodity…

What I Didn’t Understand as a Junior Programmer

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

I remember the first time I saw a codebase over a million lines. It was at my internship, a large 10+ year old system, in multiple languages, thousands of unit tests, organized into several projects and dll’s that would take the whole night to recompile. Some of the projects had complex build processes, requiring extensive scripts, and even our source control had custom hooks preventing us from committing code that didn’t follow our style guides. It looked like it would take me a week just to read through all the documentation. My lead programmer told me it usually took people a year to understand the project in depth, but my internship was only for 3 months.

Wow, I thought. Is this what it takes to be a real programmer?

Why GNU grep is fast

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

I am the original author of GNU grep.  I am also a FreeBSD user,
although I live on -stable (and older) and rarely pay attention
to -current.

However, while searching the -current mailing list for an unrelated
reason, I stumbled across some flamage regarding BSD grep vs GNU grep
performance.  You may have noticed that discussion too...

Fewer and Faster // John Nunemaker

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Sometimes I feel developers think that performance is a dark art. It is not. In my experience, well performing systems come down to this: fewer and faster. If you are doing something a lot, do it fewer times. If you are doing something that is slow, make it faster. It really is that simple. The more things you make your system do and the slower those things are, the worse your performance will be…

DRY: Do Repeat Yourself

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

I am increasingly of the opinion that the general software engineering adage “Don’t Repeat Yourself” does not always apply to web development. Also, I found that web development classes in CS academia are not very realistic. These two problems turn out to have the same root cause: a lack of appreciation of what browsers do…

DevCamp – have we really been doing this for 7 years?

An annual tradition at Three Rings is DevCamp, an event that borrows from the “hackathon” concept and expands it to a week-long code-producing factory for the volunteers of the Three Rings development team. Motivating volunteers is a very different game to motivating paid employees: you can’t offer to pay them more for working harder nor threaten to stop paying them if they don’t work hard enough, so it’s necessary to tap in to whatever it is that drives them to be a volunteer, and help them get more of that out of their volunteering.

Table full of computers at DevCamp 2011.
This photo, from DevCamp 2011, is probably the only instance where I’ve had fewer monitors out than another developer.

At least part of what appeals to all of our developers is a sense of achievement – of producing something that has practical value – as well as of learning new things, applying what they’ve learned, and having a degree of control over the parts of the project they contribute most-directly to. Incidentally, these are the same things that motivate paid developers, too, if a Google search for studies on the subject is to believed. It’s just that employers are rarely able to willing to offer all of those things (and even if they can, you can’t use them to pay your mortgage), so they have to put money on the table too. With my team at Three Rings, I don’t have money to give them, so I have to make up for it with a surplus of those things that developers actually want.

A developer hides inside a handmade camera obscura to watch the solar eclipse at DevCamp 2015.
At the 2015 DevCamp, developers used the solar eclipse as an excuse for an impromptu teambuilding activity: making a camera obscura out of stuff we had lying about.

It seems strange to me in hindsight that for the last seven years I’ve spent a week of my year taking leave from my day job in order to work longer, harder, and unpaid for a voluntary project… but that I haven’t yet blogged about it. Over the same timescale I’ve spent about twice as long at DevCamp than I have, for example, skiing, yet I’ve managed to knock out several blog posts on that subject. Part of that might be borne out of the secretive nature of Three Rings, especially in its early days (when involvement with Three Rings pretty-much fingered you as being a Nightline volunteer, which was frowned upon), but nowadays we’ve got a couple of dozen volunteers with backgrounds in a variety of organisations: and many of those of us that ever were Nightliner volunteers have long since graduated and moved-on to other volunteering work besides.

DevCamp and DocsCamp 2016 volunteers play Betrayal at the House on the Hill
Semi-cooperative horror-themed board games by candlelight are a motivator for everybody, right?

Part of the motivation – one of the perks of being a Three Rings developer – for me at least, is DevCamp itself. Because it’s an opportunity to drop all of my “day job” stuff for a week, go to some beatiful far-flung corner of the country, and (between early-morning geocaching/hiking expeditions and late night drinking tomfoolery) get to spend long days contributing to something awesome. And hanging out with like-minded people while I do so. I like I good hackathon of any variety, but I love me some Three Rings DevCamp!

Geocache GC4EE6C, with accompanying caterpillar and mushroom
The geocaches near DevCamp 2016 were particularly fabulous, though. Like this one – GC4EE6C – part of an Alice In Wonderland-themed series.

So yeah: DevCamp is awesome. It’s more than a little different than those days back in 2003 when I wrote all the code and Kit worked hard at distracting me with facts about the laws of Hawaii – for the majority of DevCamp 2016 we had half a dozen developers plus two documentation writers in attendance! – but it’s still fundamentally about the same thing: producing a piece of software that helps about 25,000 volunteers do amazing things and make the world a better place. We’ve collectively given tens, maybe hundreds of thousands of hours of time in developing and supporting it, but that in turn has helped to streamline the organisation of about 16 million person-hours of other volunteering.

So that’s nice.

Developers marvel at one another's code, etc.
An end-of-day “Show & Tell” session at DevCamp 2016.

Oh, and I was delighted that one of my contributions this DevCamp was that I’ve finally gotten around to expanding the functionality of the “gender” property so that there are now more than three options. That’s almost more-exciting than the geocaches. Almost.

Edit: added a missing word in the sentence about how much time our volunteers had given, making it both more-believable and more-impressive.

× × × × ×

Twee2 – Interactive Fiction Authoring for Geeks

There’s a wonderful tool for making web-based “choose your own adventure”-type games, called Twine. One of the best things about it is that it’s so accessible: if you wanted to, you could be underway writing your first ever story with it in about 5 minutes from now, without installing anything at all, and when it was done you could publish it on the web and it would just work.

Screenshot of a Twine 2 story map
A “story map” in Twine 2. Easy interactive fiction writing for normal people.

But the problem with Twine is that, in its latest and best versions, you’re trapped into using the Twine IDE. The Twine IDE is an easy-to-use, highly visual, ‘drag-and-drop’ interface for making interactive stories. Which is probably great if you’re into IDEs or if you don’t “know better”… but for those of us who prefer to do our writing in a nice clean, empty text editor like Sublime or TextMate or to script/automate our builds, it’s just frustrating to lose access to the tools we love. Plus, highly-visual IDEs make it notoriously hard to collaborate with other authors on the same work without simply passing it back and forwards between you: unless they’ve been built with this goal in mind, you generally can’t have two people working in the same file at the same time.

Sublime Text demonstrating multi-line-selection.
Now THIS is what code editing should look like.

Earlier versions of Twine had a command-line tool called Twee that perfectly filled this gap. But the shiny new versions don’t. That’s where I came in.

In that way that people who know me are probably used to by now, I was very-slightly unsatisfied with one aspect of an otherwise fantastic product and decided that the correct course of action was to reimplement it myself. So that’s how, a few weeks ago, I came to release Twee2.

Twee2 logo
Twee2’s logo integrates the ‘branching’ design of Twine adventures with the ‘double-colon’ syntax of Twee.

If you’re interested in writing your own “Choose Your Own Adventure”-type interactive fiction, whether for the world or just for friends, but you find user-friendly IDEs like Twine limiting (or you just prefer a good old-fashioned text editor), then give Twee2 a go. I’ve written a simple 2-minute tutorial to get you started, it works on Windows, MacOS, Linux, and just-about everything else, and it’s completely open-source if you’d like to expand or change it yourself.

(there are further discussions about the concept and my tool on Reddit here, here, here and here, and on the Twinery forums herehere and here)

Get Twee2

× ×

Non-Rails Frameworks in Ruby: Cuba, Sinatra, Padrino, Lotus

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

It’s common for a Ruby developer to describe themselves as a Rails developer. It’s also common for someone’s entire Ruby experience to be through Rails. Rails began in 2003 by David Heinemeier Hansson, quickly becoming the most popular web framework and also serving as an introduction to Ruby for many developers.

Rails is great. I use it every day for my job, it was my introduction to Ruby, and this is the most fun I’ve ever had as a developer. That being said, there are a number of great framework options out there that aren’t Rails.

This article intends to highlight the differences between Cuba, Sinatra, Padrino, Lotus, and how they compare to or differ from Rails. Let’s have a look at some Non-Rails Frameworks in Ruby.

Calculating Pi (when you’re ill)

So, I’ve not been well lately. And because a few days lying on my back with insufficient mental stimulation is a quick route to insanity for me, I’ve been trying to spend my most-conscious moment doing things that keep my brain ticking over. And that’s how I ended up calculating pi.

Dan, shortly before inpatient admission but already recovering from the worst parts of his hospital visit, last week.
When I say I’ve been unwell, that might be an understatement. But we’ll get to that another time.

Pi (or π) is, of course, the ratio of the circumference of a circle to its diameter, for every circle. You’ll probably have learned it in school as 3.14, 3.142, or 3.14159, unless you were one of those creepy kids who tried to memorise a lot more digits. Over the years, we’ve been able to calculate it to increasing precision, and although there’s no practical or theoretical reason that we need to know it beyond the 32 digits worked out by Ludolph van Ceulen in the 16th Century, it’s still a fascinating topic that attracts research and debate.

Graph illustrating the calculation of digits of pi over the millenia. Note the logarithmic scale on the left and the staggered scale on the bottom axis.
Our calculation of pi has rocketed since the development of the digital computer.

Most of the computer-based systems we use today are hard to explain, but there’s a really fun computer-based experimental method that can be used to estimate the value of pi that I’m going to share with you. As I’ve been stuck in bed (and often asleep) for the last few days, I’ve not been able to do much productive work, but I have found myself able to implement an example of how to calculate pi. Recovery like a nerd, am I right?

A "pi pie", from a Pi Day celebration.
Pi goes on forever. Pie, sadly, comes to an end.

Remember in school, when you’ll have learned that the formula to describe a circle (of radius 1) on a cartesian coordinate system is x2 + y2 = 1? Well you can work this backwards, too: if you have a point on a grid, (x,y), then you can tell whether it’s inside or outside that circle. If x2 + y2 < 1, it’s inside, and if x2 + y2 > 1, it’s outside. Meanwhile, the difference between the area of a circle and the area of a square that exactly contains it is π/4.

A circle of radius 1 at the intersection of the axes of a cartesian coordinate system.
Think back to your school days. Ever draw a circle like this? Do the words “Cartesian coordinates” ring any bells?

Take those two facts together and you can develop an experimental way to determine pi, called a Monte Carlo method. Take a circle of radius 1 inside a square that exactly contains it. Then randomly choose points within the square. Statistically speaking, these random points have a π/4 chance of occurring within the circle (rather than outside it). So if we take the number of points that lie within the circle, divide that by the total number of points, and then multiply by 4, we should get something that approaches the value of pi. You could even do it by hand!

Output of Dan's demonstration of the Monte Carlo method as used to approximate the value of pi.
I wrote some software to do exactly that. Here’s what it looks like – the red points are inside the circle, and the black points are outside.

The software illustration I’ve written is raw JavaScript, HTML, and SVG, and should work in any modern web browser (though it can get a little slow once it’s drawn a few thousand points!). Give it a go, here! When you go to that page, your browser will start drawing dots at random points, colouring them red if the sum of the squares of their coordinates is less than 1, which is the radius of the circle (and the width of the square that encompasses it). As it goes along, it uses the formula I described above to approximate the value of pi. You’ll probably get as far as 3.14 before you get bored, but there’s no reason that this method couldn’t be used to go as far as you like: it’s not the best tool for the job, but it’s super-easy to understand and explain.

Oh, and it’s all completely open-source, so you’re welcome to take it and do with it what you wish. Turn off the graphical output to make it run faster, and see if you can get an accurate approximation to 5 digits of pi! Or slow it down so you can see how the appearance of each and every point affects the calculation. Or adapt it into a teaching tool and show your maths students one way that pi can be derived experimentally. It’s all yours: have fun.

And I’ll update you on my health at some other point.

× × × × ×

An idea to help MegaMen find one another more easily in the big wide world of Reddit. What do you think?

This self-post was originally posted to /r/MegaManlounge. See more things from Dan's Reddit account.

[this post was originally made to a private subreddit]

I love the MegaLounges, and I really love the MegaManLounge. We’re a hugely disparate group of people yet we’ve come together into a wonderful community that I’m proud to be a part of. And I felt like it’d be nice to give something back. But what?

If you’re like me, you love the experience of bumping into another MMLer elsewhere in the Redditverse (or around the Internet in general). I mean, what’d be really awesome is if we could find one another in the real world, but that’s a project for another day. Anyway: my point is that I get a thrill when I spot a fellow MMLer wandering around in Redditland. But oftentimes I don’t look closely at people’s usernames, and I’m sure there must be times that I’ve just overlooked one of you in some long thread in /r/AskReddit or /r/TodayILearned or something. I’d rather know that you were there, my MML brothers and sisters.

So I spent this afternoon putting together a tool that does just that. Here’s a screenshot to show you what I’m talking about.

I’ve written a basic browser plugin that highlights MMLers (and other MegaLounge-like folks) anywhere on Reddit. So the idea is, if you install this plugin, you’ll always know if somebody’s an MMLer or a MegaLounger because they’ll get one or two icons next to their name. In the screenshot – taken on /r/MegaLoungeVenus (the 23rd MegaLounge) you’ll see a snipped of a conversation between our very own /u/love_the_heat and /u/teiu88. /u/love_the_heat has two icons: the first one (obviously) indicates that he’s a MegaMan, and the second one shows that he’s reached MegaLounge level thirty-one (yes, there are quite a lot of MegaLounge levels now). /u/teiu88 only has one icon (he’s not a MegaMan!), showing that he’s at MegaLounge level twenty-three. Note that it’s coloured differently to show that this is the level that I’m looking at right now: this helps because I can see whether people are commenting at their highest lounge level or not, which may factor into my decision about where and when to gild them.

Someday, I’d like to make this available to MegaLoungers in general, but first I’d like to show it off to you, fine MegaMen, and hear what you think. Is this tool useful to anybody? Should I make a production-grade version to share with you all? Or am I solving a problem that nobody actually has?

Just to add: there are several things I’d like to add and questions I’ve not yet answered before I release it to you; notably:

  • Right now it identifies members of the Super Secret MegaLounge, which is a violation of the rules of that lounge, so obviously I can’t release it yet. I’d like to find a way to have it identify such people but only to other members of that lounge, but failing that, I need to have it just “skip” that lounge when showing how high somebody’s ascended.
  • On which note: what do you think about it identifying MegaMen? If I ever make this tool more-widely available than the MegaManLounge, should the version used by non-MegaManLounge people identify MegaManLounge members, or not? I can see arguments either way, but I will of course go with the will of you fabulous people on this matter.
  • I’d like to add tooltips so that people who haven’t got the entire MegaLounge ascension mapped out in their minds can work out what’s what.
  • Similarly, I’d like to improve the icons so that they e.g. have gemstones next to the gemstone lounges, planets next to the planetary ones, etc.
  • Oh, and I really ought to make it work in more than just Firefox. I’d like it to work in Chrome, at the very least, too. IE can suck it, mind.

What do you think?

tl;dr: I’ve made a browser plugin that makes Reddit look like this, showing people’s highest MegaLounge and MegaManLounge status. Is it a good idea?

Teaching Kids to Code – Why It Matters

The BBC ran a story this week about changes to the National Curriculum that’ll introduce the concepts of computing programming to children at Key Stage 1: that is, between the ages of five and seven. I for one think that this is a very important change, long overdue in our schools. But I don’t feel that way because I think there’ll be a huge market for computer programmers in 13+ years, when these children leave school: rather, I think that learning these programming skills provide – as a secondary benefit – an understanding of technology that kids today lack.

Computer program that asks if you're a boy or a girl and then says it likes that gender. Photograph copyright Steven Luscher, used under Creative Commons license.
Ignoring the implied gender binary (fair enough) and the resulting inefficiency (why do you need to ask two questions), this is a great example of a simple algorithm.

Last year, teacher and geek Marc Scott wrote an excellent blog post entitled Kids Can’t Use Computers… And This Is Why It Should Worry You. In it, he spoke of an argument with a colleague who subscribed to the popular belief that children who use computers are more technically-literate than computer-literate adults. Marc refutes this, retorting that while children today make use of computers more than most adults (and far more than was typical during the childhood of today’s adults), they typically know far less about what Marc calls “how to use a computer”. His article is well worth reading: if you don’t have the time you should make the time, and if you can’t do that then here’s the bottom line: competency with Facebook, YouTube, Minecraft, and even Microsoft Office does not in itself demonstrate an understanding of “how to use a computer”. (Marc has since written a follow-up post which is also worth reading.)

Children programming on laptops. Photograph copyright Steven Luscher, used under Creative Commons license.
If the can of Mountain Dew wasn’t clue enough, these children are coding.

An oft-used analogy is that of the automobile. A hundred years ago, very few people owned cars, but those people that did knew a lot about the maintenance and inner workings of their cars, but today you can get by knowing very little (I’ve had car-owning friends who wouldn’t know how to change to their spare tyre after a puncture, for example). In future, the requirements will be even less: little Annabel might be allowed to ‘drive’ without ever taking a driving test, albeit in a ‘driverless’ computerised car. A similar thing happened with computers: when I was young, few homes had a computer, but in those that did one or more members of the family invariably knew a lot about setting up, configuring, maintaining, and often programming it. Nowadays, most of the everyday tasks that most people do with a computer (Facebook, YouTube, Minecraft, Microsoft Office etc.) don’t need that level of knowledge. But I still think it’s important.

A 2-year-old using a MacBook. Photograph copyright Donnie Ray Jones, used under Creative Commons license.
A future computer-literate, or just another computer “user”?

Why? Because understanding computers remains fundamental to getting the most out of them. Many of us now carry powerful general-purpose computers in our pockets (disguised as single-purpose devices like phones) and most of us have access to extremely powerful general-purpose computers in the form of laptops and desktops (but only a handful of us use them in a ‘general purpose’ way; for many people, they’re nothing more than a web browser and a word processor). However, we expect people to be able to understand the issues when we ask them – via their elected officials – to make sweeping decisions that affect all of us: decisions about the censorship of the ‘net (should we do it, and to what extent, and can we expect it to work?) or about the automation of our jobs (is it possible, is it desirable, and what role will that mean for humans?). We expect people to know how to protect themselves from threats like malicious hackers and viruses and online scams, but we give them only a modicum of support (“be careful, and install anti-virus software”), knowing full well that most people don’t have the foundation of understanding to follow that instruction. And increasingly, we expect people to trust that software will work in the way that it’s instructed to without being able to observe any feedback. Unlike your car, where you may know that it’s not working when it doesn’t go (or, alarmingly, doesn’t stop) – how is the average person to know whether their firewall is working? You can find out how fast your car can go by pressing the pedals, but how are you to know what your computer is capable of without a deeper understanding than is commonplace?

A simple program in Hackety Hack.
I first started to learn to program in Locomotive BASIC. My microcomputer was ready to receive code from the second it booted: no waiting, just programming. Nowadays, there’s a huge barrier to entry (although tools like Hackety Hack, pictured, are trying to make it easier).

A new generation of children tought to think in terms of how computers and their programs actually work – even if they don’t go on to write programs as an adult – has the potential to usher in innovating new ways to use our technology. Just as learning a foreign language, even if you don’t go on to regularly use it, helps make you better at your native language, as well as smarter in other ways (and personally, I think we should be teaching elementary Esperanto – or better yet, Ido – to primary school children in order to improve their linguistic skills generally), learning the fundamentals of programming will give children a far greater awareness about computers in general. They’ll be better-able to understand how they work, and thus why they sometimes don’t do what you expect, and better-equipped to solve problems when they see them. They’ll have the comprehension to explain what they want their computer to be able to do, and to come up with new ideas for ways in which general-purpose computers can be used. And, I’ve no doubt, they’ll be better at expressing logical concepts in mutually-intelligble ways, which improves human communication on the whole.

Let’s teach our kids to be able to understand computers, not just “use” them.

×

You Can’t Do It Alone!

Hot on the heels of the Oxford Steganography Series, a series of “hidden in plain sight”-themed geocaches I placed earlier this year, I’ve recently placed another geocache – GC591VV – and I think that – conceptually – it might be completely unique.

Spoiler warning: this article contains minor spoilers about Oxford-based geocache GC591VV – “You Can’t Do It Alone”. You won’t find any shortcuts by reading this page, but you might ruin part of the surprise.

Geocache GC591VV, ready to hide
Geocache GC591VV, ready to hide

The cache’s name is the first clue that there’s something unusual about it, and this theme continues in its description, where it insists that this is a cache that is impossible to find by yourself. Experienced ‘cachers may have come across geocaches that benefit from a second pair of hands: usually to help “bunk you up” to a high spot. Some really clever caches use your “buddy” to press a battery-powered radio doorbell while you “listen” for the cache’s hiding place nearby, or use your friend to stop up the holes in a pipe as you pour water into it (in order to raise a floating cache to the surface). But every single one of these has a “workaround” – a way in which you can do it by yourself, if you’re imaginative enough. I wanted to make a cache that genuinely required two people.

Dan reprogrammes Box One in a forest
Creating this geocache took months of planning, scouting, construction, and – on one unfortunate occasion – standing around in the rain, deep in a forest, with a C debugger.

The cache description page repeatedly insists that to solve the cache, you need you and a friend to simultaneously visit two different waypoints. When you and your friend get there, you discover why: at each, I’ve hidden a small electronic device, specially-built for this purpose (and instructions on how to use it). The two devices are a synchronised pair, and each shows on its screen a pair of numbers. To find the location of the cache itself, you need to add the first number on Box One’s screen to the first number on Box Two’s screen… and the second number on Box One’s screen to the second number on Box Two’s screen. But… the numbers change every 15 minutes: and because both boxes are hidden on opposite sides of Oxford, there’s no way to get from one to the other within the narrow window. Truly: you can’t do it alone!

Box One and Box Two stess test
The numbers shown on the screens in this photo, taken during their month-long “stress test”, are fake (they point to N 51° 12.123, W 001° 12.123: that’s not where the cache is).

Once the two cachers have each other’s numbers, they can head on to the final coordinates: the actual location of the cache: they can race there, if they like (it’s close-to equidistant from the two points) – though if they’re feeling that competitive, they’ll probably want to agree on some key exchange mechanism by which they can swap numbers without giving the person to speak first a disadvantage: I’ll leave that mathematical exercise for somebody else to solve, though! In any case, I’ve been sure to put two “first to find” prizes into the cache: one for each of the people who worked together to find it.

Visual schematic for You Can't Do It Alone boxes
Each box is operated by a pile of dirt-cheap Chinese-made components, inexpertly soldered together by me.

How does this magic work? Well, it’s reasonably simple, so long as you’re familiar with the conceptual workings of time-based two-factor tokens and the predictability of computer random number generators. I’m offering the source code and support in construction to anybody who successfully finds the cache, in order to try to inspire a new generation of digital caches in Oxfordshire (and further afield!). But the essence of it is an ATmega328 chip acting like an Arduino Nano, hooked up to a clock chip (powered by a long-life lithium “watch battery”) that keeps it in sync with its partner, and – while a switch is pushed – fully-powered by a stack of AA batteries (which provide enough power to do the maths and light up the screen).

Harmless Electronic Game Piece, written above Box One, in situ
Given how alarmed people sometimes get when they find sealed black boxes with digital screens, slowly counting down, I decided to include a reassuring note with each box.

The whole package is sealed up inside a custom-built acrylic box (courtesy of RazorLAB, whom I discovered after Rory did a craft project using them), and I’m hoping that they’ll live at least a year before I need to get out there and replace the AA batteries.

This cache represents a huge leap in complexity over anything I’ve placed before, and – I think! – might be completely unique in design, worldwide. I’m really looking forward to seeing what the community make of it! Want to go find it yourself? Start here!

Update (2018): This cache has now been retired. If you’re interested, you can read the source code here.

× × × × ×

Programming Is Not Math

This is a repost promoting content originally published elsewhere. See more things Dan's reposted.

When I learned to program, back when dinosaurs walked the earth and the internet had no cats, there was an idea: if you were good at math, you’d be good at programming. I was great at math as a kid, but perhaps because I didn’t like it much, no one steered me towards programming. I…