Blog

More than you expected?

You're reading everything on Dan's blog - including notes, reposts, checkins, videos and comics.
That might be more than you wanted to see, if you're only interested in blog posts (articles) Dan has written.

Ted Chiang Explains the Disaster Novel We All Suddenly Live In

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

While there has been plenty of fiction written about pandemics, I think the biggest difference between those scenarios and our reality is how poorly our government has handled it. If your goal is to dramatize the threat posed by an unknown virus, there’s no advantage in depicting the officials responding as incompetent, because that minimizes the threat; it leads the reader to conclude that the virus wouldn’t be dangerous if competent people were on the job. A pandemic story like that would be similar to what’s known as an “idiot plot,” a plot that would be resolved very quickly if your protagonist weren’t an idiot. What we’re living through is only partly a disaster novel; it’s also—and perhaps mostly—a grotesque political satire.

What will “normal” look like after the coronavirus crisis has passed? Will it be the same normal as we’re used to? Or could we actually learn some lessons from this and progress towards something better?

I love Ted Chiang’s writing; enough to reshare this interview even though I’m only lukewarm about it!

Third-party libraries and security issues

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Earlier this week, I wrote about why you should still use vanilla JS when so many amazing third-party libraries exist.

A few folks wrote to me to mention something I missed: security.

When you use code you didn’t author, you’re taking a risk. You’re trusting that the third-party code does not have security issues, that the author has good intent.

Chris makes a very good point, especially for those developers of the npm install every-damn-thing persuasion: getting an enormous framework that you don’t completely understand just because you need  a small portion of its features is bad security practice. And the target is a juicy one: a bad actor who finds (or introduces) a vulnerability in a big and widely-used library has a whole lot of power. Security concerns are a major part of why I go vanilla/stdlib where possible.

But as always with security the answer isn’t so clear-cut and simple, and I’d argue that it’s dangerous to encourage people to write their own solutions as a matter of course, for security reasons. For a start, you should never roll your own cryptographic libraries because you’re almost certainly going to fuck it up: an undetectable and easy-to-make mistake in your crypto implementation can lead to a catastrophic cascade and completely undermine the value of your cryptography. If you’re smart enough about crypto to implement crypto properly, you should contribute towards one of the major libraries. And if you’re not smart enough about crypto (and if you’re not sure, then you’re not), you should use one of those libraries. And even then you should take care to integrate and use it properly: people have been tripped over before by badly initialised keys or the use of the wrong kind of cipher for their use-case. Crypto is hard enough that even experts fuck it up and important enough that you can’t afford to get it wrong.

The same rule applies to a much lesser extent to other parts of your application, and especially for beginner developers. Implementing an authentication/authorisation system isn’t hard, but it’s another thing where getting it wrong can have disastrous consequences. Beginner (and even intermediate) developers routinely make mistakes with this kind of feature: unhashed, reversibly-encrypted, or incorrectly-hashed (wrong algorithm, no salt, etc.) passwords, badly-thought-out password reset strategies, incompletely applied access controls, etc. I’m confident that Chris and I would be in agreement that the best approach is for a developer to learn to implement these things properly and then do so. But if having to learn to implement them properly is a barrier to getting started, I’d rather than a beginner developer instead use a tried-and-tested off-the-shelf like Devise/Warden.

Other examples of things that beginner/intermediate developers sometimes get wrong might be XSS protection and SQL parameter escaping. And again, for languages that don’t have safety features built in, a framework can fill the gap. Rolling your own DOM whitelisting code for a social application is possible, but using a solution like DOMPurify is almost-certainly going to be more-secure for most developers because, you guessed it, this is another area where it’s easy to make a mess of things.

My inclination is to adapt Chris’s advice on this issue, to instead say that for the best security:

  1. Ideally: understand what all your code does, for example because you wrote it yourself.
  2. However: if you’re not confident in your ability to implement something securely (and especially with cryptography), use an off-the-shelf library.
  3. If you use a library: use the usual rules (popularity, maintenance cycle, etc.) to filter the list, but be sure to use the library with the smallest possible footprint – the best library should (a) do only the one specific task you need done, and no more, and (b) be written in a way that lends itself to you learning from it, understanding it, and hopefully being able to maintain it yourself.

Just my tuppence worth.

The 7 Types Of StackOverflow Answers

StackOverflow‘s one of the most-popular and widely-used resources for software developers. It dominates the search results when you’re looking for answers to techy questions. If you know how to read it, it can be invaluable.

But… I’m not sure what it is about the platform or the culture surrounding it that creates a certain… pattern to the answers that you can expect to receive on StackOverflow. To illustrate, let’s suppose we have a question:

SnackOverflow question: Let's say I'm camping and I need to make toast. I have a loaf of bread and a campfire. What's the best way to make toast?

Here are the answers you might see:

The Golden Hammer

The top answer is often somebody answering not the question you asked, but the question they’d like to think you asked.

Answer: Just plug a toaster in. You can do this with: npm install toaster

Never mind that you specifically said that you were using a campfire, the answer suggests that you use a toaster. Look back a few years and you’ll see countless examples of people asking for solutions using “vanilla” JavaScript and being told to use some heavyweight, everything-but-the-kitchen sink jQuery plugin. Now we’re in a more enlightened time, those same people are being told to use some heavyweight, everything-but-the-kitchen-sink npm module. How far we’ve come.

The Belligerent

Far often than you might expect, a perfectly reasonable “how do I do this?” question is met with an aggressive response of “why would you want to do that?”

Answer: Why would you want to make toast on a campfire? When you're camping you should be eating beans, soup, and spit-roasted meats and fish. Every time I've tried to toast bread over a campfire I've ended up unsatisfied. Uneven toasting, burnt bits, even the whole slice catching fire. I can't imagine why anybody would ever want toast like that! If you want toast you should stay at home. It's still pretty pointless, though: toast isn't a very good meal. It's basically empty calories with no protein, no vitamins, no minerals. I mean, it'd be okay as a snack but that's clearly not what you're asking about. There's a reason that the Chef's Guide To Camping doesn't include a recipe for toast. Just don't do it!

These are particularly infuriating to read when you come to a closed thread and you know that you do want to be doing the “forbidden” thing. You’ve considered the other options, you’ve assessed the situation… and now some arrogant bugger’s telling you that you’re wrong!

This kind of response is among the most annoying, second only to…

The Kindred Spirit

You’re getting a strange and inexplicable error message. You search for it and get exactly one result. Reading the thread, after hours of tearing your hair out, you suddenly feel a sense of relief: you’ve found another soul in this crazy world that’s suffering in precisely the same way as you are. Every word you read reconfirms for you that you and they have the same issue. At last, a solution is in reach!

Answer: I'm having almost exactly the same issue. I've brought bagels to my campfire, though. If anybody knows how to toast either bread or bagels on a campfire please let me know how! Edit: NM, I've worked it out.Nope.

Not only have you not got a solution, but the saviour you thought you’d found? They do have a solution, but they were thinking only about themselves when they got it, so they didn’t share it.

I get it: when you’re deep in focus on a problem you forget that the forum you’re on will receive search traffic indefinitely. But “NM, I’ve worked it out” is the most infuriating sentence on the Internet. When you solve a tough problem that you’d talked about online, for the love of God put the solution online too.

The Expert

There’s always somebody who answers the question but in a way you’d need a PhD to comprehend.

Answer: What you're looking to do is increase the ratio of 6-Acetyl-2,3,4,5-tetrahydropyridine on the surface of the bread, as described by Louis-Camille Maillard. Aim to maximise the surface area exposed to heat to accelerate the reaction of the carbonyls with the nucleophilic amino acids, without increasing the temperature enough to produce significant amounts of benzopyrene nor acrylamide.

StackOverflow is often used by beginners. Make your answer beginner-friendly if possible.

The Hero We Don’t Need

Like the Golden Hammer, the Hero We Don’t Need answers the question that they know the answer to rather than the question you actually asked. Unlike the Golden Hammer, the question they answer isn’t even remotely related to the question you asked.

Answer: Place the loaf down on a broad flat surface. Use a serrated blade in a moderately-rapid back-and-forth motion to cut through it. Now the bread will be sliced and ready to use. Don't cut any more than you need at once: sliced bread goes stale much faster.

Perhaps some future site visitor who chose their search terms badly might benefit from this out-of-the-box look at a completely different problem. But I wouldn’t count on it.

The Correct Answer

Eventually, if you’re lucky, somebody will provide the actual answer to the question. You’ll often have to scroll about this far down the page to find it.

Answer: There are two approaches. Both are equally valid - choose the one that's right for you. Method #1: place flat rocks near to your campfire and allow them to heat up. Slice your bread, and lay each slice on a hot rock, being careful not to touch the rock. Turn it over when it's done on one side. Method #2: use a long fork, skewer, or stick to impale a slice of bread lengthways (here's a diagram) and suspend it over the fire either by holding the utensil or by poking the other end into the ground. If holding it, be sure to keep your hand lower than the bread as heat will travel up metal implements. Happy camping!Still, at least there’s an answer. And it only took four hours between posting the question and it appearing. Sometimes that’s what it takes, and at least the answer will be there for the next person, assuming that they, too, scroll down far enough.

Unfortunately hundreds of novice developers will have no way to tell that this alone is the correct answer amongst the endless stream of bullshit in which it resides.

The Echo

And finally, there’s always some idiot who repeats one of the same (useless) answers from before. Just to keep the noise-to-signal ratio up, I guess.

Answer: Just install toaster from NPM. Comment 1: @KISS DRY already said this. This is the correct answer. Comment 2: Can toaster slice bread, too?

StackOverflow’s given me so many useful answers to so many questions, over the years. But it’s also been a great source of frustration for me at the hands of six of these seven archetypes. Did I miss any?

idTech 4 WebAssembly port – Doom 3 Demo

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Doom 3 running in Dan's web browser

Back in 2011, some folks cross-compiled Doom (the original, not the reboot, obviously) to JavaScript, leveraging the capabilities of the then-relatively-young <canvas> element and APIs. I was really impressed to see that JavaScript had come so far and that performance on desktop devices was so slick. Sure, this was an 18-year-old video game, but it was playable in a browser, which was a long way from the environment for which it was originally developed.

Now Doom 3‘s playable in a browser, and my mind’s blown all over again. This follows almost the same curve – Doom 3’s 16 years old – but it still goes to show that there’s little limit to the power of client-side browser programming. They’ve done this magic with WebAssembly; while WebAssembly goes slightly against my ideas about the open-source nature of the Web, I still respect the power it commands to do heavyweight crunching tasks like this one.

How long until AAA developers start developing with the Web as an additional platform?

That Discomfort You’re Feeling Is Grief

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

There is something powerful about naming this as grief. It helps us feel what’s inside of us. So many have told me in the past week, “I’m telling my coworkers I’m having a hard time,” or “I cried last night.” When you name it, you feel it and it moves through you. Emotions need motion. It’s important we acknowledge what we go through.

Scott makes a good point; the experience of the coronavirus crisis and lockdowns is distinctly grief-like. Insofar as the Kübler-Ross model is applicable in general, it’s a good predictor of individuals’ reactions to their temporary “new normal”. But the lesson to take from this article, I think, isn’t about understanding the feelings and behaviour of your fellow humans but, as the author says, in giving a name to your own.

The realisation that what you’re experiencing is grief and that it’s okay to need an indefinite amount of time to process that is empowering and reassuring.

Apple just killed Offline Web Apps while purporting to protect your privacy

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

On the face of it, WebKit’s announcement yesterday titled Full Third-Party Cookie Blocking and More sounds like something I would wholeheartedly welcome. Unfortunately, I can’t because the “and more” bit effectively kills off Offline Web Apps and, with it, the chance to have privacy-respecting apps like the prototype I was exploring earlier in the year based on DAT.

Block all third-party cookies, yes, by all means1. But deleting all local storage (including Indexed DB, etc.) after 7 days effectively blocks any future decentralised apps using the browser (client side) as a trusted replication node in a peer-to-peer network. And that’s a huge blow to the future of privacy.

Like Aral and doubtless many others, I was initially delighted to see that Safari has beaten Chrome to the punch, blocking basically all third-party cookies through its Intelligent Tracking Protection. I don’t even routinely use Safari (although I do block virtually all third-party and many first-party cookies using uMatrix for Firefox), but I loved this announcement because I knew that this, coupled with Google’s promise to (eventually) do the same in their browser, would make a significant impact on the profitability of surveillance capitalism on the Web. Hurrah!

But as Aral goes on to point out, Apple’s latest changes also effectively undermines the capability of people to make Progressive Web Applications that run completely-offline, because their new privacy features delete the cache of all offline storage if it’s not accessed for 7 days.

PWAs have had a bumpy ride. They were brought to the foreground by Apple in the first place when Steve Jobs suggested that something-like-this would be the way that apps should one day be delivered to the iPhone, but then that idea got sidelined by the App Store. In recent years, we’ve begun to see the concept take off again as Chrome, Firefox and Edge gradually added support for service workers (allowing offline-first), larger local storage, new JavaScript interfaces for e.g. cameras, position, accelerometers, and Bluetooth, and other PWA-ready technologies. And for a while I thought that the day of the PWA might be drawing near… but it looks like we might have to wait a bit longer.

I hope that Google doesn’t follow Apple’s lead on this particular “privacy” point, although I’m sure that it’s tempting for them to do so. Offline Web applications have the potential to provide an open, simple, and secure ecosystem for the “apps” of tomorrow, and after several good steps forwards… this week we took a big step back.

Gratitude

In these challenging times, and especially because my work and social circles have me communicate regularly with people in many different countries and with many different backgrounds, I’m especially grateful for the following:

  1. My partner, her husband, and I each have jobs that we can do remotely and so we’re not out-of-work during the crisis.
  2. Our employers are understanding of our need to reduce and adjust our hours to fit around our new lifestyle now that schools and nurseries are (broadly) closed.
  3. Our kids are healthy and not at significant risk of serious illness.
  4. We’ve got the means, time, and experience to provide an adequate homeschooling environment for them in the immediate term.
  5. (Even though we’d hoped to have moved house by now and haven’t, perhaps at least in part because of COVID-19,) we have a place to live that mostly meets our needs.
  6. We have easy access to a number of supermarkets with different demographics, and even where we’ve been impacted by them we’ve always been able to work-around the where panic-buying-induced shortages have reasonably quickly.
  7. We’re well-off enough that we were able to buy or order everything we’d need to prepare for lockdown without financial risk.
  8. Having three adults gives us more hands on deck than most people get for childcare, self-care, etc. (we’re “parenting on easy mode”).
  9. We live in a country in which the government (eventually) imposed the requisite amount of lockdown necessary to limit the spread of the virus.
  10. We’ve “only” got the catastrophes of COVID-19 and Brexit to deal with, which is a bearable amount of crisis, unlike my colleague in Zagreb for example.
Bowl of ice, glasses of water, salt and sugar supplies.
Today’s homeschool science experiment was about what factors make ice melt faster. Because of course that’s the kind of thing I’d do with the kids when we’re stuck at home.

Whenever you find the current crisis getting you down, stop and think about the things that aren’t-so-bad or are even good. Stopping and expressing your gratitude for them in whatever form works for you is good for your happiness and mental health.