Because I work somewhere hip enough to let people tweak their job titles, mine is “Code Magician”.
LinkedIn isn’t as hip as Automattic, though. That’s why they keep emailing me sector updates… for the “Magician” sector… 😅
Because I work somewhere hip enough to let people tweak their job titles, mine is “Code Magician”.
LinkedIn isn’t as hip as Automattic, though. That’s why they keep emailing me sector updates… for the “Magician” sector… 😅
During a conversation with a colleague last week, I claimed that while I blog more-frequently than I did 5-10 years ago, it’s still with a much lower frequency than say 15-20 years ago.
Only later did I stop to think: is that actually true? It’s time for a graph!
I’ve previously graphed my blogging in an ad-hoc way, e.g. in 2016 I did a word-count and in 2021 I graphed posts-by-month-of-year, but I’ve never made an “eternal”, automatically-updating, interactive1 graph. Until now:
Generating a chart...
If this message doesn't go away, the JavaScript that makes this magic work probably isn't doing its job right: please tell Dan so he can fix it.
Generating a chart...
If this message doesn't go away, the JavaScript that makes this magic work probably isn't doing its job right: please tell Dan so he can fix it.
If you consider just articles (and optionally notes, which some older content might have been better classified-as, in retrospect) it looks like I’m right. Long gone are months like February 2005 when I posted an average of three times every two days! November 2018 was a bit of an anomaly as a I live-tweeted Challenge Robin II: my recent output’s mostly been comparable to the “quiet period” from 2008-20102.
Looking at number of posts by month of the year, it’s interesting to see a pronounced “dip” in all kinds of output roundabout March, less reposts in Summer and Autumn, and – perhaps unsurprisingly – more checkins (which often represent geocaching/geohashing logs) in the warmer months. Even on this scale, you can see the impact of the November “Challenge Robin spike” in the notes:
Generating a chart...
If this message doesn't go away, the JavaScript that makes this magic work probably isn't doing its job right: please tell Dan so he can fix it.
Anyway, now I’ve actually automated these kinds of stats its easier than ever for me to ask questions about how and when I write in my blog. I’ve put living copies of the charts plus additional treats (want to know when my longest “daily streak” was?) on a special page dedicated to that purpose. It’ll be interesting to see how it looks on this blog’s 25th anniversary, in a little under a year!
1 Try clicking on any of the post kinds in the legend to add/remove them, or click-and-drag a range across the chart to zoom in.
2 In hindsight, I was clearly depressed in and around 2009 and this doubtless impacted my ability to engage in “creative” pursuits.
Because I like my blog to be as fast, accessible, and resilient, I try not to use JavaScript for anything I don’t have to1. One example would be my “lightbox”: the way in which images are blown-up if you click on them:
My solution ensures that:
Here’s how it works –
<figure id="img3336" aria-describedby="caption-img3336"> <a href="#lightbox-img3336" role="button"> <img src="small-image.jpg" alt="Alt text is important." width="640" height="480"> </a> <figcaption id="caption-img3336"> Here's the caption. </figcaption> </figure> ... (rest of blog post) ... <dialog id="lightbox-img3336" class="lightbox"> <a href="large-image.jpg"> <img src="large-image.jpg" loading="lazy" alt="Alt text is important."> </a> <a class="close" href="#img3336" title="Close image" role="button">×</a> </dialog>
For each lightboxed image in a post, a <dialog> for that image is appended to the post. That dialog contains a larger copy of the image (set to
loading="lazy" so the browser have to download it until it’s needed), and a “close” button.
The image in the post contains an anchor link to the dialog; the close button in the dialog links back to the image in the post.3 I wrap the lightbox image itself in a link to the full version of the image, which makes it easier for users to zoom in further using their browser’s own tools, if they like.
Even without CSS, this works (albeit with “scrolling” up and down to the larger image). But the clever bit’s yet to come:
body:has(dialog:target) { /* Prevent page scrolling when lightbox open (for browsers that support :has()) */ position: fixed; } a[href^='#lightbox-'] { /* Show 'zoom in' cursor over lightboxed images. */ cursor: zoom-in; } .lightbox { /* Lightboxes are hidden by-default, but occupy the full screen and top z-index layer when shown. */ all: unset; display: none; position: fixed; top: 0; left: 0; width: 100%; height: 100%; z-index: 2; background: #333; } .lightbox:target { /* If the target of the URL points to the lightbox, it becomes visible. */ display: flex; } .lightbox img { /* Images fill the lightbox. */ object-fit: contain; height: 100%; width: 100%; } /* ... extra CSS for styling the close button etc. ... */
Lightboxes are hidden by default (display: none), but configured to fill the window when shown.
They’re shown by the selector .lightbox:target, which is triggered by the id of the <dialog> being referenced by the anchor part of
the URL in your address bar!
It’s neither the most-elegant nor cleanest solution to the problem, but for me it hits a sweet spot between developer experience and user experience. I’m always disappointed when somebody’s “lightbox” requires some heavyweight third-party JavaScript (often loaded from a CDN), because that seems to be the epitome of the “take what the Web gives you for free, throw it away, and reimplement it badly in JavaScript” antipattern.
There’s things I’ve considered adding to my lightbox. Progressively-enhanced JavaScript that adds extra value and/or uses the Popover API where available, perhaps? View Transitions to animate the image “blowing up” to the larger size, while the full-size image loads in the background? Optimistic preloading when hovering over the image4? “Previous/next” image links when lightboxing a gallery? There’s lots of potential to expand it without breaking the core concept here.
I’d also like to take a deeper dive into the accessibility implications of this approach: I think it’s pretty good, but accessibility is a big topic and there’s always more to learn.
I hope the idea’s of use to somebody else looking to achieve this kind of thing, too.
1 Where JavaScript is absolutely necessary, I (a) host it on the same domain, for performance and privacy-respecting reasons, and (b) try to provide a functional alternative that doesn’t require JavaScript, ideally seamlessly.
2 In practice, the lightbox images get lazy-loaded, so there can be a short round trip to fetch the image the first time. But after that, it’s instantaneous.
3 The pair – post image and lightbox image – work basically the same way as footnotes, like this one.
4 I already do this with links in general using the excellent instant.page.
This is a repost promoting content originally published elsewhere. See more things Dan's reposted.
The web loves data. Data about you. Data about who you are, about what you do, what you love doing, what you love eating.
…
I, on the other end, couldn’t care less about your data. I don’t run analytics on this website. I don’t care which articles you read, I don’t care if you read them. I don’t care about which post is the most read or the most clicked. I don’t A/B test, I don’t try to overthink my content. I just don’t care.
…
Manu speaks my mind. Among the many hacks I’ve made to this site, I actively try not to invade on your privacy by collecting analytics, and I try not to let others to so either!
My blog is for myself first and foremost (if you enjoy it too, that’s just a bonus). This leads to two conclusions:
I’ve gone into more detail about this on my privacy page and hinted at it on my colophon. But I don’t know if anybody ever reads either of those pages, of course!
A Birmingham-based punky trio, Luxury Nan Smell, have released an EP called (Derogatory). The first track on that album? freedeedpoll.org.uk. Named in reference to my website of the same name.
Naturally, I was delighted, not least because it gives me an excuse to use the “deed poll” and “music” tags simultaneously on a post for the first time.
Don’t ask me what my “real” name is,
I’ve already told you what it was,
And I’m planning on burning my birth certificate.
The song’s about discovering and asserting self-identity through an assumed, rather than given, name. Which is fucking awesome.
Like virtually all of my sites, including this one, freedeedpoll.org.uk deliberately retains minimal logs and has no analytics tools. As a result, I have very little concept of how popular it is, how widely it’s used etc., except when people reach out to me.
People do: I get a few emails every month from people who’ve got questions1, or who are having trouble getting their homemade deed poll accepted by troublesome banks. I’m happy to help them, but without additional context, I can’t be sure whether these folks represent the entirety of the site’s users, a tiny fraction, or somewhere in-between.
So it’s obviously going to be a special surprise for me to have my website featured in a song.
I’ve been having a challenging couple of weeks2, and it was hugely uplifting for me to bump into these appreciative references to my work in the wider Internet.
1 Common questions I receive are about legal gender recognition, about changing the names of children, about changing one’s name while still a minor without parental consent, or about citizenship requirements. I’ve learned a lot about some fascinating bits of law.
2 I’ve been struggling with a combination of the usual challenges at this time of year and a lack of self-care and also a handful of bonus household stresses: everything seems to be breaking all at once!
This post is also available as an article. So if you'd rather read a conventional blog post of this content, you can!
This is a video version of my blog post, Length Extension Attack. In it, I talk through the theory of length extension attacks and demonstrate an SHA-1 length extension attack against an (imaginary) website.
The video can also be found on:
This post is also available as a video. If you'd prefer to watch/listen to me talk about this topic, give it a look.
Prefer to watch/listen than read? There’s a vloggy/video version of this post in which I explain all the key concepts and demonstrate an SHA-1 length extension attack against an imaginary site.
I understood the concept of a length traversal attack and when/how I needed to mitigate them for a long time before I truly understood why they worked. It took until work provided me an opportunity to play with one in practice (plus reading Ron Bowes’ excellent article on the subject) before I really grokked it.
Would you like to learn? I’ve put together a practical demo that you can try for yourself!
You can check out the code and run it using the instructions in the repository if you’d like to play along.
The site “Images R Us” will let you download images you’ve purchased, but not ones you haven’t. Links to the images are protected by a SHA-1 hash1, generated as follows:
When a “download” link is generated for a legitimate user, the algorithm produces a hash which is appended to the link. When the download link is clicked, the same process is followed and the calculated hash compared to the provided hash. If they differ, the input must have been tampered with and the request is rejected.
Without knowing the secret key – stored only on the server – it’s not possible for an attacker to generate a valid hash for URL parameters of the attacker’s choice. Or is it?
download=free to download=valuable invalidates the hash, and the request is denied.
Actually, it is possible for an attacker to manipulate the parameters. To understand how, you must first understand a little about how SHA-1 and its siblings actually work:
SECRET_KEY + URL_PARAMS) is cut into blocks of a fixed size.2
In SHA-1, blocks are 512 bits long and the padding is a 1, followed by as many 0s as is necessary,
leaving 64 bits at the end in which to specify how many bits of the block were actually data.
Looking at the final block in a given message, it’s apparent that there are two pieces of data that could produce exactly the same output for a given function:
Therefore, if we can manipulate the input of the message, and we know the length of the message, we can append to it. Bear that in mind as we move on to the other half of what makes this attack possible.
“Images R Us” is implemented in PHP. In common with most server-side scripting languages, when PHP sees a HTTP query string full of key/value pairs, if a key is repeated then it overrides any earlier iterations of the same key.
$_SERVER['QUERY_STRING'] in PHP, where you’ll find the entire query string.
You could even implement your own query string handler that instead makes the first instance of each key the canonical one, if you really wanted.6
download=free parameter in the query string at “Images R Us”, e.g. making it
download=free&download=valuable! But we can’t: not without breaking the hash, which is calculated based on the entire query string (minus the &key=...
bit).
But with our new knowledge about appending to the input for SHA-1 first a padding string, then an extra block containing our payload (the variable we want to override and its new value), and then calculating a hash for this new block using the known output of the old final block as the IV… we’ve got everything we need to put the attack together.
We have a legitimate link with the query string download=free&key=ee1cce71179386ecd1f3784144c55bc5d763afcc. This tells us that somewhere on the server, this is
what’s happening:
download=free with some special characters to replicate the padding that would otherwise be added to this final8 block, we can add a second block containing
an overriding value of download, specifically &download=valuable. The first value of download=, which will be the word free followed by
a stack of garbage padding characters, will be discarded.
And we can calculate the hash for this new block, and therefore the entire string, by using the known output from the previous block, like this:
Of course, you’re not going to want to do all this by hand! But an understanding of why it works is important to being able to execute it properly. In the wild, exploitable implementations are rarely as tidy as this, and a solid comprehension of exactly what’s happening behind the scenes is far more-valuable than simply knowing which tool to run and what options to pass.
That said: you’ll want to find a tool you can run and know what options to pass to it! There are plenty of choices, but I’ve bundled one called hash_extender into my example, which will do the job pretty nicely:
$ docker exec hash_extender hash_extender \
--format=sha1 \
--data="download=free" \
--secret=16 \
--signature=ee1cce71179386ecd1f3784144c55bc5d763afcc \
--append="&download=valuable" \
--out-data-format=html
Type: sha1
Secret length: 16
New signature: 7b315dfdbebc98ebe696a5f62430070a1651631b
New string: download%3dfree%80%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%e8%26download%3dvaluable
I’m telling hash_extender:
sha1), which can usually be derived from the hash length,
download=free), so it can determine the length,
16 bytes), which I’ve guessed but could brute-force,
ee1cce71179386ecd1f3784144c55bc5d763afcc),
&download=valuable), and
html the most-useful generally, but it’s got some encoding quirks that you need to be aware of!
hash_extender outputs the new signature, which we can put into the key=... parameter, and the new string that replaces download=free, including
the necessary padding to push into the next block and your new payload that follows.
Unfortunately it does over-encode a little: it’s encoded all the& and = (as %26 and %3d respectively), which isn’t what we
wanted, so you need to convert them back. But eventually you end up with the URL:
http://localhost:8818/?download=free%80%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%e8&download=valuable&key=7b315dfdbebc98ebe696a5f62430070a1651631b.
And that’s how you can manipulate a hash-protected string without access to its salt (in some circumstances).
The correct way to fix the problem is by using a HMAC in place
of a simple hash signature. Instead of calling sha1( SECRET_KEY . urldecode( $params ) ), the code should call hash_hmac( 'sha1', urldecode( $params ), SECRET_KEY
). HMACs are theoretically-immune to length extension attacks, so long as the output of the hash function used is
functionally-random9.
Ideally, it should also use hash_equals( $validDownloadKey, $_GET['key'] ) rather than ===, to mitigate the possibility of a timing attack. But that’s another story.
1 This attack isn’t SHA1-specific: it works just as well on many other popular hashing algorithms too.
2 SHA-1‘s blocks are 64 bytes long; other algorithms vary.
3 For SHA-1, the padding bits
consist of a 1 followed by 0s, except the final 8-bytes are a big-endian number representing the length of the message.
4 SHA-1‘s IV is 67452301 EFCDAB89 98BADCFE 10325476 C3D2E1F0, which you’ll observe is little-endian counting from 0 to
F, then back from F to 0, then alternating between counting from 3 to 0 and C to F. It’s
considered good practice when developing a new cryptographic system to ensure that the hard-coded cryptographic primitives are simple, logical, independently-discoverable numbers like
simple sequences and well-known mathematical constants. This helps to prove that the inventor isn’t “hiding” something in there, e.g. a mathematical weakness that depends on a
specific primitive for which they alone (they hope!) have pre-calculated an exploit. If that sounds paranoid, it’s worth knowing that there’s plenty of evidence that various spy
agencies have deliberately done this, at various points: consider the widespread exposure of the BULLRUN programme and its likely influence on Dual EC
DRBG.
5 The padding characters I’ve used aren’t accurate, just representative. But there’s the right number of them!
6 You shouldn’t do this: you’ll cause yourself many headaches in the long run. But you could.
7 It’s also not always obvious which inputs are included in hash generation and how they’re manipulated: if you’re actually using this technique adversarily, be prepared to do a little experimentation.
8 In this example, the hash operates over a single block, but the exact same principle applies regardless of the number of blocks.
9 Imagining the implementation of a nontrivial hashing algorithm, the predictability of whose output makes their HMAC vulnerable to a length extension attack, is left as an exercise for the reader.
Received my physical copy of Planets In The Wires today, and I must say it was really cool of Pagan Wanderer Lu to include one of Fred “Thickie” Holden’s famous Tension Sheets as a freebie.
I added a stupid feature to my blog.
On some posts, including this one, you can now send an “emoji reaction”. Y’know, for if you’re too lazy to write a comment.
The available reactions vary by post.
That is all.
Pagan Wanderer Lu‘s new album Planets In The Wires dropped today, and I just cried my eyes out at track 5 (The Frosted Pane).
It opens almost apologetically, like an explanation for the gap in new releases for most of the twenty-teens. But it quickly becomes a poetic exploration of a detached depression of a man trapped under the weight of the world. It’s sad, and beautiful, and relatable.
Lately, Ruth and I have been learning to dance Argentine Tango.
Let me tell you everything I know about tango1:
This adventure began, in theory at least, on my birthday in January. I’ve long expressed an interest in taking a dance class together, and so when Ruth pitched me a few options for a birthday gift, I jumped on the opportunity to learn tango. My knowledge of the dance was basically limited to what I’d seen in films and television, but it had always looked like such an amazing dance: careful, controlled… synchronised, sexy.
After shopping around for a bit, Ruth decided that the best approach was for us to do a “beginners” video course in the comfort of our living room, and then take a weekend getaway to do an “improvers” class.
After all, we’d definitely have time to complete the beginners’ course and get a lot of practice in before we had to take to the dance floor with a group of other “improvers”, right?2
Okay, let me try again to enumerate you everything I actually know about tango3:
Ultimately, it was entirely our own fault we felt out-of-our-depth up in Edinburgh at the weekend. We tried to run before we could walk, or – to put it another way – to milonga before we could caminar.
A somewhat-rushed video course and a little practice on carpet in your living room is not a substitute for a more-thorough práctica on a proper-sized dance floor, no matter how often you and your partner use any excuse of coming together (in the kitchen, in an elevator, etc.) to embrace and walk a couple of steps! Getting a hang of the fluid connections and movement of tango requires time, and practice, and discipline.
But, not least because of our inexperience, we did learn a lot during our weekend’s deep-dive. We got to watch (and, briefly, partner with) some much better dancers and learned some advanced lessons that we’ll doubtless reflect back upon when we’re at the point of being ready for them. Because yes: we are continuing! Our next step is a Zoom-based lesson, and then we’re going to try to find a more-local group.
Also, we enjoyed the benefits of some one-on-one time with Jenny and Ricardo, the amazingly friendly and supportive teachers whose video course got us started and whose in-person event made us feel out of our depth (again: entirely our own fault).
If you’ve any interest whatsoever in learning to dance tango, I can wholeheartedly recommend Ricardo and Jenny Oria as teachers. They run courses in Edinburgh and occasionally elsewhere in the UK as well as providing online resources, and they’re the most amazingly supportive, friendly, and approachable pair imaginable!
Just… learn from my mistake and start with a beginner course if you’re a beginner, okay? 😬
1 I’m exaggerating how little I know for effect. But it might not be as much of an exaggeration as you’d hope.
2 We did not.
3 Still with a hint of sarcasm, though.
4 Tango’s progressive enough that it’s come to reject describing the roles in binary gendered terms, using “leader” and “follower” in place of what was once described as “man” and “woman”, respectively. This is great for improving access to pairs of dancers who don’t consist of a man and a woman, as well as those who simply don’t want to take dance roles imposed by their gender.
Almost a decade ago I shared a process that my domestic polyfamily and I had been using (by then, for around four years) to manage our household finances. That post isn’t really accurate any more, so it’s time for an update (there’s a link if you just want the updated spreadsheet):
For my examples below, assume a three-person family. I’m using unrealistic numbers for easy arithmetic.
We’ve never done things this way, but for completeness sake I’ll mention it: the simplest way that households can split their costs is by dividing them between the participants equally: if the family make a £60 shopping trip, £20 should be paid by each of Alice, Bob, and Chris.
My example above shows exactly why this might not be a smart choice: this model would have each participant contribute £833.33 over the course of the month, which is more than Chris earned. If this month is representative, then Chris will gradually burn through their savings and go broke, while Alice will put over a grand into her savings account every month!
We’re a bunch of leftie socialist types, and wanted to reflect our political outlook in our household finances, too. So rather than just splitting our costs equally between us, we initially implemented a means-assessment system based on the relative differences between our incomes. The thinking was that somebody that earns twice as much should contribute twice as much towards the costs of running the household.
Using our example family above, here’s how that might look:
By analogy: The “Income-Assessed” model is functionally equivalent to splitting each and every expense according to the participants income – e.g. if a £100 bill landed on their doormat, Alice would pay £57, Bob £29, and Chris £14 of it – but has the convenience that everybody just pays for things “as they go along” and then square everything up when their paycheques come in.
Over time, our expenditures grew and changed and our incomes grew, but they didn’t do so in an entirely simple fashion, and we needed to make some tweaks to our income-assessed model of household finance contributions. For example:
Eventually, we came to see that what we were doing was trying to patch a partially-broken system, and tried something new!
In 2022, we transitioned to a same-residual system that attempts to share out out money in an even-more egalitarian way. Instead of each person contributing in accordance with their income, the model attempts to leave each person with the same average amount of disposable personal income at the end. The difference is most-profound where the relative incomes are most-diverse.
With the example family above, that would mean:
That’s a very different result than the Income-Assessed calculation came up with for the same family! Instead of Chris giving money to Alice and Bob, because those two contributed to household costs disproportionately highly for their relative incomes, Alice gives money to Bob and Chris, because their incomes (and expenditures) were much lower. Ignoring any non-household costs, all three would expect to have the same bank balance at the start of the month as at the end, after settlement.
By analogy: The “Same-Residual” model is functionally equivalent to having everybody’s salary paid into a shared bank account, out of which all household expenditures are paid, and at the end of the month everything that’s left in the bank account gets split equally between the participants.
We’ve made tweaks to this model, too, of course. For example: we’ve set a “target” residual and, where we spend little enough in a month that we would each be eligible for more than that, we instead sweep the excess into our family savings account. It’s a nice approach to help build up a savings reserve without feeling a pinch.
I’m sure our model will continue to evolve, as it has for the last decade and a half, but for now it seems stable, fair, and reasonable. Maybe it’ll work for your household too (whether or not you’re also a polyamorous family!): take a look at the spreadsheet in Google Drive and give it a go.
Semi-inspired by a similar project by Kev Quirk, I’ve got a project I want to run on my blog in 2024.
I want you to be my pen pal for a month. Get in touch by emailing penpals@danq.me or any other way you like and let’s do this!
I don’t know much about the people who read my blog, whether they’re ad-hoc visitors or regular followers1.
So here’s the plan: I’m looking to do is to fill a “dance card” of interesting people each of with whom I’ll “pen pal” for a month.
The following month, I’ll blog about the experience: who I met, what I learned about them, what I learned about myself. Have a look below and see if there’s a slot for you: I’d love to chat to you about, well – anything!
Want in? Leave a
comment, at-me on the Fediverse @dan@danq.me, fill my
contact form, or just email penpals@danq.me. Okay; looks like I’ve got a full year of people
to meet! Awesome!
| Penpal with… | …during… | …and blog in: | Notes: |
|---|---|---|---|
| Colin Walker | December 2023 | January 2024 | Colin’s announcement |
| Thom Denholm | January 2024 | February 2024 | |
| Ru | February 2024 | March 2024 | |
| Dr. Alex Bowyer | March 2024 | April 2024 | Agreement via LinkedIn |
| Roslyn Cook | April 2024 | May 2024 | |
| Garrett Coakley | May 2024 | June 2024 | |
| Derek Kedziora | June 2024 | July 2024 | |
| Aarón Fas | July 2024 | August 2024 | |
| Cal Desmond-Pearson | August 2024 | September 2024 | |
| Tyoma | September 2024 | October 2024 | |
| Farai | October 2024 | November 2024 | |
| Katie | November 2024 | December 2024 | Katie’s comment |
I’ll update this table as people get in touch.
You! If you’re reading this, you’re probably somebody I want to meet! But I’d be especially interested in penpalling with people who tick one or more of the following boxes:
If you read this far and didn’t email penpals@danq.me yet, go do that. I’m looking forward to hearing from you!
1 Not-knowing who reads my blog might come at least in part from the fact that I actively sabotage any plugin that might give me any analytics! One might say I’ve shot myself in the foot, there.
2 If we stay in touch afterwards that’s fine too, but it’s not essential.
3 I’m looking for longer-form, but slower, communication than you get via e.g. instant messengers and whatnot: a more “penpal” experience.