Roll Your Own Antispam

Three Rings operates a Web contact form to help people get in touch with us: the idea is that it provides a quick and easy way to reach out if you’re a charity who might be able to make use of the system, a user who’s having difficulty with the features of the software, or maybe a potential new volunteer willing to give your time to the project.

But then the volume of spam it received increased dramatically. We don’t want our support team volunteers to spend all their time categorising spam: even if it doesn’t take long, it’s demoralising. So what could we do?

Clearly-spammy message shown in a ticket management system.
It’s clearly spam, but if it takes you 2 seconds to categorise it and there are 30 in your Inbox, that’s still a drag.

Our conventional antispam tools are configured pretty liberally: we don’t want to reject a contact from a legitimate user just because their message hits lots of scammy keywords (e.g. if a user’s having difficulty logging in and has copy-pasted all of the error messages they received, that can look a lot like a password reset spoofing scam to a spam filter). And we don’t want to add a CAPTCHA, because not only do those create a barrier to humans – while not necessarily reducing spam very much, nowadays – they’re often terrible for accessibility, privacy, or both.

But it didn’t take much analysis to spot some patterns unique to our contact form and the questions it asks that might provide an opportunity. For example, we discovered that spam messages would more-often-than-average:

  • Fill in both the “name” and (optional) “Three Rings username” field with the same value. While it’s cetainly possible for Three Rings users to have a login username that’s identical to their name, it’s very rare. But automated form-fillers seem to disproportionately pair-up these two fields.
  • Fill the phone number field with a known-fake phone number or a non-internationalised phone number from a country in which we currently support no charities. Legitimate non-UK contacts tend to put international-format phone numbers into this optional field, if they fill it at all. Spammers often put NANP (North American Numbering Plan) numbers.
  • Include many links in the body of the message. A few links, especially if they’re to our services (e.g. when people are asking for help) is not-uncommon in legitimate messages. Many links, few of which point to our servers, almost certainly means spam.
  • Choose the first option for the choose -one question “how can we help you?” Of course real humans sometimes pick this option too, but spammers almost always choose it.

None of these characteristics alone, or any of the half dozen or so others we analysed (including invisible checks like honeypots and IP-based geofencing), are reason to suspect a message of being spam. But taken together, they’re almost a sure thing.

To begin with, we assigned scores to each characteristic and automated the tagging of messages in our ticketing system with these scores. At this point, we didn’t do anything to block such messages: we were just collecting data. Over time, this allowed us to find a safe “threshold” score above which a message was certainly spam.

Three Rings contact form filled by Spammy McSpamface, showing a 'Security Checks Failed' error message and tips on refining the message.
Even when a message fails our customised spam checks, we only ‘soft-block’ it: telling the user their message was rejected and providing suggestions on working around that or emailing us conventionally. Our experience shows that the spammers aren’t willing to work to overcome this additional hurdle, but on the very rare ocassion a human hits them, they are.

Once we’d found our threshold we were able to engage a soft-block of submissions that exceeded it, and immediately the volume of spam making it to the ticketing system dropped considerably. Under 70 lines of PHP code (which sadly I can’t share with you) and we reduced our spam rate by over 80% while having, as far as we can see, no impact on the false-positive rate.

Where conventional antispam solutions weren’t quite cutting it, implementing a few rules specific to our particular use-case made all the difference. Sometimes you’ve just got to roll your sleeves up and look at the actual data you do/don’t want, and adapt your filters accordingly.

× ×

Dan Q found GC4NX9Y Light Pyramid (MK Artwalks)

This checkin to GC4NX9Y Light Pyramid (MK Artwalks) reflects a geocaching.com log entry. See more of Dan's cache logs.

Found the host easily, but had to wait for a gap in thir lunchtime dog walkers to be able to mount a good search. After checking in a few obvious places I picked something up and there was the cache!

Took a walk up to the Light Pyramid where I snapped the attached photo of me pointing towards the X-Scape centre, where I’ve been working today (my kids have ski lessons, so I’ve been sitting in the cafe with my laptop with the exception of this, my lunch break!).

On which note, I’d better go find myself a sandwich! Thanks for bringing me up here, and TFTC.

Dan, near a white pyramid-shaped sculpture, points at a distant hill-shaped building.

×

2024 in Videogames

Duration

Podcast Version

This post is also available as a podcast. Listen here, download for later, or subscribe wherever you consume podcasts.

My life affords me less time for videogames than it used to, and so my tastes have changed accordingly:

  • I appreciate games that I can drop at a moment’s notice and pick up again some other time, without losing lots of progress1.
  • And if the game can remind me what it was I was trying to achieve when I come back… perhaps weeks or months later… that’s a bonus!
  • I’ve a reduced tolerance for dynamically-generated content (oh, you want me to fetch you another five nirnroot do you? – hard pass2): if I might only get to throw 20 hours total at a game, I’d much prefer to spend that time exploring content deliberately and thoughtfully authored by a human.
  • And, y’know, it has to be fun. I rarely buy games on impulse anymore, and usually wait weeks or months after release dates even for titles I’ve been anticipating, to see what the reviewers make of it.

That said, I’ve played three excellent videogames this year that I’d like to recommend to you (no spoilers):


Horizon: Forbidden West

I loved Horizon: Zero Dawn. Even if this review persuades you that you should play its sequel, Forbidden West, you really oughta play Zero Dawn first3. There’s a direct continuation of plot going on there that you’ll appreciate better that way. Also: Zero Dawn stands alone as a great game in its own right.

Aloy, the protagonist of the Horizon games, wearing Mark of Pride face paint and red-stained Quen Deadeye armour, stands at sunset in a jungle environment.
Horizon gives a lot to love, from a rich world and story, immersive environments, near-seamless loading, excellent voice acting, and a rewarding difficulty curve. But perhaps all are second-place to what a kickass character the protagonist is.

The Horizon series tells the story of Aloy from her childhood onwards, growing up an outcast in a tribal society on a future Earth inhabited by robotic reimaginings of creatures familiar to us today (albeit some of them extinct). Once relatively docile, a mysterious event known as the derangement, shortly before Aloy’s birth, made these machines aggressive and dangerous, leading to a hostile world in which Aloy seeks to prove herself a worthy hunter to the tribe that cast her out.

All of which leads to a series of adventures that gradually explain the nature of the world and how it became that way, and provide a path by which Aloy can perhaps provide a brighter future for humankind. It’s well-written and clever and you’ll fight and die over and over as you learn your way around the countless permutations of weapons, tools, traps, and strategies that you’ll employ. But it’s the kind of learning curve that’s more rewarding than frustrating, and there are so many paths to victory that when I watch Ruth play she uses tactics that I’d never even conceived of.

Aloy aims a precision longbow at a Tremortusk, an elephant-like machine, in a sunny desert environment.
Horizon: Forbidden West is like Zero Dawn but… more. More quests, more exploration, more machines, more characters, and more of the same story, answering questions you might have found yourself thinking during the prequel. But it’s not just more-of-the-same.

Forbidden West is in some ways more-of-the-same, but it outgrows the mould of its predecessor, too. Faced with bigger challenges than she can take on by herself, Aloy comes to assemble a team of trusted party members, and when you’re not out fighting giant robots or spelunking underwater caves or exploring the ruins of ancient San Francisco you’re working alongside them, and that’s one of the places the game really shines. Your associates chatter to each other, grow and change, and each brings something special to the story that invites you to care for each of them as individuals.

The musical score – cinematic in its scope – has been revamped too, and shows off its ability to adapt dynamically to different situations. Face off against one of the terrifying new aquatic enemies and you’ll be treated to a nautical theme, for example. And the formulaic quests of the predecessor (get to the place, climb the thing…), which were already fine, are riddled with new quirks and complexities to keep you thinking.

And finally: I love the game’s commitment to demonstrating the diversity of humanity: both speaking and background characters express a rarely-seen mixture of races, genders, and sexualities, and the story sensitively and compassionately touches on issues of disability, neurodiversity, and transgender identity. It’s more presence than representation (“Hey look, it’s Sappho and her friend!”), but it’s still much better than I’m used to seeing in major video game releases.

Thank Goodness You’re Here!

If ever I need to explain to an American colleague why that one time they visited London does not give them an understanding of what life is like in the North of England… this is the videogame I’ll point them at.

Main menu for Thank Goodness You're Here, featuring options "Gu On Then", "Faff", and "Si' Thi", superimposed on a picture of a street in Barnsley, Yorkshire.
Among the many language options available for the game are “English”, as you’d probably expect, and “Dialect”, which imposes a South Yorkshire accent to everything, as illustrated here by the main menu.

A short, somewhat minigame-driven, absurd to the point of Monty Python-ism, wildly British comedy game, Thank Goodness You’re Here! is a gem. It’s not challenging by any stretch of the imagination, but that only serves to turn focus even more on the weird and wonderful game world of Barnsworth (itself clearly inspired by real-world Barnsley).

Playing a salesman sent to the town to meet the lord mayor, the player ends up stuck with nothing to do4, and takes on a couple of dozen odd-jobs for the inhabitants of the town, meeting a mixed bag of stereotypes and tropes as they go along.

Hand-drawn advertisement for Big Ron's Big Pies (Barnsworth's Best since 1904).
Ahm gowin t’shop to gi’ sumof Big Ron’s Big Pies! Y’wanout, buggerlugs? Players without a grounding in Yorkshire English, and especially non-Brits, might benefit from turning the subtitles on.

Presented in a hand-drawn style that’s as distinctive and bizarre as it is an expression of the effort that must’ve gone into it, this game’s clearly a project of passion for Yorkshire-based developers Coal Supper (yes, that’s really what they call themselves). I particularly enjoyed a recurring joke in which the player is performing some chore (mowing grass for the park keeper, chopping spuds at the chippy) when the scene cuts to some typically-inanimate objects having a conversation (flowers, potatoes) while the player’s actions bring them closer and closer in the background. But it’s hard to pick out a very favourite part from this wonderful, crazy, self-aware slice of Northern life in game form.

Tactical Breach Wizards

Finally, I’ve got to sing the praises of Tactical Breach Wizards by Suspicious Developments (who for some reason don’t bother to list it on their website; the closest thing to an official page for the project other than its Steam entry might be this launch announcement!)5, the team behind Gunpoint and Heat Signature.

The game feels like a cross between XCOM/Xenonauts‘ turn-based tactical combat and Rainbow Six‘s special ops theme. Except instead of a squad of gun-toting body-armoured military/police types, your squad is a team of wizards in a world in which magical combat specialists work alongside conventionally-equipped soldiers on missions where their powers make all the difference.

Jen, the Storm Witch, throws a bolt of lightning through three enemies on a moving train carriage.
Jen the Storm Witch primarily uses large static shocks to fling targets around: relatively harmless, unless she and her teammates have arranged for/tricked enemies to be standing next to something they can be thrown into… or near a window they can be flung out of!

By itself, that could be enough: there’s certainly sufficient differences between all of the powers that the magic users exploit that you’ll find all kinds of ways to combine them. How about having your teleport-capable medic blink themselves to a corner so your witch’s multi-step lightning bolt can use them as a channel to get around a corner and zap a target there? Or what about using the time-manipulation powers of your Navy Seer (yes, really) to give your siege cleric enough actions that they can shield-push your opponent within range of the turret you hacked? And so on.

But Tactical Breach Wizards, which stands somewhere between a tactical squad-based shooter and a deterministic positional puzzle game, goes beyond that by virtue of its storytelling. Despite the limitations of the format, the game manages to pack in a lot of background and personality for every one of your team and even many of the NPCs too (Steve Clark, Traffic Warlock is a riot). Oh, and much of the dialogue is laugh-out-loud funny, to boot.

Three spec ops wizards have a conversation about an upcoming assault.
The dialogue between your teammates – most of it right as they’re about to breach a door – reads like lighthearted banter but exposes the underpinning backstory of the setting.

The writing’s great, to the extent that when I got to the epilogue – interactive segments during the credits where you can influence “what happens next” to each of the characters you’ve come to know – I genuinely flip-flopped on a few of them to give some of them a greater opportunity to continue to feature in one another’s lives. Even though the game was clearly over. It’s that compelling.

And puzzling out some of the tougher levels, especially if you’re going for the advanced (“Confidence”) challenges, too, is really fun. But with autosaves every turn, the opportunity to skip and return to levels that are too challenging, and a within-turn “undo” feature that lets you explore different strategies before you commit to one, this is a great game for someone who, like me, doesn’t have much time to dedicate to play.


So yeah: that’s what I’ve been up to in videogaming-time so far this year. Any suggestions for the autumn/winter?

Footnotes

1 If a game loads quickly that’s a bonus. I still play a little of my favourite variant of the Sid Meier’s Civilization series – that is, Civilization V + Vox Populi (alongside a few quality-of-life mods) but I swear I’d play more of it if it didn’t take so long to load. Even after hacking around it to dodge the launcher, logos, and introduction, my 8P+4E-core i7 processor takes ~80 seconds from clicking to launch the game to having loaded my latest save, which if I’m only going to have time to play three turns is frustratingly long! Contrast Horizon: Forbidden West, which I also mention in this post, a game 13 years younger and with much higher hardware requirements, which takes ~17 seconds to achieve the same. Possibly I’m overanalysing this…

2 This isn’t a criticism of the Elder Scrolls games specifically, but of the relatively-lazy writing that goes into some videogames that feel like they’re using Perchance to come up with their quests, in order to stretch the gameplay. I suppose a better example might have been the on-the-whole disappointment that was Starfield, but I figured an Elder Scrolls reference might be easier to identify at-a-glance. Fetch-questing 100 tonnes of Beryllium just doesn’t have the same ring to it.

3 In fact, if you’re trying to consume the Horizon story as thoroughly as possible and strictly in chronological order, you probably should read the graphic novel between one and the other, which covers some of the events that occur between the two.

4 Did you ever see the alternate ending to Far Cry 4, by the way? If you did, you might appreciate that a similar trick can be used to shortcut Thank Goodness You’re Here! too…

5 They’re also missing a trick by using the domain they’ve registered, wizards.cool, only to redirect to Steam.

× × × × × ×

Note #24342

If the most useful thing I achieve this Bank Holiday Monday will have been to make it easier to post short geotagged notes from my mobile to my blog (and Mastodon), it will have been a success.

Dan sits on the grass in a garden, with his French Bulldog.

This has been a test post. Feel free to ignore it.

×

Dan Q found GC4PYCF Thames Path – Floodproof

This checkin to GC4PYCF Thames Path - Floodproof reflects a geocaching.com log entry. See more of Dan's cache logs.

Excellent cache, which I was pleased to observe has the largest conceivable container possible for its hiding place: nice one! I love a good treetop cache!

Once I’d free the right tree, getting up was relatively easy: the limb next over from the one mentioned in the hint provided a good launching-off point and a short scramble later I was sat at height with the container in hand. Getting down, though, proved more challenging as I slipped on a low bough and plummeted to the ground!

Dan, up a tree, holds a geocache secured high up the trunk.

Aside from my pride, the biggest injury was to my thumb, which nicked some kind of fierce plant on the way down and is bleeding as I type this. Still 100% a worthwhile effort to find a great cache, so an FP awarded.

A hand with a small chunk torn out of the back of the thumb, exposing bright red flesh beneath.

Now I’ve gotta start jogging again if I’m to have any chance of catching up to my partner Ruth, who I’ve joined in this leg of her effort to walk the entire Thames Path (I swear I didn’t just agree to tag along for the caching opportunities!).

× ×

Dan Q wrote note for GC4QAED Leonard

This checkin to GC4QAED Leonard reflects a geocaching.com log entry. See more of Dan's cache logs.

Skipped this one as I’m still playing catchup to the rest of my group and wasn’t interested in a long search. Many previous finders note that the coordinates are significantly off but I couldn’t see anybody posting alternates. Maybe next time I’m down here!

Dan Q found GC4QAE6 Bernadette

This checkin to GC4QAE6 Bernadette reflects a geocaching.com log entry. See more of Dan's cache logs.

After a brief overshoot – too excited to finally be catching up to Ruth and the rest of my squad! – doubled back to find this easy location. Cache was lying on the floor which I assume isn’t the right hiding place, so I returned it to the V. While running from the last cache I’ve dropped my writing implement somewhere, so have photographed the (almost pristine!) logbook as proof that I actually found it. This has been my favourite of this mini-series so far; FP awarded for the enjoyable container theming if nothing else!

A replacement geocache logbook with only a single entry.

×

Dan Q did not find GC4QADB Sheldon

This checkin to GC4QADB Sheldon reflects a geocaching.com log entry. See more of Dan's cache logs.

No luck here. Couldn’t spare more than a little while to hunt without Ruth and the rest of the Thames Path Source-to-Mouth party getting too far ahead but spent that time getting thoroughly nettled. They’re fierce around here! Maybe another time.

Dan Q found GC2W97E Messing about by the river

This checkin to GC2W97E Messing about by the river reflects a geocaching.com log entry. See more of Dan's cache logs.

Found after a brief search while accompanying my partner Ruth on the latest leg of her effort to (by instalments) walk the entire length of the Thames from its source to its mouth. Initially looked in the wrong place but once I was willing to brave the nettles and hack my way off the footpath the cache location became obvious. A bit damp, but serviceable. TFTC!

Dan, on an overgrown riverside footpath, waves to the camera.

×

VaultPress to the Rescue

OMG VaultPress Jetpack Backup to the rescue.

One of the best Internet people drew me a picture and when I replied to it, it got scrambled. 😱

But even though I had to modify core WordPress columns to store drawings, the backup respected that and I was able to restore it.

I used to pay for VaultPress. Nowadays I get it for free as one of the many awesome perks of my job. But I’d probably still pay for it because it’s a lifesaver.