Geohashing.site's Better Banhammer

A blunt instrument

Yesterday, Terence Eden shared the technique he's been using to make a PHP-powered honeypot that bans badly-behaved bots that disregard his robots.txt file. The essence of his technique is this:

1. A URL is explicitly disallowed in robots.txt.
2. A disallowed link, hidden to humans, appears on web pages.
3. Anybody visiting the link has their IP address (carefully) written to a .htaccess file.
4. His (Apache) webserver uses this .htaccess file as a blocklist.

There are a few problems with this approach, but I picked up on the potential for abuse: an attacker who derives the "if you visit this then you get banned" URL can easily trick legitimate users into visiting it. I've demonstrated this potential for abuse before: an asset load, a redirect, a form submission, a link, an iframe... any of these and more besides can be used to turn this kind of honeypot into a denial-of-service attack against legitimate users of the site.

A sharper instrument

Here's my solution: one extra step, that ensures that the person you're banning is actually the person to whom you originally sent the honeypot link:

1. A URL is explicitly disallowed in robots.txt.
2. A disallowed link, hidden to humans, appears on web pages. The link contains a signed token that encodes the user's IP address.
3. Anybody visiting the link has the signed token checked to see if it matches their current IP before it's written to a log.
4. The log is used to feed the blocklist (I use Fail2Ban because it's much better-suited to the task).

My real-world implementation

I've been doing this for-real on Geohashing.site shortly after I upgraded its version of MediaWiki (among other changes). Here's how I do it. I'm using Caddy as my webserver, but that's irrelevant to my implementation.

First, I installed Fail2Ban and nftables and set up a file, /var/log/honeypot/honeypot.log to contain my "hits":

sudo apt install fail2ban nftables
sudo systemctl enable --now nftables fail2ban

Here's my configuration. I have a /etc/fail2ban/filter.d/php-honeypot.confthat describes the format of the logs I'll create:

[Definition]
failregex = ^\s*HONEYPOT <ADDR> \S+$
ignoreregex =

And a /etc/fail2ban/jail.d/php-honeypot.local that says what to do when something hits it:

[php-honeypot]
enabled   = true
filter    = php-honeypot
backend   = polling
findtime  = 1h
# Here's where my log is kept:
logpath   = /var/log/honeypot/honeypot.log
# A single "hit" is enough to trigger a ban:
maxretry  = 1
# A ban lasts for 24 hours:
bantime   = 24h
# I ban EVERYTHING, not just HTTP(S):
banaction = nftables[type=allports]
# I ban TCP _and_ UDP because I support HTTP/3 (QUIC);
# optionally drop this line if you only do HTTP/2 and below:
protocol  = tcp, udp
# I allowlist a few IPs so they never get blocked; 1.2.3.4 is an example:
ignoreip  = 127.0.0.1/8 ::1 1.2.3.4

I added a /etc/logrotate.d/honeypot to rotate my honeypot logs so they don't get out of control:

/var/log/honeypot/honeypot.log {
  weekly
  rotate 4
  compress
  missingok
  notifempty
  create 0640 www-data adm
}

I use almost-exactly the code I shared yesterday. I declare a robots.txt Disallow rule for a directory, let's call it /honeypot/, and put this file in it. Then I add secret links (hidden from screenreaders with aria-hidden, from keyboard navigation with tabindex="-1", and from everybody else using CSS) and  My honeypot_hit() function looks like this:

/**
 * Applies the banhammer to the current user.
 */
function honeypot_hit(){
  $ip = $_SERVER['REMOTE_ADDR'];
  if(in_array($ip, HONEYPOT_NEVER_BAN)) return; // Return immediately if this IP is on the never-ban list
  $line = sprintf("%s HONEYPOT %s honeypot-hit\n", date('c'), $ip);
  file_put_contents('/var/log/honeypot/honeypot.log', $line, FILE_APPEND | LOCK_EX);
}

The only other change I make from the code I shared yesterday is to tweak the HTML page displayed at the end to make it look more like a "real" page (to catch the AI bots) and to add text that indicates that humans should follow a link back to the homepage rather than the "trapped" one... just in case anybody gets "tricked" into visiting the honeypot's page. That's the safety-net that my implementation provides!

I don't remember if you need to sudo systemctl restart fail2ban after changing the configuration, but I did and mine works. Maybe just do it.

I don't currently bother adding "hot"/armed honeypot links on the site itself, but for uncached pages it'd be easy enough to do so (MediaWiki already depends on PHP, after all!). So it's always two steps to get yourself banned... but it's two steps that badly-behaved bots keep falling for!

If you implement the above on your own site, you also ought to note that:

• sudo fail2ban-client status php-honeypot will list the stats e.g. how many IPs are banned
• sudo fail2ban-client set php-honeypot unbanip 1.2.3.4 will un-ban an IP; this is useful when you're testing (e.g. from your mobile) to check that the ban works as you'd expect

Of course, there's nothing to stop you expanding upon this if you want a mixture of "hot" (instant-ban) and "warm" honeypots: a couple of PHP tweaks and maybe Fail2Ban changes and you could e.g. do a 5-minute ban for anybody hitting common scraper URLs like /wp-admin/... and a much-longer ban for anybody who drills all the way through to the honeypot page and still continues on! Tweak it to your own preference!