Importing Geocaching Logs into WordPress

Background

As an ocassional geocacher and geohasher, I’m encouraged to post logs describing my adventures, and each major provider wants me to post my logs into their silo (see e.g. my logs on geocaching.com, on opencache.uk, and on the geohashing wiki). But as a believer in the ideals behind the IndieWeb (since long before anybody said “IndieWeb”), I’m opposed to keeping the only copy of content that I produce in an environment controlled by somebody else (why?).

How do I reconcile this?

Wrist-mounted GPS in the snow.
Just another hundred metres to the cache, then it’s time to freeze my ass back to base.

What I’d prefer would be to be able to write my logs here, on my own blog, and for my content to by syndicated via some process into the logging systems of the various silo sites I prefer. This approach is called POSSE – Publish on Own Site, Syndicate Elsewhere. In addition to the widely-described benefits of this syndication strategy, such a system would also make it possible for me to:

  • write single posts amalgamating multiple locations (e.g. a geohashing expedition that included geocache finds) or,
  • write single posts that represent the same location published on multiple silos (e.g. a visit to a geocache published on two different listing sites [e.g. 1, 2])

Applying such an tool would require some work as different silos have different acceptable content rules (geocaching.com, for example, effectively forbids mention of the existence of other geocache listing sites), but that’d theoretically be workable.

POSSE would involve posts being made first to my blog and then converted via some process into logs in each relevant silo
The ideal solution would be POSSE-based.

Unfortunately, content rules aren’t the only factor making PESOS – writing content into each silo and then copying it to my blog – preferable to POSSE. There’s also:

  • Not all of the silos offer suitable (published) APIs, and where they do, the APIs are all distinctly different.
  • Geocaching.com specifically forbids the use of unapproved automated robots to access the site (and almost certainly wouldn’t approve the kind of tool that would be ideal).
  • The siloed services are well-supported by official and third-party apps with medium-specific logic which make them the best existing way to produce logs.
PESOS would mean that posts were made "the usual way" to the silos and then a process duplicates them onto my blog
A PESOS-based solution is far easier to implement, in this case.

Needless to say: as much as I’d have loved to POSSE my geo* logs, PESOS will do.

Implementation

My implementation is a WordPress plugin which does two things. The first is that it provides a Javascript bookmarklet and an accompanying dynamically-generated Javascript file (the former loads the latter) served from my blog’s domain. That Javascript file contains reference to every log already published to my blog, so that the Javascript code can deliberately omit these logs from any import. When executed on a log listing page like those linked above, it copies all of the details of that log into a form which submits them back to my blog, where it’s received by the second part of the plugin.

Geocache logs to WordPress importer seen running on geocaching.com
The import controls appear in a new, right-most column (GCVote is also visible running in my browser).

The second part of the plugin takes this data and creates a new draft post. My plugin is pretty opinionated on this part because it’s geared strongly towards my use-case, so if you want to use it yourself you’ll probably want to tweak the code a little (e.g. it applies specific tags and names metadata fields a particular way).

Import plugin running on OpenCache.uk
When run on OpenCache.uk effectively the same interface is presented, even though the underlying mechanisms and data locations are different.

It’s not fully-automated and it’s not POSSE,but it’s “good enough” and it’s enabled me to synchronise all of my cache logs to my blog. I’ve plans to extend it to support other GPS game services to streamline my de-siloisation even further.

And of course, I’ve open-sourced the whole thing. If it’s any use to you (probably in an adapted form), it’s all yours.

× × ×

Bypassing WordPress / Jetpack’s “Prove your humanity:” CAPTCHA

One of the most-popular WordPress plugins is Jetpack, a product of Automattic (best-known for providing the widely-used WordPress hosting service “WordPress.com“). Among Jetpack’s features (many of which are very good) is Jetpack Protect which adds – among other things – the possibility for a CAPTCHA to appear on your login pages. This feature is slightly worse than pointless as it makes it harder for humans to log in but has no significant impact upon automated robots; at best, it provides a false sense of security and merely frustrates and slows down legitimate human editors.

WordPress/Jetpack's CAPTCHA, asking for the solution to "9+10="
Thanks, WordPress, for slowing me down with a CAPTCHA that a robot can solve more-easily than a human.

“Proving your humanity”, as you’re asked to do, is a task that’s significantly easier for a robot to perform than a human. Eventually, of course, all tests of this nature seem likely to fail as robots become smarter than humans (especially as the most-popular system is specifically geared towards training robots), but that’s hardly an excuse for inventing a system that was a failure from its inception. Jetpack’s approach is fundamentally flawed because it makes absolutely no effort to disguise the challenge in a way that humans are able to read any-differently than robots. I’ll demonstrate that in a moment.

Jetpack security settings: "Protect" switch
Don’t just disable this, though! Other “Protect” features make sense. If only you could disable just the one that doesn’t…

A while back, a colleague of mine network-enabled Jetpack Protect across a handful of websites that I occasionally need to log into, and it bugged me that it ‘broke’ my password safe’s ability to automatically log me in. So to streamline my workflow – as well as to demonstrate quite how broken Jetpack Protect’s CAPTCHA is, I’ve written a userscript that you can install into your web browser that will completely circumvent it, solving the maths problems on your behalf so that you don’t have to. Here’s how to use it:

  1. Install a userscript manager into your browser if you don’t have one already: I use Tampermonkey, but it ought to work with almost any of them.
  2. Install Jetpack Maths Solver.

From now on, whenever you go to a page whose web path begins with “/wp-login.php” that contains a Jetpack Protect maths problem, the answer will be automatically calculated and filled-in on your behalf. The usual userscript rules apply: if you don’t trust me, read the source code (there are really only five lines to check) and disable automatic updates for it (especially as it operates across all domains), and feel free to adapt/improve however you see fit. Maybe if we can get enough people using it Automattic will fix this half-hearted CAPTCHA – or at least give us a switch to disable it in the first place.

Update: 15 October 2018 – the latest version of Jetpack makes an insignificant change to this CAPTCHA; version 1.2 of this script (linked above) works around the change.

OpenID For WordPress

Update: 12th October 2007 – this project is to be considered abandoned. Please see How To Set Up OpenID For WordPress Comments instead. Thanks for the support and for your interest in OpenID.

THIS IS ALL HORRIBLY OUT OF DATE. THE DOWNLOAD LINKS DON’T WORK, I KNOW. GET OVER IT. More seriously now, I am working on a new version of this that actually works as a WordPress 2.0.x plugin. It’s very nice, but it’s not finished. Watch this space. In the meantime, why not take a look at OpenID Comments For WordPress (which is based on my preliminary work, here). Thanks for all the attention, guys.

As promised, I’m releasing the first usable version (v0.4) of my WordPress OpenID plugin tool. It’s very, very messy and a little buggy. Plus, installing it requires that you hack a few PHP files… use at your own risk. You’ll need a WordPress v1.5 weblog. Download this package and decompress it to your WordPress directory. It will create an openid_icons directory, a file called openid.php (the main codebase), and a file called openidform.php (the form that appears on your blog). Edit openid.php and substitute your own weblog URL in at the appropriate places (near the top). Link in the login form wherever you like. I’ve done so in my theme’s “sidebar.php” file, with the following code: <?php include (TEMPLATEPATH . '/openidform.php'); ?> In your main index.php, add a line to include the openid.php file. This will allow logins and logouts to be processed. Something like this: <?php require_once('openid.php'); ?> In wp-comments-post.php (the comments processor), substitute the following code in under “// If the user is logged in”: // If the user is logged in get_currentuserinfo(); if ( $user_ID ) { $comment_author = addslashes($user_identity); $comment_author_email = addslashes($user_email); $comment_author_url = addslashes($user_url); } elseif ($_SESSION['sess_openid_auth_code'] != "") { $comment_author = addslashes($_SESSION['sess_openid_auth_code']); $comment_author_email = "openid@example.com"; $comment_author_url = addslashes($_SESSION['sess_openid_auth']); } else { if ( get_option('comment_registration') ) die( __('Sorry, you must be logged in to post a comment.') ); } Notice the extra section, relying upon $_SESSION[‘sess_openid_auth_code’]. That’s the magic bit. And it should ‘just work’. Let me know if it doesn’t; I’ll be improving the codebase over the coming weeks and I’d like to include your suggestions. If you need any help setting it up, I can probably help with that too, or even with adapting the code to work with other applications (than WordPress). Features so far:

  • Authenticate OpenID users
  • Easily authenticate OpenID users from particular servers, including members of LiveJournal, DeadJournal, and Level9
  • Authenticated OpenID users can post comments

Features to come:

  • Cookie-based “remember me”
  • Ability to authenticate WordPress users (e.g. the weblog owner) by an OpenID
  • “Friends Only” protected posts, which can only be read by certain authenticated users
  • AJAX-powered log-in (to save users from having their browsers redirected excessively, and because it can be made to look swish), where supported

If you want to help code, just drop me a message.

OpenID And Scatmania

Over the last few weeks I’ve playing playing with an exciting new technology known as OpenID. Do you remember Microsoft Passport and it’s opposite number, Liberty Alliance? Well; we all know that these services weren’t all they cracked up to be. They claimed to be “distributed log-on services”, but in actual fact they were centralised log-on services (controlled, for example – in the case of Passport – by Microsoft – do you want Microsoft to know everything you do on the web?), and not really distributed at all…

…OpenID really is a distributed log-on service. Anybody can set up an OpenID server and start giving out OpenID accounts. If you have a weblog with LiveJournal, for example, you already have one, and soon folks on other similar blogging services will have them too.

I’d love to see a future where OpenID catches on, because it really is a beautiful and elegant (from a technical point of view) way of doing things, and it’s really easy to use from a user’s point of view, too. I’ve spent a little while implementing the beginnings of a WordPress (the blogging engine that powers this site) plug-in, and it’s taking shape: if you look in the upper-right of the page, you should find that you’re able to log in to this web site using your LiveJournal account. That means that WordPress users like myself, in future, should be able to do things like LiveJournal’s “friends only” posts, and allow LiveJournal users to make comments in a way that proves they are who they say they are, and many other benefits, too.

But, of course, it doesn’t stop there: DeadJournal will be next. Then TypePad. Then Blogger and the forum sites – phpBB and the like. Then the wiki sites. All of these sites will be able to authenticate against one another, and make content private, or accessible, without having to have silly “sign up” systems of the type we’re starting to see everywhere these days.

It’s all very exciting, but it’s early days for now. Right now, my WordPress plugin doesn’t do a lot – you can log in and out, and that’s about it. But give me a go, and tell me what you think – log in to my blog using your LiveJournal account, and give me some feedback. And when I finally get this code to a production level (right now it’s buggy as hell), I’ll release it as a WordPress plugin, and the world will be great.

Blogspam A Problem… No More

As I’ve mentioned in previous posts, I’ve been getting more than my fair share of blogspam of late. I’ve been spending about twenty minutes every three or so days clearing out the ‘moderation’ queue and updating my keyword lists. Worse still, some spam has been getting through nonetheless (hopefully I’ve always been quick to remove it, and so none of you – my readers – have had to see any of it).

So: I’ve implemented a new anti-blogspam solution: whenever you post a comment to my weblog from now on you’ll be asked a simple question. The answer is usually obvious… to a human… but very difficult to automate a computer to answer. I appreciate any feedback on this (why not leave a comment to this post), and I’ll let you know whether it fixes the problem. And, of course, if it does, I’ll offer my code snippet back to the WordPress development team in order to include it, perhaps, with a future version: or, at least, offer it to friends of mine who use similar blog engines and are troubled by spam.

I need sleep.

In other (almost equally geeky) news, I’ve been spending a good deal of time working on my new RockMonkey WikiGameTromaNightAdventure. If I can keep up a reasonable development rate on it this weekend (which could be tough – I’ve lots to do, and Gareth is visiting and keeps distracting me with cool technology like GPS devices and VoIP telephones), it’ll be ready on Tuesday evening. Watch this space.