The Far Side in FreshRSS

A few yeras ago, I wanted to subscribe to The Far Side‘s “Daily Dose” via my RSS reader. The Far Side doesn’t have an RSS feed, so I implemented a proxy/middleware to bridge the two.

Browser debugger running document.evaluate('//li[@class="blog__post-preview"]', document).iterateNext() on Beverley's weblog and getting the first blog entry.
If you’re looking for a more-general instruction on using XPath scraping in FreshRSS, this isn’t it.
The release of version 1.20.0 of my favourite RSS reader FreshRSS provided a new mechanism for subscribing to content from sites that didn’t provide feeds: XPath scraping. I demonstrated the use of this to subscribe to my friend Beverley‘s blog, but this week I figured it was time to have a go at retiring my middleware and subscribing directly to The Far Side from FreshRSS.

It turns out that FreshRSS’s XPath Scraping is almost enough to achieve exactly what I want. The big problem is that the image server on The Far Side website tries to prevent hotlinking by checking the Referer: header on requests, so we need a proxy to spoof that. I threw together a quick PHP program to act as a proxy (if you don’t have this, you’ll have to click-through to read each comic), then configured my FreshRSS feed as follows:

FreshRSS "HTML + XPath" configuration page, configured as described below.

  • Feed URL: https://www.thefarside.com/
    The “Daily Dose” gets published to The Far Side‘s homepage each day.
  • XPath for finding new items: //div[@class="card tfs-comic js-comic"]
    Finds each comic on the page. This is probably a little over-specific and brittle; I should probably switch to using the contains function at some point. I subsequently have to use parent:: and ancestor:: selectors which is usually a sign that your screen-scraping is suboptimal, but in this case it’s necessary because it’s only at this deep level that we start seeing really specific classes.
  • Item title: concat("Far Side #", parent::div/@data-id)
    The comics don’t have titles (“The one with the cow”?), but these seem to have unique IDs in the data-id attribute of the parent <div>, so I’m using those as a reference.
  • Item content: descendant::div[@class="card-body"]
    Within each item, the <div class="card-body"> contains the comic and its text. The comic itself can’t be loaded this way for two reasons: (1) the <img src="..."> just points to a placeholder (the site uses JavaScript-powered lazy-loading, ugh – the actual source is in the data-src attribute), and (2) as mentioned above, there’s anti-hotlink protection we need to work around.
  • Item link: descendant::input[@data-copy-item]/@value
    Each comic does have a unique link which you can access by clicking the “share” button under it. This makes a hidden text <input> appear, which we can identify by the presence of the data-copy-item attribute. The contents of this textbox is the sharing URL for the comic.
  • Item thumbnail: concat("https://example.com/referer-faker.php?pw=YOUR-SECRET-PASSWORD-GOES-HERE&referer=https://www.thefarside.com/&url=", descendant::div[@class="tfs-comic__image"]/img/@data-src)
    Here’s where I hook into my special proxy server, which spoofs the Referer: header to work around the anti-hotlinking code. If you wanted you might be able to come up with an alternative solution using a custom JavaScript loaded into your FreshRSS instance (there’s a plugin for that!), perhaps to load an iframe of the sharing URL? Or you can host a copy of my proxy server yourself (you can’t use mine, it’s got a password and that password isn’t YOUR-SECRET-PASSWORD-GOES-HERE!)
  • Item date: ancestor::div[@class="tfs-page__full tfs-page__full--md"]/descendant::h3
    There’s nothing associating each comic with the date it appeared in the Daily Dose, so we have to ascend up to the top level of the page to find the date from the heading.
  • Item unique ID: parent::div/@data-id
    Giving FreshRSS a unique ID can help it stop showing duplicates. We use the unique ID we discovered earlier; this way, if the Daily Dose does a re-run of something it already did since I subscribed, I won’t be shown it again. Omit this if you want to see reruns.
Far Side comic #12326, from 23 November 2022, shown in FreshRSS. The comic shows two bulls dressed in trenchcoats and hats browsing a china shop; one staff member says to the other "I got a bad feeling about this, Harriet."
Hurrah; once again I can laugh at repeats of Gary Larson’s best work alongside my other morning feeds.

There’s a moral to this story: when you make your website deliberately hard to consume, fewer people will access it in the way you want! The Far Side‘s website is actively hostile to users (JavaScript lazy-loading, anti-right click scripts, hotlink protection, incorrect MIME types, no feeds etc.), and an inevitable consequence of that is that people like me will find and share workarounds to that hostility.

If you’re ad-supported or collect webstats and want to keep traffic “on your site” on this side of 2004, you should make it as easy as possible for people to subscribe to content. Consider The Oatmeal or Oglaf, for example, which offer RSS feeds that include only a partial thumbnail of each comic and a link through to the full thing. I don’t feel the need to screen-scrape those sites because they’ve given me a subscription option that works, and I routinely click-through to both of them to enjoy their latest content!

Conversely, the Far Side‘s aggressive anti-subscription technology ultimately means that there are fewer actual visitors to their website… because folks like me work to circumvent them.

And now you know how I did so.

Story Time

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

First frame of "Story Time" comic from Channelate. An old man sits in an armchair and talks to two small children sitting on the floor in front of him. The old man says "I ever tel you squirts about the year 2020?"

When this comic (go read the full thing) came out at the tail end of last year, I thought to myself: yeah, that’s about right. I’m resharing that on my birthday in a week or so.

‘Cos I’m forty today, and I sort of had a half-baked dream that I’d throw some kind of big party and get people together. My surprise party for my thirtieth birthday party was an excellent (and much-needed) bash, and I guess I’d thought I’d try to replicate the feel of that, but a decade on (and not a surprise party… although in the end the last one wasn’t either).

But 2020’s the year that keeps on giving, so I’m postponing my party plans to… “some other time”. And so this comic really spoke to me.

Having Kids In Our Poly Triad by Sara Valta

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Having Kids In Our Poly Triad by Sara Valta title showing picture of two men and a woman hugging the outline of a baby

Sara’s back! You might remember a couple of years ago she’d shared with us a comic on her first year in a polyamory! We’re happy to have her back with a slice of life and a frank n’ real conversation about having kids in her Poly Triad relationship.

This sort of wholesome loving chat is just the thing we need for the start of 2021.

Start your year with a delightful comic about the author negotiating possible future children in a queer polyamorous triad, published via Oh Joy Sex Toy. Sara previously published a great polyamory-themed comic via OJST too, which is also worth a look.

The Perfect Art Heist: Hack the Money, Leave the Painting

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Thieves didn’t even bother with a London art gallery’s Constable landscape—and they still walked away with $3 million.

This comic is perhaps the best way to enjoy this news story, which describes the theft of £2.4 million during an unusual… let’s call it an “art heist”… in 2018. It has many the characteristics of the kind of heist you’re thinking about: the bad guys got the money, and nobody gets to see the art. But there’s a twist: the criminals never came anywhere near the painting.

A View from Hampstead Heath, ca. 1825, by John Constable

This theft was committed entirely in cyberspace: the victim was tricked into wiring the money to pay for the painting into the wrong account. The art buyer claims that he made the payment in good faith, though, and that he’s not culpable because it was the seller’s email that must have been hacked. Until it’s resolved, the painting’s not on display, so not only do the criminals have the cash, the painting isn’t on display.

Anyway; go read the comic if you haven’t already.

Lucky Devil Eats

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Lucky Devil Eats

The lockdown’s having an obvious huge impact on strippers, whose work is typically in-person, up close, and classed as non-essential. And their work isn’t eligible for US programmes to support furloughed workers. So Lucky Devil Lounge in Portland decided to adapt their services into one that is classed as essential by providing a drive-through food service. With strippers.

This is Erika Moen’s comic about the experience of visiting the drive-through. Her comics are awesome and I’ve shared them with you a few times before (I even paid for the product she recommended in the last of those), of course.

Difference Between VR and AR

This weekend, my sister Sarah challenged me to define the difference between Virtual Reality and Augmented Reality. And the more I talked about the differences between them, the more I realised that I don’t have a concrete definition, and I don’t think that anybody else does either.

A man wearing a VR headset while writing on a whiteboard.
AR: the man sees simulated reality and, probably, the whiteboard.
VR: the man sees simulated reality only, which may or may not include a whiteboard.
Either way: what the hell is he doing?

After all: from a technical perspective, any fully-immersive AR system – for example a hypothetical future version of the Microsoft Hololens that solves the current edition’s FOV problems – exists in a theoretical superset of any current-generation VR system. That AR augments the reality you can genuinely see, rather than replacing it entirely, becomes irrelevant if that AR system could superimpose a virtual environment covering your entire view. So the argument that compared to VR, AR only covers part of your vision is not a reliable definition of the difference.

Spectrum showing AR view of the Louvre on the left (with 5% of the view occluded by UI) and VR view of a 3D model of the Louvre on the right (with 100% of the view occluded). The middle of the spectrum remains undefined.
The difference by how much of your view is occluded by machine-generated images fails to define where the boundary between AR and VR lies. 5%? 50%? 99%?

This isn’t a new conundrum. Way back in 1994 back when the Sega VR-1 was our idea of cutting edge, Milgram et al. developed a series of metaphorical spectra to describe the relationship between different kinds of “mixed reality” systems. The core difference, they argue, is whether or not the computer-generated content represents a “world” in itself (VR) is just an “overlay” (AR).

But that’s unsatisfying for the same reason as above. The HTC Vive headset can be configured to use its front-facing camera(s) to fade seamlessly from the game world to the real world as the player gets close to the boundaries of their play space. This is a safety feature, but it doesn’t have to be: there’s no reason that a HTC Vive couldn’t be adapted to function as what Milgram would describe as a “class 4” device, which is functionally the same as a headset-mounted AR device. So what’s the difference?

You might argue that the difference between AR and VR is content-based: that is, it’s the thing that you’re expected to focus on that dictates which is which. If you’re expected to look at the “real world”, it’s an augmentation, and if not then it’s a virtualisation. But that approach fails to describe Google’s tech demo of putting artefacts in your living room via augmented reality (which I’ve written about before), because your focus is expected to be on the artefact rather than the “real world” around it. The real world only exists to help with the interpretation of scale: it’s not what the experience is about and your countertop is as valid a real world target as the Louvre: Google doesn’t care.

Milgram et al. (1994)'s Table 1: Some major differences between classes of Mixed Reality (MR) displays, showing seven classes: 1. monitor-based video with CG overlays, 2. headset-based video with CG overlays, 3. headset-based see-through display with CG overlays, 4. headset-based see-through video with CG overlays, 5. monitor/CG world with video overlays, 6. headset-based CG world with video overlays, and 7. CG-based world with real object intervention.
Categories 3 and 4 would probably be used to describe most contemporary AR; categories 6 and 7 to describe contemporary VR.

Many researchers echo Milgram’s idea that what turns AR into VR is when the computer-generated content completely covers your vision.

But even if we accept this explanation, the definition gets muddied by the wider field of “extended reality” (XR). Originally an umbrella term to cover both AR and VR (and “MR“, if you believe that’s a separate and independent thing), XR gets used to describe interactive experiences that cover other senses, too. If I play a VR game with real-world “props” that I can pick up and move around, but that appear differently in my vision, am I not “augmenting” reality? Is my experience, therefore, more or less “VR” than if the interactive objects exist only on my screen? What about if – as in a recent VR escape room I attended – the experience is enhanced by fans to simulate the movement of air around you? What about smell? (You know already that somebody’s working on bridging virtual reality with Smell-O-Vision.)

Dan and Robin concluding a VR Escape Room.
Not sure if you’re real or if you’re dreaming? Don’t ask an XR researcher; they don’t know either.

Increasingly, then, I’m beginning to feel that XR itself is a spectrum, and a pretty woolly one. Just as it’s hard to specify in a concrete way where the boundary exists between being asleep and being awake, it’s hard to mark where “our” reality gives way to the virtual and vice-versa.

It’s based upon the addition of information to our senses, by a computer, and there can be more (as in fully-immersive VR) or less (as in the subtle application of AR) of it… but the edges are very fuzzy. I guess that the spectrum of the visual experience of XR might look a little like this:

Comic showing different levels of "XR-ness", from the most to the least: The Matrix has you now, fully immersive VR, headset AR, "Magic Window" AR, conventional videogaming, HUDs, using google maps while walking, imagining playing Tetris, reality.

Honestly, I don’t know any more. But I don’t think my sister does either.

Subscribing to The Far Side (by RSS!)

Update 23 November 2022: This isn’t how I consume The Far Side in my RSS reader any more. There’s an updated guide.

Prior to his retirement in 1995 I managed to amass a collection of almost all of Gary Larson’s The Far Side books as well as a couple of calendars and other thingamabobs. After 24 years of silence I didn’t expect to hear anything more from him and so I was as surprised as most of the Internet was when he re-emerged last year with a brand new on his first ever website. Woah.

The Far Side by Gary Larson

Larson’s hinted that there might be new and original content there someday, but for the time being I’m just loving that I can read The Far Side comments (legitimately) via the Web for the first time! The site’s currently publishing a “Daily Dose” of classic strips, which is awesome. But… I don’t want to have to go to a website to get comics every day. Nor do I want to have to remember which days I’ve caught-up with, yet. That’s a job for computers, right? And it’s a solved problem: RSS (which has been around for almost as long as Larson hasn’t) and similar technologies allow a website to publicise that it’s got updates available in a way that people can “subscribe” to, so I should just use that, right?

Except… the new The Far Side website doesn’t have an RSS feed. Boo! Luckily, I’m not above automating the creation of feeds for websites that I wish had them, even (or perhaps especially) where that involves a little reverse-engineering of online comics. So with a little thanks to my RSS middleware RSSey… I can now read daily The Far Side comics in the way that’s most-convenient to me: right alongside my other subscriptions in my feed reader.

The Far Side comics in Dan's RSS reader.
How screen scrapers are made.

I’m afraid I’m not going to publicly1-share a ready-to-go feed URL for this one, unlike my BBC News Without The Sport feed, because a necessary side-effect of the way it works is that the ads are removed. And if I were to republish a feed containing The Far Side website cartoons but with the ads stripped I’d be guilty of, like, all the ethical and legal faults that Larson was trying to mitigate by putting his new website up in the first place! I love The Far Side and I certainly don’t want to violate its copyright!

But – at least until Larson’s web developer puts up a proper feed (with or without ads) – for those of us who like our comics delivered fresh to us every morning, here’s the source code (as an RSSey feed definition) you could use to run your own personal-use-only “give me The Far Side Daily Dose as an RSS feed” middleware.

Thanks for deciding to join us on the Internet, Gary. I hear it’s going to be a big thing, someday!

Footnotes

1 Friends are welcome to contact me off-blog for an address if they like, if they promise to be nice and ethical about it.

It’s 2020 and you’re in the future

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

West Germany’s 1974 World Cup victory happened closer to the first World Cup in 1930 than to today.

The Wonder Years aired from 1988 and 1993 and depicted the years between 1968 and 1973. When I watched the show, it felt like it was set in a time long ago. If a new Wonder Years premiered today, it would cover the years between 2000 and 2005.

Also, remember when Jurassic Park, The Lion King, and Forrest Gump came out in theaters? Closer to the moon landing than today.

These things come around now and again, but I’m not sure of the universal validity of observing that a memorable event is now closer to another memorable event than it is to the present day. I don’t think that the relevance of events is as linear as that. Instead, perhaps, it looks something like this:

Graph showing that recent events matter a lot, but rapidly tail off for a while before levelling out again as they become long-ago events.
Recent events matter more than ancient events to the popular consciousness, all other things being equal, but relative to one another the ancient ones are less-relevant and there’s a steep drop-off somewhere between the two.

Where the drop-off in relevance occurs is hard to pinpoint and it probably varies a lot by the type of event that’s being remembered: nobody seems to care about what damn terrible thing Trump did last month or the month before when there’s some new terrible thing he did just this morning, for example (I haven’t looked at the news yet this morning, but honestly whenever you read this post he’ll probably have done something awful).

Nonetheless, this post on Wait But Why was a fun distraction, even if it’s been done before. Maybe the last time it happened was so long ago it’s irrelevant now?

XKCD 1393: Timeghost - 'Hello, Ghostbusters?' 'ooOOoooo people born years after that movie came out are having a second chiiiild right now ooOoooOoo'
Of course, there’s a relevant XKCD. And it was published closer to the theatrical releases of Cloudy with a Chance of Meatballs and Paranormal Activity than it was to today. OoooOOoooOOoh.

Forward #100 – Gamification

This article is a repost promoting content originally published elsewhere. See more things Dan's reposted.

Forward Comic issue 100, frame 1

(It should be noted that I have now made 100 Forward strips. I’m not saying that I’ve earned myself a blowjob, but I’m also not not saying that.)

Sure: I think you’ve earned a blowjob, Tailsteak. I think we might need a spin-off of Patreon where you can offer content creators sexual favours in exchange for their work…

(In other news, I’ve said before that you should read Forward, and now’s a great time to start. You can catch up, don’t worry.)