Day: 16 July 2024

It’s time for… Dan Shares Yet Another FreshRSS XPath Scraping Recipe!

Vmail

I’m a huge fan of the XPath scraping feature of FreshRSS, my favourite feed reader (and one of the most important applications in my digital ecosystem). I’ve previously demonstrated how to use the feature to subscribe to Forward, reruns of The Far Side, and new The Far Side content, despite none of those sites having “official” feeds.

Vmail is cool. It’s vole.wtf’s (of ARCC etc. fame) community newsletter, and it’s as batshit crazy as you’d expect if you were to get the kinds of people who enjoy that site and asked them all to chip in on a newsletter.

Totes bonkers.

But email’s not how I like to consume this kind of media. So obviously, I scraped it.

Recipe

Want to subscribe to Vmail using your own copy of FreshRSS? Here’s the settings you’re looking for –

• Type of feed source: HTML + XPath (Web scraping)
• XPath for finding news items: //table/tbody/tr
  It’s just a table with each row being a newsletter; simple!
• XPath for item title: descendant::a
• XPath for item content: .
• XPath for item link (URL): descendant::a/@href
• XPath for item date: descendant::td[1]
• Custom date/time format: d M *y
  The dates are in a format that’s like 01 May ’24 – two-digit days with leading zeros, three-letter months, and a two-digit year preceded by a curly quote, separated by spaces. That curl quote screws up PHP’s date parser, so we have to give it a hint.
• XPath for unique item ID: descendant::th
  Optional, but each issue’s got its own unique ID already anyway; we might as well use it!
• Article CSS selector on original website: #vmail
  Optional, but recommended: this option lets you read the entire content of each newsletter without leaving FreshRSS.

So yeah, FreshRSS continues to be amazing. And lately it’s helped me keep on top of the amazing/crazy of vole.wtf too.

×

×