Pushing RSS to WhatsApp (for free?)

I’m always keen to experiment with new ways to subscribe to my blogObviously RSS is the best and everybody who can use it should. But some people, for their own reasons1, prefer to learn about updates to their favourite sites via the Fediverse, or Facebook, or Telegram, or… I don’t know… LiveJournal or something (yes, those are all places you can follow this blog, if you really wanted to).

But I’m pretty sure there are some people who’d rather receive updates to my blog via WhatsApp. And now, they can. Here’s how I set up an RSS-to-WhatsApp gateway, in case you want to run one of your own2.

Diagram showing how GitHub Actions request and receive an RSS feed, push an update to Whapi, which pushes the update to WhatsApp.

Prerequisites

You will need:

  1. A GitHub account – a free one is fine
  2. A Whapi account connected to your WhatsApp account3 – when you set up an account you’ll get a free trial; when it ends you need to find the link to say that you want to carry on with the free tier (or upgrade to the paid tier if you expect to send more messages than the free tier’s limit)
  3. A WhatsApp channel to which you want to push your RSS feed: I’d recommend that you make a newsletter (from the Updates tab in WhatsApp, press the kekab menu then Create Channel) rather than a traditional group: groups are designed for multiple people to talk and discuss and everybody can see one another’s identity, but a newsletter keeps everybody’s identity private and only allows the administrator(s) permission to post updates.
A WhatsApp 'Subscription Channel'/'Newsletter' group for DanQ.me, showing some recent posts, on a mobile screen.
You probably want to use the kind of channel that’s for one-to-many ‘push’ communication, not a discussion group.

Instructions

  1. Fork the Dan-Q/rss-to-whapi.cloud repository into your own GitHub account.
  2. In Settings > Secrets and Variables > Actions, add two new Repository Secrets:
    • WHATSAPP_API_TOKEN: set to the token on your Whapi dashboard
    • WHATSAPP_CHANNEL: set to your newsletter ID (will look like 123456789012345678@newsletter) or group ID (will look like 123456789012345678@g.us): you can get this from the Newsletters or Groups section of Whapi by executing a test GET /newsletters or GET /groups request4.
  3. Make a feeds.json file (a feeds.json.example is provided as a guide) containing the URLs of the RSS feeds you’d like to subscribe to.
  4. Do a test run: from the Actions tab select the “Process feeds” action and click “Run workflow”. If it finishes successfully (and you get the WhatsApp message), you’re done! If it fails, click on the failed action and drill-in to the failed task to see the error message and correct accordingly.

By default, the processor will run on-demand and every 30 minutes, but you can modify that in .github/workflows/process-feeds.yml. It’s configured to send the single oldest un-sent item in any of the RSS feeds it’s subscribed to, on each run (it tracks which ones it’s sent already by their guids, in a "seen": [...] array in feeds.json): sending a single link per run ensures that WhatsApp’s link previews work as expected. At that rate, you could theoretically run it once every 10 minutes and never hit the 150-messages-per-day limit of Whapi’s free tier5), but you’ll want to work out your own optimal rate based on the anticipated update frequency of your feeds and the number of RSS-to-WhatsApp channels you’re running.

You can, of course, run it on your own infrastructure in a similar way. Just check out the repository to your local system with Ruby 3.2+ running, run bundle to install the dependencies, then set up a cron job or some other automation to run ./process_feeds.rb. Doing this could be used to hook it up to your RSS feed updating pipeline, for example, to check for new feed items right after a new post is published.

Footnotes

1 Their own incomprehensible, illogical, weird reasons.

2 I hope that the title gives it away, but you can do this completely for free. So long as you keep your fork of the GitHub repository open-source then you can run GitHub Actions for free, and so long as you’re pushing out no more than 150 updates per day to no more than 5 different channels in a month then you can do it within Whapi’s free tier: that’s probably fine for a personal blogger, and there’s a reasonable pricing structure (plus some value-added extras) for companies that want to use this same workflow as part of a grander WhatsApp offering.

3 Setting this up requires giving Whapi access to your WhatsApp account. If you don’t like the security implications of that, you could get a cheap eSIM, set that up with WhatsApp, and use that account: if you do this, just remember to “warm up” your new WhatsApp account with some conversations with yourself so it doesn’t look so much like a spammer! Also note that the way Whapi works “uses up” one of the ~4 devices on which you can simultaneously use WhatsApp Web/WhatsApp Desktop etc.

4 Prefer the command-line? So long as you’ve got curl and jq then you can get a list of your newsletters (or groups) and their IDs with curl -H 'Authorization: Bearer YOUR_API_TOKEN' -H 'accept: application/json' https://gate.whapi.cloud/newsletters?count=100 | jq '.newsletters[] | { id: .id, name: .name }' or curl -H 'Authorization: Bearer YOUR_API_TOKEN' -H 'accept: application/json' https://gate.whapi.cloud/groups?count=100 | jq '.groups[] | { id: .id, name: .name }', respectively.

5 Going beyond the free tier would require sending one message, on average, every 9 minutes and 36 seconds.

Watch Together with WhatsApp on the side

A virtual party

This weekend, I threw a Virtual Free Fringe party for some friends. The party was under-attended, but it’s fine because I got to experiment with some tech that I’d been meaning to try.

Phootgraph of a wall-mounted television screen. On the screen, comedian Peter Buckley Hill sits with his guitar on his lap in front of an audience: the "PBH's Free Fringe" logo is on the curtain behind him. On the left of the screen a series of WhatsApp messages appear, including one showing a photo of Dan holding a can of Old Speckled Hen beer.
The Abnibbers and I have experimented with watching things together, but apart, before, but this is the first time we’ve watched stand-up comedy this way.

If you ever want to run something like this yourself1, here’s how I did it.

My goals were:

  • A web page at which any attendee could “watch together” a streaming video2,
  • A “chat” overlay, powered by a WhatsApp group3 (the friend group I was inviting were all using WhatsApp anyway, so this was an obvious choice), and
  • To do all the above cheaply or for free.
Selfie photograph of Dan, in a bar with a rooftop view of daylight out the windows in the background, looks concerned as he stares at the a frothy, bubbling flask of yellow liquid he's holding.
I’m a big fan of experiments. Contrary to this picture, though, they’re usually software experiments.

There were two parts to this project:

  1. Setting up a streaming server that everybody can connect to, and
  2. Decorating the stream with a WhatsApp channel

Setting up a streaming server

Linode offers a free trial of $100 of hosting credit over 60 days and has a ready-to-go recipe for installing Owncast, an open-source streaming server I’ve used before, so I used their recipe, opting for a 4GB dedicated server in their London datacentre: at $36/mo, there’d be no risk of running out of my free trial credit even if I failed to shut down and delete the virtual machine in good time. If you prefer the command-line, here’s the API call for that:

curl -H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-X POST -d '{
    "authorized_users": [
        "[YOUR LINODE USERNAME]"
    ],
    "backups_enabled": false,
    "booted": true,
    "image": "linode/debian10",
    "label": "owncast-eu-west",
    "private_ip": false,
    "region": "eu-west",
    "root_pass": "[YOUR ROOT PASSWORD]",
    "stackscript_data": {
        "server_hostname": "[YOUR DOMAIN NAME]",
        "email_address": "[YOUR EMAIL ADDRESS]"
    },
    "stackscript_id": 804172,
    "tags": [],
    "type": "g6-dedicated-2"
}' https://api.linode.com/v4/linode/instances

The IP address got assigned before the machine finished booting, so I had time to copy that into my DNS configuration so the domain was already pointing to the machine before it was fully running. This enabled it to get its SSL certificate set up rightaway (if not, I’d have had to finish waiting for the DNS change to propogate and then reboot it).

Out of the box, Owncast is insecure-by-default, so I wanted to jump in and change some passwords. For some reason you’re initially only able to correct this over unencrypted HTTP! I opted to take the risk on this server (which would only be alive for a few hours) and just configure it with this limitation, logging in at http://mydomain:8080/admin with the default username and password (admin / abc123), changing the credentials to something more-secure. I also tweaked the configuration in general: setting the service name, URL, disabling chat features, and so on, and generating a new stream key to replace the default one.

Now I was ready to configure OBS Studio to stream video to my new Owncast server, which would distribute it to anybody who tuned-in.

Screenshot showing OBS Studio window with Start Streaming enabled. The layers "VLC", "Abnib Logo", "WhatsApp icon", WhatsApp prompt", and "WhatsApp" are visible. Elsewhere on the screen, a WhatsApp Web view is visible, with its CSS tweaked to give it a red background, among other changes.
Next up, we need to make WhatsApp appear on the stream with a little bit of CSS hackery.

Decorating the stream

I configured OBS Studio with a “Custom…” stream service with server rtmp://mydomain:1935/live and the stream key I chose when configuring Owncast and kicked off a test stream to ensure that I could access it via https://mydomain. I added a VLC source4 to OBS and fed it a playlist of videos, and added some branding.

With that all working, I now needed a way to display the WhatsApp chat superimposed over the video.For this, I added a Window Capture source and pointed it at a Firefox window that was showing a WhatsApp Web view of the relevant channel. I added a Crop/Pad filter to trim off the unnecessary chrome.

OBS Studio screenshot showing a WhatsApp Web (Window) source tied to a Firefox window and with Crop/Pad and Chroma Key filters applied.
The same technique, of course, could be used to superimpose any web page or whatever other content you like onto a stream.

Next, I used the Firefox debugger “Style Editor” to inject some extra CSS into WhatsApp Web. The class names vary frequently, so there’s no point we re-documenting all of them here, but the essence of the changes were:

  1. Changing the chat background to a solid bright color (I used red) that can then be removed/made transparent using OBS’s Chroma Key filter. Because you have a good solid color you can turn the Similarity and Smoothness way down.
  2. Making all messages appear the same (rather than making my messages appear different from everybody else’s). To do this, I added:
    • .message-in, .message-out { align-items: flex-start !important; } to align them all to the left
    • [aria-label="You:"]::after { content: "Dan Q"; height: 15px !important; display: block; color: #00f !important; padding: 8px 0 0 8px; } to force my name to appear even on my own messages
    • [aria-label^="Open chat details for "] { display: none; } to remove people’s avatars
    • [data-testid="msg-meta"] { display: none !important; } to remove message metadata
    • A hacky bit of CSS to make the backgrounds all white and to remove the speech bubble “tails”
  3. Removing all the sending/received/read etc. icons with [data-icon] { display: none; }

I aimed where possible to exploit selectors that probably won’t change frequently, like [aria-label]s; this improves the chance that I can use the same code next time. I also manually removed “old” messages from the channel that didn’t need to be displayed on the big screen. I wasn’t able to consistently remove “X new messages” notifications, but I’ll probably try again another time, perhaps with the help of an injected userscript.

A little bit of a shame that more people didn’t get to see the results of this experiment, but I’m sure I’ll use the techniques I’ve learned on another ocassion.

Footnotes

1 Or, let’s be honest, if you’re Future Dan and you’re trying to remember how you did it in last time.

2 We were to watch a show by one of my favourite comedians Peter Buckley Hill, the man behind the Free Fringe. I’ve written about him previously… here, there, also several times in 2012 when I also helped make an official digital map of Free Fringe venues. I was especially delighted to have my photo taken with him in 2006. I might be a bit of a fanboy.

3 This could probably be adapted for any other chat system that has a web interface, so if you prefer Telegram or Slack or whatever ever, that’s fine.

4 OBS’s VLC source is just amazing: not only can you give it files, but you can give it URLs, meaning that you can set up a playlist of YouTube videos, or RTSP security camera feeds, or pretty much anything else you feel like (and have the codecs for).

× × × ×