Surely it wouldn’t be too hard to remove all cached articles from a certain feed when that feed is removed? I think if someone removes their feed, for whatever reason, in a way they’re retracting their content so it should be deleted.

Yes, people can cache stuff locally, and yes Google caches it all anyway, blah blah, and there are a thousand reasons why removing content from the database if futile, but I still would have thought it is the right thing to do.