RSS Feed Readers Part III - Tools

26 Nov 2017 . Category: Linux
RSS

In addition to the Rawdog / FreshRSS feed readers, I’ve also been looking into a selection of tools which I can use to perform the following tasks:

Create a feed from a URL where no RSS feed exists
Fetch full article where only a partial description exists.

The tools which I had been looking into were nblock/Feeds, morss, Full-Text-RSS, f43.me and python-ftr. My preference was for a python based tool, rather than php (though the PHP ones seemed more complete).

After not having much success (see below), I suspect I’ll end up writing something for myself using the php library graby.

Morss

This tool looked interesting, as could both extract full text, rather than snippet and also can create feeds out of html. It still looks to be under active developent and had several examples on how to set up.

There were a few ways to configure this tool: Install on a server and run it as a cgi script to return xml on the fly. This method would mean I could just passs the links into any reader I like and it should just work. The other method was to explicity run the script in the feed reader, if the reader supporeted that. Rawdog had a plug-in to allow scripts to be ran, so I thought I’d try this method first.

Install was a bit of a nightmare. I had trouble installing some of teh python dependencies, specifically building lxml which appeared to be related to teh lack of memory on the Pi. The next difficulty was installing he script, as I’ve not sued the setup.py method before. Once I got it installed This happened:

# python -m morss debug http://feeds.bbci.co.uk/news/rss.xml
ERROR: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

With the help of stackoverflow and grep, I chased this down to the crawlers.py file and added in the 4th line below, then re-installed:

class SQLiteCache(BaseCache):
	def __init__(self, filename=':memory:'):
		self.con = sqlite3.connect(filename or sqlite_default, detect_types=sqlite3.PARSE_DECLTYPES, check_same_thread=False)
		self.con.text_factory = str

After that, it appeared to work. I raised an issue on Git to let the developer know about the problem and give him a possible solution. However, I then found I was getting frequent errors:

ERROR: initial_value must be unicode or None, not str

The developer fixed this after my issue, but I was then hit by another problem. At this point, I decided to move on and try something else as there were quite a few options available.

NBlock/Feeds

Facility to spider web pages and create feeds from the results. I installed this, after some (usual) struggling with dependencies. After trying to run, I got another issue (package at wrong version). Not wanting to waste more time I think I’ll move on.

Python-ftr

This could well be useful, as looks like it could creat full rss descriptions on the fly. Possibly also create feeds from html. There wasn’t any usage examples that I could see. Last commit was 3 years ago, and so I decided to leave this for now but suspect I could come back to and get working.

Full-Text-RSS

I tried their demo on the website, which didn’t work. This put me off trying to install it.