The Old Joel on Software Forum: Part 4 (of 5) - RSS scrapers for web boards?

RSS scrapers for web boards?

Are there any free scripts just lying around for the taking that can generate an RSS feed from common web boards?

I envision setting up a chron job on my server that periodically scrapes the web board in question.

I am particularly interested in scraping ezboard.com pages. As far as I know, ezboard has no RSS feed capability. If someone knows of a script that does some other common web board like UBB, then it should not be terribly difficult to modify.

Thanks.

Bored Bystander
Tuesday, February 24, 2004

PS: I've done all the standard web and NG searches and no luck.

Bored Bystander
Tuesday, February 24, 2004

http://www.boutell.com/morning/

That might do the trick. It's Morning Paper from Thomas Boutell's site. It's $10, which isn't a bad price for something potentially this useful.

In any event, he writes some great software and it's worth supporting him.

Clay Dowling
Tuesday, February 24, 2004

Just last night I started looking at XSLT used in combination with jPluck to scrape vBulletin forum sites.

Looks like a good approach, but if someones already done it, bring it on.

DaveG
Wednesday, February 25, 2004

Hi,

We're currently developing a public web facility for RSS scraping-due to be released in a few weeks, under a branded ID.

It's quite a sophisticated engine, with filters (include + exclude), rules etc etc etc. It will offer free and paid models.

It will simply allow virtually any webpage to be scraped and cleaned up, witha very straightforward user interface.

The develoment is all at back-end for now, so I've scraped an EZboard page (chosen at random)..if anyone would like me to scrape a particular page for them I'll do that at no cost, of course (That feed will have our sponsorship details in it).

EZboard page, chosen at random
http://b15.ezboard.com/facidrain56387frm18

Resultant RSS feed
http://portrayer.co.uk/cache/b15.ezboard.com31367.rss

Hopefully, the new service will be launched within 2 -3 weeks, feel free to email me and ask for more details should anyone wish to.

Take care,

Mike

MikeB
Tuesday, April 27, 2004