Fog Creek Software
Discussion Board




Web page watcher?

Does anyone use a tool to watch web pages and notify them when they change? I think ideally it would be something that plugs into IE so you can right-click on the page or click a toolbar button to "watch" the page.

The notification process I'm flexible on.

Philo

Philo
Saturday, September 13, 2003

OmniWeb (for the Mac) has this feature, and I believe Opera does as well.  In OmniWeb you bookmark the page and a green arrow appears letting you know that the content on the page has changed.  I'm sure the feature isn't too complicated to implement, the HTTP protocol supports a request to see if the page has changed.

Lou
Saturday, September 13, 2003

What you want to do is get a HTTP Fetch component or something and grab the body. The header will always change.

Run a CRC32 or Hash and cache the key.  Compare as you go through out the day while grabbing the same page.

By the way the header also tells you if the content has changed, but it's a bit useless with dynamic pages which is why you'll want to do this whole Hash work around.

It's not the perfect solution because you are spamming the web server with requests you'll never read.

Li-fan Chen
Saturday, September 13, 2003

True, but so long as you pull one request of just the page every five minutes or more it shouldn't be a big deal.

BTW, make that hash of everything between <Body> tags, and you'll want to parse out the viewstate if it's an aspx page.

Philo

Philo
Saturday, September 13, 2003

Let me guess, you are trying to write a watch dog :-)
I have been meaning to write one..

Li-fan Chen
Sunday, September 14, 2003

Some sites do: http://www.rba.co.uk/sources/monitor.htm

BillyBill
Sunday, September 14, 2003

There is a notification by email feature already built in to IE. Make a bookmark, then right click on the bookmark and make the page available offline. On one of the offline page options there is an option to notify you by email when the page changes.

Matthew Lock
Sunday, September 14, 2003

banner ads usually screws up the thing, displaying current date as well.

wouldn't be nice if things were built "correctly" in the first place, where these issues are easier to handle

na
Sunday, September 14, 2003

In Mozilla/Firebird:

Right click any bookmark and select properties. In the bookmark properties window on the Schedule tab you can set how often you want to check for updates. The Notify tab allows you to select how you want to be notified.

jan Derk
Sunday, September 14, 2003

Does the notification feature actually work in Firebird? I have tried it heaps of times but nothing actually happens.

Matthew Lock
Sunday, September 14, 2003

I was a very happy user of MindIT! Notification service, and when it went down (or became pay-ware, I'm not too sure) I went around looking for _good_ sw to do the same: look at pages, notify when there are changes, etc.

Yes, I know you can hack-your-own, and there are plenty perl-scrpits out there that will do it for you, so installing Cygwin could be another possibility. But I wanted it to be hassle-free, to have support for changes highlighting, regexp-matching, mail notification, programmable checking intervals and all the bells & whistles:

And I found it: Website Watcher
http://www.aignes.com/index.htm
All that, and many more things that I haven't had time to really look over (for instance, logging-in before check, to check password protected pages, etc).

A _really_ cool tool. I have it running on my systray all the time doing its thing, and when I want, I just have a look at the changed sites, with the changed pages "lighted up". Of course, having an always-on connection is preferable to really get the best out of it. (I have over 300 "watched" URLs at the moment :D)

DISCLAIMER: I am in no way affiliated with aignes.com or its products; I am just a very happy user.

Javier J
Sunday, September 14, 2003

WebMon works very well and is freeware: http://www.markwell.btinternet.co.uk/webmon/

Noah
Sunday, September 14, 2003

Hmmm. Matthew, you are right. It used to work when I tested it in an older Mozilla build, but it sure is broken in the latest Firebird version.

jan Derk
Sunday, September 14, 2003

Its been broken for a while, which is a pain, it kept me up todate without me having to do anything at all.

Simon Lucy
Sunday, September 14, 2003

One more for WebSite-Watcher - it does exactly what you've asked. It amazed me so much that I've purchased it a week after trying it out .. A real tool for tracking hunderds of pages (in my case). Btw, it has started to support the RSS in it's latest betas - I tried feeding it Joel's chanel - looked very appealing, although I still tend to watch only for plain old HTMLs .. It has regex matchers, very powerful filters (Ignore user defined strings, Watch only user dfined strings)and a very comfortable navigation through all links. The support is very helpful - I usually receive letter in the same day.

One of my favorites, really ..

Evgeny Goldin
Monday, September 15, 2003

Javier J, you've got me - I have only 267 links. Although, I'm starting to turn off the Autowatcher for many of them .. And use the WsW as a bookmark manager.

Evgeny Goldin
Monday, September 15, 2003

http://www.changenotes.com/

T.S.
Tuesday, September 16, 2003

ChangeDetect is an online service that does what you are looking for:

http://www.changedetect.com/

Not totally free, but you get what you pay for.

Bam Craig
Saturday, April 03, 2004

*  Recent Topics

*  Fog Creek Home