Fog Creek Software
Discussion Board




anti-spam

What do you think of the following way to fight spam? Does anybody know a solution which already implements this:

Traditionally spam filters try to define rules which would classify e-mail as spam. This has become very cumbersome and unreliable. Bulk mailer become more and more sophisticated, so they can make mail filters fail on them.

Looking at it from a different angle, why don't we classify every e-mail as spam until we have been convinced that we're wrong? On the mail server there is a white list of e-mail addresses (or domains) you trust. Only e-mail coming from one of those addresses will be directly routed to your mail account. If somebody sends you an e-mail who is not on the white list, he'll automatically get an e-mail with an URL to a web interface he has to use to authenticate himself. Once he provided his name and the reason he wants to e-mail you, this information will be emailed to you. Within this e-mail there will be two links, “add to white list” and “deny from white list”. If you add him to your white list you will get the e-mail he tried to send to you originally. Since he is a known sender now, he won't be bothered anymore to authenticate himself. If you deny his request, he'll be notified by e-mail that it doesn't make sense for him to e-mail you anymore.

There should be a web interface which can be used to manually maintain the white list. For example if you want to receive certain newsletters you should be able to add the domain or e-mail address of the newsletters sender to your whitelist. The same you should do with people’s addresses you already know and trust.

It's quite easy to do that via procmail, perl, elm ...
But do you think it would be acceptable to the people who'd email you?

Stephan
Thursday, January 16, 2003

definitely look into the latest excitement about bayesian filtering.  It is the dream spam filter.  Just google for it, look on slashdot or just use mozilla.

bccfan
Thursday, January 16, 2003

http://www.mailshell.com

does what you want, i think
Thursday, January 16, 2003

"It's quite easy to do that via procmail, perl, elm ...
But do you think it would be acceptable to the people who'd email you?"

Yes for personal mail, no for business.

Just me (Sir to you)
Thursday, January 16, 2003

For a Bayesian filter that will work for any POP mail client use POPFile,  use http://popfile.sourceforge.net

Its best when starting to have a good population of both your good and spam email to feed it.  Otherwise you have to spend a lot of time training it with the mail you get.

Once it is trained by either method its a good 98% accurate for my email.  As spam is roughly 45% of all my mail that's more than worthwhile.

Today I just got rid of my '!' filter which before using POPFile was the most efficient non-specific mail filter I had.

Simon Lucy
Thursday, January 16, 2003

Amen to the "guilty until proven innocent" policy. 

I've been looking for some way to do this in Outlook for years.  Why can't I validate incoming email against my contact list?  If the address is not in my contact list, presume it's spam and send it to some folder or the recycle bin.  From there you can review before deleting.  If a valid email was filtered, add that address to your address book/contact list (i.e. the "white list") and it'll never be filtered again.  After a while you'll have all your valid addresses defined.

Seems simple enough.

Joe Paradise
Thursday, January 16, 2003

For those who are interested, there will be a Spam conference hosted the 17th of January:

http://www.spamconference.org/

Robert Chevallier
Thursday, January 16, 2003

Looks like way too much effort for me (on my own part as well as on the sender's part).

My private e-mail address is published on my homepage and on other places in the web. I normally use either my real private e-mail or my business e-mail adress to post to newsgroups and forums (even though I use spam blockers in in the usenet, meaning I write capellaNOSPAM@jutta-jordans.de, hoping that a human being is sensible enough to get rid of it)

The amount of spam in my postbox is ok (about 5 per week I'd say, inlcuding some curious "newsletters" I never orderered and which prove very difficult to get rid of).

I met some very interesting people, who just contacted me after reading either my homepage or something I wrote in a forum or newsgroup. I doubt that they would have been willing to sign up on my white list first before writing to me for the first time. It would have been a real loss.

I think, I rather live with some spam and remain open to those who want to get in contact with me than closing up like a clam to get rid of the regular penis enlargement or finance my Nigerian law firm spam mail.

Have fun,

Jutta Jordans
Thursday, January 16, 2003

After receiving numerous spam e-mails (including e-mails with viruses) we decided to add a spam filter to PopUpBuster. It works fairly well for us. It is based on black and white lists.

PopUpBuster
Thursday, January 16, 2003

The Mail Matador product is the system you`re talking about. I`ve received a funny verification e-mail from someone that uses it. I had to count puppies on the picture to prove that I`m not a robot:)

Slava
Thursday, January 16, 2003

The spam filter on the new version of Opera (7)  seems quite good. It's only in beta, but it's reasonably(!) stable.

www.opera.com

Yandoo
Thursday, January 16, 2003

I've tried a couple since my previous post, and SpamNet is my current favorite:

- tight integration with Outlook
- simple interface
- supports whitelists
- it's free

http://www.cloudmark.com/products/spamnet/

Joe Paradise
Thursday, January 16, 2003

Also check out SpamBayes:

http://spambayes.sf.net/

While their approach is also "Bayesian" in the sense of being based on probability learning, the math and tokenizer used are much different than that of many other current "Bayesian" filters.  The Spambayes tools can give "better than chance" results with as little as 1 spam and 1 "ham" message for training, and within a few hundred messages of training can exceed 99% accuracy.  Spambayes also uses three-way classfication: ham, spam, and "unsure".  Paul Graham's "A Plan for Spam" algorithm is *highly* susceptible to claiming that a message is definitely ham or definitely spam, even when a human reader would be unsure.  When Spambayes sees something that's sufficiently unlike anything it has seen before, it will mark it "unsure" so that you can decide for yourself, and let it "learn" accordingly.  (Of course, you can set your own upper and lower probability threshholds for what you want to have tagged as "unsure".)

The project so far includes an Outlook 2000 client as well as a POP proxy similar in principle to POPfile.  The main downside is that the Spambayes stuff is only available via Sourceforge and there's not much of an installation process for most of the tools yet.

Phillip J. Eby
Thursday, January 16, 2003

I have heard that some bulk mailing outfits are putting email addresses from the same domain as the target email address in the From field.  So say your email address is bob@mydomain.com, and your co-worker / boss / whoever jeff@mydomain.com is also on the spammers email list, you might actually get an email "from" jeff@mydomain.com about life insurance, viagra etc

Something that amuses me is that spammers are putting more and more random characters into the from, subject, message and to fields of the emails in order to trick spam filters, so that some emails really do look like total junk.

I also found Cloudmark's SpamNet to work very well - half the reason is probably the auto-update feature which allows the writers to come up with new algorithms to keep up with the different ways that spammers are randomizing their emails.

It would be interesting if there was a "fight back" button as well so that when you get a junk email, you press "fight back" and your computer sends some TCP traffic to the source of the emails every second for a couple of minutes.  No noticeable affect on your computer but if 100,000 people are doing the same thing ... in order to successfully send out spam mail, the spammers would need more expensive hardware etc to get around the DOS load, so the prices would go up, pushing low-life spammers out of the picture...

Mike Mueller
Friday, January 17, 2003

If I ever get chance to do it (the hall floor beckons to being laid), I'd want to convert POPFile from a client proxy to a server proxy so that everyone benefits.

Simon Lucy
Friday, January 17, 2003

I have something resembling a "guilty until proved innocent" solution, and I set it up easily.

I have a folder called "Real Inbox".  I use that as my default inbox.  I've set up filters to filter only e-mails from known good addresses into the "Real Inbox" folder.

I check the e-mail application's inbox every so often to see if an e-mail slipped through the cracks.  Warning: It took me weeks to build up a solid list of positive addresses.

Brent P. Newhall
Friday, January 17, 2003

Looking to recycle,import, export, buy, sell or trade scrap, recyclable, recycled materials? We specialize in integrating importers, exporters, traders, individuals and organizations of the recycle industry, making it fast and effective to buy and sell scrap, recyclable, recycled materials.

URL:http://www.recycleINme.com

Gunaseelan
Monday, July 05, 2004


Web URl : http://www.exchangeandplant.com


Title :     New and Used Heavy Construction Machinery market place

Jefferson
Saturday, August 07, 2004

*  Recent Topics

*  Fog Creek Home