Fog Creek Software
Discussion Board




What's the daily spam solution?

It seems you can't go a day without some new suggestion on how to solve the spam problem.

Does anyone else notice that the "receiving end" solutions are not and never will be effective?

So far, the only "serving end" solution that doesn't put people out is the "computational puzzle" that only slows spam, not eliminate it.

People are paying big bucks for filtering messages on the POP3 servers, or whatever thing is receiving mail.

A better place for filters is on the SMTP side of things.  It could say "I'm sorry, but this mail looks like spam to me, so I will not send it."

That will reduce a large bulk of spam, since it will never get onto the network.

I think it would be better for false positives too.  The sender knows they sent it, whereas the receiver doesn't know that.  They get a message with a reason why it was flagged as spam, they fix it and resend.

Given it has holes like all other solutions, but isn't it better to keep it off the Net than dealing with it after it's in the pipe?

Click here for more information
Thursday, February 05, 2004

"So far, the only "serving end" solution that doesn't put people out is the "computational puzzle" that only slows spam, not eliminate it."

Actually the point is to slow the sending of email messages to the point where it becomes unprofitable to send them.  Currently one user can send a million email messages without much trouble.  If it took one second to send each email message, it would take 11 days to send that many messages.

There are a couple of problems with email; one is email relaying.  Why do I connect to my local server mail.myisp.com to send messages?  If I send a message to someone@yourisp.com then my mail client should connect to yourisp.com and transmit it directly into your mailbox.  This is no different from using the web (my browser connects to your webserver) or IM.

Almost Anonymous
Thursday, February 05, 2004

The problem is spammers have a level of control over the SMTP servers they use.  Whether they installed it on their own machine or on an unsuspecting user's trojaned desktop, they can use an accomadating SMTP server of their choice.

Michael Mata
Thursday, February 05, 2004

Maybe we can get SMTP servers categorized as WMD and get some military action to help solve problems. :P

m
Thursday, February 05, 2004

Funny you should mention it.  We recently started using ASSP (Anti Spam SMTP Proxy) here at work, and it basically works just like it sounds.

Basically it's an SMTP server with Bayesian filtering, anything it sees as spam it rejects with an SMTP error.  It also has the added benefit of blocking executable attachments if you want it to, so we've had very little of Mydoon to deal with on the AV side.

You can get it at http://assp.sourceforge.net/ .

--Steve

Steve Barbour
Thursday, February 05, 2004

http://www.spambayes.org

So far:  1 week of training, 1 week of correctly identifying spam.

I'll give it 1 month to make sure it's still tagging the spam correctly, then I'll just have outlook express delete the messages from the server.

enlightened one
Thursday, February 05, 2004

where 'training' means training the filter.

ps:  It's written in python.  You will have to RTFM.

enlightened one
Thursday, February 05, 2004

I love SpamBayes and have had excellent success with it.

Joel Spolsky
Fog Creek Software
Thursday, February 05, 2004

>> If it took one second to send each email message, it would take 11 days

Exactly.

If there was some heavy-duty encryption involved, or requesting the public key from the server, or some other cpu-intensive task for every email... spammers would need supercomputers.

Alex.ro
Thursday, February 05, 2004

We should future proof the design and make it so that the receiver sets the complexity of the calculation and have the entire calculation algorithm scalable.  So that when we are all running 200Ghz machines, it still takes the same amount of time to send a message.

The main problem I do see with this is the number of low-power devices that can send email.  They may all have to be equiped with mail calculation hardware to get any level of performance when sending messages.

Almost Anonymous
Thursday, February 05, 2004

>> when we are all running 200Ghz machines

Right.

On the other hand, downloading something (like a public key) should always take a palpable amount of time.

Still... by the time we have 200Ghz machines, who's to say we won't have multi-gigabit connections.

Bill Gates' 10c-per-email solution is nice.

Joel -- SpamBayes isn't workable for dial-up because I can't wait for the ~1 MB of spam to crawl through, just so I can squash it.

Alex.ro
Thursday, February 05, 2004

"On the other hand, downloading something (like a public key) should always take a palpable amount of time."

Perhaps, rather than a mathematical compution, each email requires you to connect to my server.  Just as I suggested further up.  Now I set my server to wait 1 second before it responds to you -- now we have the same result but with a simpler implementation.  And it won't matter how fast (or slow) your machine is at computation.

Almost Anonymous
Thursday, February 05, 2004

Alex: if you can run software on your server, SpamBayes has a combination web UI/POP3 proxy that can save you from downloading the spams.  The POP3 proxy only presents you with clean mail to download, and the web UI lets you do the training and review the subject lines of rejected messages.

Phillip J. Eby
Thursday, February 05, 2004

Almost Anon,

That solution requires that you be running a mail server 24x7 though, doesn't it? Very few people have that option.

Chris Tavares
Thursday, February 05, 2004

I didn't mean that you would run a server on your home machine.  Rather, your ISP or webhost would run the server just as they do now.  The difference is that to send you mail, users would connect directly to your server rather than sending mail via their ISP's (or anyone elses) SMTP server.

Almost Anonymous (from P800)
Thursday, February 05, 2004

+1 For ASSP

I was using SpamBayes in Outlook, and love it. But I still had the problem of sitting down in the morning at a client's office and having to pull 200 spams back off my server.

ASSP has solved that for me by putting SpamBayes level accuracy earlier up the chain.

Damian
Thursday, February 05, 2004

Here's a question for y'all--what if you a) don't own the mail server (I use Speakeasy's mail) and b) check mail via shell, web, and client?  So, server-based solutions are pretty much out (they run one, but it's not great), and client-based solutions only get used at one client.

Suggestions?

Rich
Friday, February 06, 2004

"Does anyone else notice that the "receiving end" solutions are not and never will be effective?"

I disagree. Being sensible with who I give my e-mail address to, in combination with using K9* works great for me.

* http://www.keir.net/k9.html

John Topley (www.johntopley.com)
Friday, February 06, 2004

"Does anyone else notice that the "receiving end" solutions are not and never will be effective?"

The idea of having the SMTP server filter the mail at the source is pretty naive.

First, most spammers find SMTP servers that are misconfigured to begin with (In my book anyone who has an SMTP server set up to allow anyone to relay from it doesnt know what they are doing, or intends to allow people to use the server for spamming), so expecting those "admins" to set up their server (unless its the default setting) to use the filter isn't likely.  And even if it is the default setting, they will just find another server that allows relaying.

The rest use their own SMTP servers, most of which are in foreign countries where the actions are "accepted".  Even certain companies in the US allow it (just not many and only until enough people or legal action compells them to stop.)

Installing a "filter" on the SMTP server to reject what  "could" be considered spam would be have a negative impact on the users.

Yes, the user can fix it,  but what happens when you try to send a link for viagra or some other product and the server deems it "spam" and rejects it.  How do you fix it, elimiate the link? Then what is the point of the email? 

And that is the ones who are smart enough to actually read the bounceback message and understand it, which, with no disrespect intended to anyone, is about 8 or 9 % of the userbase for most ISP's.

Unfortunately spam is the cockroach of the internet, you can slow it down, but you can never kill it.  At the present its pretty much "action/reaction".  Spammers do something to get around anti-spam software, anti-spammer software reacts and fixes it. 

Personally I hate it, but the software I use filters out the majority of it, and if a few trickle thru, ohwell, easy enough to delete.

Daremo
Thursday, February 12, 2004

*  Recent Topics

*  Fog Creek Home