Fog Creek Software
Discussion Board




SMTP Replacement...

Joel wrote "Let's find a protocol with decent authentication and with micropayments to make spam uneconomical, and let's set a deadline, maybe two years in the future, when SMTP will simply be turned off."

Micropayments?  That's not going to fly with 90% of the population (myself included), I'd rather deal with spam than pay to send email.  This just comes back to the fact that once you are used to getting something for free, you will not ever (in most cases) be willing to pay for it.  What if cities started charging pedestrians tolls for walking down the street?  I think most people would be unwilling to do so.

GiorgioG
Tuesday, April 22, 2003

I can remember when TV was free.  Now days people seem quite willing to pay tens of dollars a month for it.

mackinac
Tuesday, April 22, 2003

Grandma sending e-mail.

What are all those people going to do who can <i>barely</i> send e-mail?

All that "Always Click Yes" mentality that has saved them so far is going to be pretty negative when their bank accounts are empty.

Pleeeeze don't let me connect my bank account to my e-mail!

I guess that's what Passport was for . . .

R. C.
Tuesday, April 22, 2003

There are other reasons than SPAM for why SMTP is bad.  CNN is reporting that some idiots sent email out as a real person and ruined their reputation

http://www.cnn.com/2003/TECH/internet/04/21/hate.email.ap/index.html

> By exploiting the simplicity and openness of the
> Internet's mail protocols, unidentified
> provocateurs have been sending incendiary
> messages posing as Shora and other Arab-
> Americans.

Lou
Tuesday, April 22, 2003

I can remember when TV was free too.  However, I would not pay for the same TV experience once it was free.  Email is a commodity. 

So what will I get for the extra $/email I must now pay?  What additional value can be added to email to make it worth paying for?  With cable, I get 200 channels, movies that are uncut and commercial free. 

As someone pointed out, spam is not that big a problem for many of us.  For others, the problem seems to be an almost unnatural willingness to put their full name in the public eye.  If my email address was not hidden from bots, I would not use it here.  The best would be a variation a person would know, but a bot would not.  (janedoeATjohnsmith.com)

While a solution may become necessary, making the good Samaritans pay because we cannot punish the bad ones seems the wrong approach.

Mike Gamerland
Tuesday, April 22, 2003

Suppose there was a charge of 1 cent for sending mail, of which the recipient got 7/8ths of a cent and the payment processor got 1/8th of a cent.

A typical user sends, what, 100 emails a month? 200? That's one or two bucks. Barely noticible. But legit email users also receive email, about the same number. So they will actually be paying closer to 2 bucks a year. This is nothing compared to how much they pay for their ISP and their computer.

Currently, a spammer might send a message to a million people (at least) in hopes of getting a couple of dozen suckers to respond. Under the new system that would cost $10,000. This would not be worth it just for getting a couple of dozen suckers. Suddenly the scam spam mail goes away.

However "legit" direct marketing businesses (not scams) would probably have a higher response rate so they might still spam you. In any case, they now have a financial incentive to reduce their lists to people who might actually respond instead of spamming the world. Arguably, I'd rather see legit businesses advertising via email rather than paper mail; less damage to the environment, and with this scheme, they would be paying you 7/8ths of a cent to receive their ads, which would subsidize your mail.

Joel Spolsky
Tuesday, April 22, 2003

the bayesian filter i'm using (CRM114) lets through a maximum 2 spams per week. compared to the 20-40 spams per DAY I used to receive. thus, I don't believe one needs to posit an email surcharge to cure spam.

now, if you want to make email a legitimate means of advertisement, this might be a different story. however, I don't read junk physical mail, either, so i'm not sure why you think spam is an effective advertising tool.

choppy
Tuesday, April 22, 2003

Well, most people (in the UK at least, don't know about the US) have to pay a small amount of money every time they send a text message to a mobile phone. I still get junk messages from time to time, but nowhere near as bad as the amount of spam / real email I'm getting. (Then again, I use email more than text messaging.)

Better than being unemployed...
Tuesday, April 22, 2003

Having developed SMTP software for the last few years I have a few exception with the article:

The US postal service charges money for delivery but I still get junkmail so how would charging micropayments be any different? We would still get spam just on a smaller level and it would be considered legit commercial advertisement that we could do nothing about.

I'd like to see one try to implement a micropayment system worldwide and tell _everyone_ on the Internet that what they used to get for free now costs money. I have doubts about that. How would one implement a worlwide payment system? How does one account for all of the transactions? This is a tremendous hurdle that should not be scoffed at.


From the article:
[Speaking of not caring, I don't care about the anonymity problem. It's not the only problem out there and it doesn't completely trump others, like anonymous pornographers e-mailing our kids.]

This is a freedom that I think many will not give up, including myself. So instead of being anonymous in this proposed system we will now be known by our certificates. So now these legit bulkmailers (legit because the pay to send mail) can identify me absolutely and target directly. As could oppresive governments identify dissedents.  Or maybe you just want to post politically incorrect thoughts. Sometimes it is better to remain anonymous. Not necessarily to shield you from accountability, but to instead shield you from being tracked by endless institutions that will data mine and categorize your life until your right to privacy is a dream that got thrown out in the name of fighting junkmail.

Why on earth would you be willing to give this up? Does email really scare you that much? I have a kid also and it certainly doesn't scare me so much that I would be willing to give up my anonymity.


The point I am trying to make is he really doesn't iron out any of the details of his proposed solution beyond just complaining like everyone else. If you can think of a solution that will solve this _and_ keep everyone happy then you have your work cut out for you. Draft an RFC and go to town.


A system that implements some of the proposed authentication and filtering is Vipul's Razor (SpamNet): http://razor.sourceforge.net/. It's got some very interesting methods for not only filtering spam but also authentication of senders.

Ian Stallings
Tuesday, April 22, 2003

I figure that we could get a lot of mileage out of assuring the identity of each server along the path.

Which can be done in a web-of-trust sort of system, instead of a Verisign sort of system where each server needs to pay $$$ for a certificate.  You would also most likely institute the protocol requirement that if you don't have your own mail server certificate, you would only be able to send mail from whoever hosts your mail.

The server "vouches" for the user, allowing perhaps better digitial identity while also preventing the need for Verisign-like large scale key servers.

Once you can unambiguously identify which server spam came from, you can blackhole individual servers with much greater ease. 

I can picture all kinds of abuse if you paid 1 cent per message sent and received 7/8 of a cent per message received.

Of course, we could fix all of this by preventing ISPs from signing pink contracts with spammers or charging negligent ISPs per spam message sent.  But that's harder when the ISP is in China or one of the Koreas or places like that. 

Flamebait Sr.
Tuesday, April 22, 2003

Yes, you receive postal junkmail, probably about 2-3 pieces a day. Most direct mailers assume that sending one of these things out costs $1. So unless spam email cost more than $1, you couldn't eliminate that particular type of mail.

But I don't think legit marketing is what bothers people about most spam. What bothers people is scams, penis enlargement, viagra, pornography in your inbox, nigerian frauds, fake credit scrubbers, and all kinds of other scams and frauds that are just offensive to look at.

If you are going to try to eliminate all commercial messages, you have a much higher bar, one that is likely to be unachievable. Can we settle for eliminating the worst stuff?

Joel Spolsky
Tuesday, April 22, 2003

"Can we settle for eliminating the worst stuff?"

solved problem, bayesian filtering works.                     

choppy
Tuesday, April 22, 2003

"A typical user sends, what, 100 emails a month? 200?"

I send at least 50 emails per day with an average of at least 3 recipients per. Times 30, that's 4,500.

Much of it is work email which presumably could be cut out. And replies could possibly be exempted. But 100-200 per month seems extremely low.

pb
Tuesday, April 22, 2003

[If you are going to try to eliminate all commercial messages, you have a much higher bar, one that is likely to be unachievable. Can we settle for eliminating the worst stuff? ]

I agree that setting the bar at "no commercial messages period" is unrealistic and we do need a way to reduce the amount of spam and the ease in which a spammer can operate. I would rather have ads for Amazon.com than cheap viagra pills and penis enlargement but nevertheless the hurdles we must overcome to implement a micropayment system world wide would be tremendous.

Also, certificates are now in use, just not widespread. If everyone had a pgp sig (or something equal) we could verify and whitelist legit senders and scrutinize all others. But that system has yet to be used widespread, especially by the common man.

More likely, if Internet history tells us anything, there will be many disparate systems used for defeating spam and widespread use of one or another by the average user will move forward to become standard.

Also, spam is not just a problem for email, Instant Messaging is now a hot target and the same measures must be taken to reduce the amount of junkmail one receives. According to Gartner, Instant Messaging use is expected to surpass SMTP by 2005.

Ian Stallings
Tuesday, April 22, 2003

What I *really* don't understand are the efforts to circumvent spam filtering. One recent tactic I've seen is using the spelling "seks", presumably to get around filters that look for "sex"

Uh, if I have indicated that I don't want to see emails with the word "sex" in the title, what makes you think I'll be receptive to "seks"? Ditto on the "click here to unsubscribe" links that seem to only verify that you're a legit email address...

It seems the real failure is that spammongers seem to be concentrating on maximizing their audience instead of maximizing a *receptive* audience.

In a related note, everyone ready for the upsurge in unemployment when the US "do not call" registry goes into effect?

Philo

Philo
Tuesday, April 22, 2003

The problem with a micropayment system is setting it up.

Vast numbers of email users do not even have a credit card, and most of those that do are not going to give it out on the web.

There would have to be one system for the whole world, there would have to be prepaid cards available everywhere, and every computer in the world that might connect to the Internet would have to be reconfigured for the new system.

Frankly it would be a lot easier to buy a few dozen unused missiles from the local neighbourhood arms dealer and take out the twenty or so outfits responsible for 90% of the Internet's Spam. We could even bid for the contract to rebuild the guys' houses afterwards :)

Stephen Jones
Tuesday, April 22, 2003

Ian is right:

The simplest strategery is to whielist PGP certs. 

Nat Ersoz
Tuesday, April 22, 2003

The problem with PGP and the like is that you need either a web of trust or a certificate signing service.

Each one is very hard to maintain for the set of every user on the 'net without charging for the effort of maintenence (and, of course, the inevitable key cancelation requests and tech support)

I'd suggest a PGP-like web of trust for each mail server and then each mail server managing authentication with it's own signing or web of trust system.  The second one is pretty much the usual notion of user accounts that any ISP already uses.  Adding a web of trust between mail servers would be the main effort, which I believe can be done without necessitating exchange of money.  Plus, all hosting customers in good standing with their ISP who don't spam would get their keys marked as trusted by the ISP as a matter of process.

Flamebait Sr.
Tuesday, April 22, 2003

What RIGHT does any of you have to stifle a mass-mailer's business? You're just a bunch of limp-minded liberal techies who begrudge someone making a few bucks, aren't you? Let the market sort out mass mailing, the last thing the internet needs is more rules and regulations from some self-appointed guardians.

small troll
Tuesday, April 22, 2003

Small troll,
                We don't want to stifle mass mailers business - we want to nuke them. If you promise to standi outside their servers as a human shield then we'll hold on until you're in place

Stephen Jones
Tuesday, April 22, 2003

Stephen, LOL! Let's do it!

Bored Bystander
Tuesday, April 22, 2003

What right does a mass mailer have to make money off the internet backbone he didn't pay for, and cost the receipients of his junk money?

Chris Tavares
Tuesday, April 22, 2003

I am of two minds on this issue:

1) Normal market forces will eventually work this out.

2) If there must be a payment plan, it should work like this:  you receive an eMail.  If you didn't want the eMail, the sender pays.  If you did, nobody pays.

Norrick
Tuesday, April 22, 2003

Micropayments are not the answer, verification of origin is the issue.

With SMTP it is trivial to fake anybody else's address. If the servers would communicate so that we could actually know that a mail really came from somebody@somewhere.com, it would become very easy for ISPs and users to reliably block particular addresses or domains.

With a verifiable origin, spammers would have to change domains very very often which would become very costly for them.  Their identities would also be more easily exposed so they could be targeted for prosecution.

T. Norman
Tuesday, April 22, 2003

Actually, Joel did a better job of explaining this the first time he made the suggestion.

His suggestion is that you could pay, or not pay. Anybody who sent you "free" e-mail that wasn't on your whitelist would be immediately suspect in your e-mail program. Anybody who paid would be considered whitelisted by mere fact that they paid (and you'd get your cut of the payment for receiving it).

I'm still not sure that that's better than Bayesian filtering, but it is an interesting idea.

~~~

Verifiable identity has a lot of problems on its own, too, mostly in terms of having a reliable web of trust that isn't ungodly costly to run.

I don't think web of trust on the servers is enough, because nobody at my ISP knows whether I'm going to be a spammer. The end result of me spamming is that they get penalized for me doing it (not trusted any more). If you say that can't reasonably get penalized, then there's not much point in using the system.

Plus, I believe there are a lot more SMTP servers than most people even suspect. For example, web sites that use a local SMTP server to send out notification e-mail messages.  I use the local IIS SMTP server on my laptop for performance and ISP annoyance reasons (like many, they won't let me connect from anywhere except home, and supporting SMTP authentication is apparently just too much trouble for them to worry about).

All these would need to participate in the web of trust. Just because it's not end users doesn't mean it's going to be easy or cheap.

Brad Wilson (dotnetguy.techieswithcats.com)
Tuesday, April 22, 2003

Wow, Joel advocating a rewrite from scratch!  Has the world gone mad?  Why not just make small piecemeal fixes to SMTP.  Its been out there for *YEARS*.

I haven't heard any good reasons why SMTP itself couldn't just be fixed.  Is everyone just afraid of it cause its all messy, ugly, grody and old?

Eric Johnson
Wednesday, April 23, 2003

There is absoltely no need to throw away smtp. All that is required is that people secure their servers from being open relays (well... duh!) since this is how most UxE is injected. Of the two suggestions that Mailshell put forward, the first is trivial (smtp-auth, RFC 2554) and already implemented in (at least) some of the most common MTAs, and Hotmail, for one, have already implemented the second I believe.

There is even a defined method for adding functionality to smtp (esmtp - rfc 1869) if, for example, more "security" is desired. It seems the author of that piece wasn't wearing his clue hat when he sat down to write.


Wednesday, April 23, 2003

In physical mail, the deliverer pays, the receiver chooses whether to throw it away or read it.

In email, they both pay and the receiver pays whether they want it or not.  Micro payments are already happening they just aren't discrete enough to know how much an individual email costs.  Some of those costs are buried in bandwidth, some in disk space and some in attention span.

With adaptions such as Bayesian filtering, the cost that spam generates is now just bandwidth, if the filtering is pushed back up the stream to the server (given there is a generality of spam identity), then the bandwidth cost becomes even smaller.

SMTP doesn't need replacing at all.

Simon Lucy
Wednesday, April 23, 2003

[What RIGHT does any of you have to stifle a mass-mailer's business? ]

There is a difference between being on a mailing list I approved of and getting penis enlargement shoved in my face. I have no problem with Bass Pro Shops sending me an email because I signed up for their mailing list and they want to send me an offer for their latest gear. I actually like targeted ads when I subscribe to a list and I will tolerate an ad from a place I bought something from. But unsolicited ads are a nuisance and should be treated as such. Because it's my money being spent, not just the senders, when I download the message.

Ian Stallings
Wednesday, April 23, 2003

What if you have a (rather large) whitelist, and people on that list do not have to pay.  People not on that list have to pay via the 7/8ths scheme.  That way, if you and some buddies send 30 emails a second to each other, you pay nothing for the privilege.

This could potentially be combined with the "sender didn't want to pay so the filter is more skeptical about this email" scheme.

Michael Kale
Wednesday, April 23, 2003

"getting penis enlargement shoved in my face"

Just what my Missus says ;-)


Thursday, April 24, 2003

Thought: if people are not willing (like me) to connect their email account to their credit card, how about another form of optional outbound authentication?

Let's imagine a system whereby some sort of one-way mathematical hash based on the sender and recipients of message is added to the headers. It takes time to compute, but can be trivially checked. It adds a few seconds background processing time to your PC per email and thus makes mass-mailing, in the way that Joel suggested, uneconomical.

Not everyone would have to use it, but it would act as a guarantor of the email and thus help it to get past Bayesian filters.

Adam in Poland
Thursday, April 24, 2003

Do remember that there are legitimate mass mailings.

Stephen Jones
Thursday, April 24, 2003

Saw this response on a blog. To be frank it does not take issue with replacing SMTP, rather with Joel's comparison of the process involved to that of introducing Euro.

http://www.srijith.net/cgi-bin/trinetre/mt-comments.cgi?entry_id=204

Chee Sung Chou
Thursday, April 24, 2003

Opps the URL should be:
http://www.srijith.net/trinetre/archives/2003/04/24/index.shtml#000204

Sorry about that!

Chee Sung Chou
Thursday, April 24, 2003

Bear in mind that it can be a gradual shift-over, and that client software (e-mail programs) need never be changed at all. Authentication-based SMTP is perfectly acceptable from client to server, wherein an entirely different protocol could be used for server-to-server communication.

Brad Wilson (dotnetguy.techieswithcats.com)
Friday, April 25, 2003

*  Recent Topics

*  Fog Creek Home