Fog Creek Software
Discussion Board




HTTP Replacement

Joel muses about replacing SMTP with a superior protocol.

Whilst we're at it, how about replacing HTTP with a stateful protocol?

John Topley
Tuesday, April 22, 2003

Good god, why? So we can ruin the scalability of the web?

Brad Wilson (dotnetguy.techieswithcats.com)
Tuesday, April 22, 2003

Cannot be done ... the IP stack was designed to work over a unreliable but highly redundant network infrastracture.

Stateful protocols would contradict the above assumptions and would invariably end up in huge problems around error handling and state synchronization.

IMO, keep'em stateless.

Cheers
Dino

Dino
Tuesday, April 22, 2003

Of course there are lots of implementations of HTTP that is "stateful". Cookies are an example of stateful HTTP, in IIS as a "session" for instance.

Dennis Forbes
Tuesday, April 22, 2003

Would the cookie-dislikers be satisfied if there were some sort of session cookie generated by the browser? I'm not too anti-cookies but I can understand the concern.

Re HTTP: it's interesting how folks awlays want to "improve" on HTTP and HTML when the sole reason for the astonshing ascent of the web is the utter simplicity of HTTP/HTML.

pb
Tuesday, April 22, 2003

Although the cookie contains status info, the "cookie protocol" itself is stateless - the client and the server do not synchronize around cookie states.

Plus servers should be able to work without cookies since users can always turn cookies off on their browsers. Alternatively one can use URL rewriting to implement user sessions (or any other method that embeds the session id in the request).

Now the "session protocols" are stateless too since they do not assume a certain state neither on the client nor on the server. Servers rather timeout and drop the session then synchronize with the client. On the client side, the client knows his session id, however it would have to reconnect if the id doesn't check out.

Stateless protocols are the price we pay for trying to build reliable services on top of unreliable transport. For more info try "Internetworking with TCP/IP " Comer, Doug.

Cheers
Dino

Dino
Tuesday, April 22, 2003

I don't think some of you are taking my post in the spirit in which it was intended and the spirit in which Joel's original proposal has been taken. Do you not think that replacing SMTP as Joel outlines would require serious re-engineering of TCP/IP? We all know that SMTP and HTTP are not going to be replaced any time soon, so it's a hypothetical scenario which means that objections based on implementation details are invalid.

HTTP and HTML are designed around static content which means that you have to resort to kludges such as cookies and URL rewriting for dynamic Web applications. I'd be very surprised if anyone argued that it's an elegant way of doing things; especially compared with how user interaction works with a desktop GUI application.

John Topley
Tuesday, April 22, 2003

P.S. Good response from Dino.

John Topley
Tuesday, April 22, 2003

pb,

cookies are the only way to implement long term sessions (lasting weeks, months, years). Then I guess cookies are here to stay.

From a different perspective, security related data must be highly volatile (or it wouldn't be secure) and should not be stored on an unsecure machine at any point in time.

Cheers
Dino

Dino
Tuesday, April 22, 2003

John,

our networks can provide a connection reliably but cannot sustain the connection equally reliable (but less ;). This because a connection request can take any number of paths in the network, while an established connection must work over one single route.

Therefore we are better off to try and connect more often than to try and keep a connection for a longer time. Simple as that.

This is why only request-response protocols are the ones that work with WAN - ie internet.  HTTP, SMTP may change in features but not in shape: they'll survive as req-resp.

More likely to change are protocols like FTP which require connections for a long time.

Cheers
Dino

Dino
Tuesday, April 22, 2003

Where does Joel outline replacing SMTP? In his latest article he merely links to a fairly shallow article on the Ziff Davies site about using some kind of protocol involving micro payments in order to cut down on spam.

And who uses HTML to design dynamic web APPLICATIONS?

Rich client GUI's are better because they are not limited by narrow bandwidth, and because they don't have to work in a sandbox.

Stephen Jones
Tuesday, April 22, 2003

While we are it, why don't we go with the Semantic Web....:-)

Prakash S
Tuesday, April 22, 2003

John,

No, I don't think replacing SMTP would require reengineering TCP/IP. However, in your straw man, you've made one fundamental mistake: SMTP is broken, HTTP is not.

Brad Wilson (dotnetguy.techieswithcats.com)
Tuesday, April 22, 2003

"Larry Seltzer wants to throw out SMTP and start over. I have to support that." from http://www.joelonsoftware.com

John Topley
Tuesday, April 22, 2003

View the world as layers, not one single location.  Otherwise, you will have a leaky abstraction.

You have a lot of guarantees about your local PC and perhaps the high-speed cluster it is part of.  It will never go down, if it does, you have BIG problems.  Communication is roughly instantaneous.

You have some guarantees about the local network.  You will most likely have at least 10 megabits shared, perhaps more.  The connection will be up as long as you are docked / within range of the AP / not experiencing network difficulties.  Servers stay up.  Connections can persist.  Latency is reasonable.

You have no guarantees about the WAN.  Stuff dies, servers dissapear, IP addresses change or are NATed, etc.  You have somewhere between 300 bits and maybe, if you are lucky, a few megabits.  You could have a full second of latency.  You could go in or out of range of a cell tower.

Trying to make stateful HTTP for rich applications is a leaky abstraction.  Stateless works great in the WAN case because it is built under the assumption of a best-effort network.  Trying to build a proper "rich" application without keeping in mind that your network is anything but best-effort (i.e. if you want a rich application, all of your UI logic needs to be local and the data needs to be cached with synchronization) is about as bright as building your house in a flood plain.  Sometimes it works, but just wait for the inevitable rainy season where it all gets washed away.  Writing apps with a stateless HTTP over the Internet would be about as fruitful as writing apps with DCOM over the Internet -- both are prime examples of a leaky abstraction.

Flamebait Sr.
Tuesday, April 22, 2003

I sense sarcasm in Joel's comments...

ease up
Tuesday, April 22, 2003

Dear John,
                  Support is hardly an outline. In fact Joel goes into more detail in a post on this forum but there is no indication that he has even began to think of the difficulties of changing the present system.

Stephen Jones
Tuesday, April 22, 2003

1) Invent new email system that solves SMTP's problems

2) Run NewMail and SMTP side by side at the same email addresses.  Mail gets "bonus non-spam" points if it arrives over NewMail

3) After time, since NewMail is vastly superior to SMTP, SMTP email goes down to a trickle and some people start turning off SMTP alltogether, hastening its demise.

4) ???

5) Profit

Richard Ponton
Tuesday, April 22, 2003

OK, I'm convinced. I'm going back into lurk mode before I humiliate myself further.

John Topley
Wednesday, April 23, 2003

I agree with John, actually.  I fully believe that web app programming is a tower of Babel made up of one kludge above another.

HTTP was orginally conceived by Berners-Lee as a mechanism to deliver static pages in one direction, with a markup language which could be displayed differently on different platforms.  Later CGI made the communication two-way; CSS turned HTML from a markup to a page design language.  Javascript and Java applets were bolted on to provide computation on the client side.  If you want anything in the way of animation, however, you need Flash.  Now the current generation of web app tools, such as ASP.Net, go to enormous lengths to make controls displayed on the client appear programmatically as if they reside on the server, pulling an HTML refresh while posting a big block of opaque data to indicate, for example, that the user clicked on a checkbox.

In the meantime, various vendors of browsers have had to struggle, implementing different sets of this functionality in different ways.  The signs of strain are already there.  What is really needed is to abandon this entire notion of different pages rendering HTML differently -- we need a smaller set of primatives: text HERE, rectangle HERE, line THERE -- something along the lines of what Terminal Services or X Windows do.

For the most part, networks these days are reliable and the lowest bandwidth typically found today is 56kbps (wireless being the exception).  Terminal Services and X Windows work decent over such a connection.  A highly graphics-intensive program might creak on a low-bandwidth connection, but the same problem exists with flashy web apps and HTML pages today.  Since networks are pretty reliable these days, I don't really see why distributed apps need to be stateless.

Alyosha`
Wednesday, April 23, 2003

There are other ways of maintaining state across connections and that's to treat the server as the proper place for the repository of those states.

That of course means that a raft of validation checks have to be done but it isn't impossible to do.

Simon Lucy
Wednesday, April 23, 2003

Now there's an interesting idea Alyosha - make rich internet client/server apps that use X11 as a transport rather than HTTP... (just don't call it X11, make up some marketing name like Super Interactive Web Protocol...)

Dan Maas
Wednesday, April 23, 2003

The lowest bandwidht typically found today is 24kbs, which is what you get if you are more than a certain number of kilometers from the exchange.

On a normal telephone line a 56kbs modem will connect at between 45 to 49kbs, but you can only send upstream at 33kbs. Much more importantly is your average download speed and I have rarely found that more than 2.5kbs for one connection. It is quite common to get periods of a minute or two when downloading comes down to a crawl.

I have used modems in England ("free" ISP but they all are now), Saudi and Sri Lanka, and there does not seem to be significant differences. Each country has its own particular faults.

Stephen Jones
Wednesday, April 23, 2003

"I agree with John, actually.  I fully believe that web app programming is a tower of Babel made up of one kludge above another."

sort of like how unix and NT themselves are evolved piles of kludges, or how the human brain (and the rest of the human body) is a pile of kludges. anyone still use a lisp machine? i didn't think so. kludges work.


"Now there's an interesting idea Alyosha - make rich internet client/server apps that use X11 as a transport rather than HTTP... (just don't call it X11)"

this already exists. it is called "VNC." also, basing anything on X11 is probably the worst idea I've heard in a while.

choppy
Wednesday, April 23, 2003

"apps that use X11 as a transport rather than HTTP"

Hey, I know the whole cryonics thing was cool in the late 70's and early 80's, and by God who would have believed it would really work. It was even funny at first, all these IT defrosties just going about their business as if the world after 84 had never happened. You know, working on the most bombastic kludge rewrites of 70's UNIX implementations and the like. But then some politico radicals got involved (got to love that neo-Orwellian newspeak) with smooth marketing pulling some sort of extra slippery Buck Rogers on some Finnish dude and slick branding with some "I'm maladapted but cute" animal for the puberty hormone swigging crowd, and it got less and less funny. Now they want all of us that learned how to stop dragging our digital knuckles during the 90's to stoop down again? Count me out for the ride pal.

Just me (Sir to you)
Friday, April 25, 2003

*  Recent Topics

*  Fog Creek Home