Fog Creek Software
Discussion Board

Back to basics - Need for Speed?

A huge thread resulted from Joel's Back to basics.

Going thru it makes me realise how many code warriors (in the positive sense) lurk in here.

So I want to know, in your opinion, what kind of software requires the most speed (+optimizations) and how current CPU architectures and language technologies affect you.

I work in the real-time computer graphics field, so naturally I think this sort of software requires the most speed optimization. The pathetic ratios between CPU speed and memory speed affects me the most, simply because of the tremendous amount of data that needs to be moved every frame. This is an area where SDR and DDR memory makes a big difference. At the API level, the graphics hardware has advanced so much, that suggestions are being made to allow an application to write directly to driver memory to reduce data copying and reformatting. This bold suggestion is made by the reputable 3DLabs in its OpenGL 2.0 proposition. I remember suggesting this in the mailing list few years back and was flamed to dust for the security issues it brings up, but now it seems like a good idea... At the language level, when I use Java, I sorely miss (and curse Sun) unsigned bytes since it is critical to manipulating images at a pixel level.

Rex Guo
Monday, December 17, 2001

Billing/Rating Systems (expecially Telco ones).

While I will not claim this to be the most performance intensive they would certainly make a top 10 list. Note speed is not required so much for response times as for throughput.

Performance problems are due to three reasons.

1. The sheer number of records to process (often 100s of millions of records in a single run)

2. Marketing departments ability to come up with weird and complex rating rules greatly outstrips the software's ability to process it.

3. Aggregation/Synchronization points which defeat divide and conquer algorithmns.

The basic solution is to divide the problem up so you can throw multiple cpus and/or servers at it. However this does not always work. It is quite annoying to see 23 CPUs idling wating for 1 frantic CPU.

Again RAM to CPU speed is one of the biggest performance problems. Rating/Billing (like most real applications) is very data intensive, most operations involve trawling through data and . It is interesting to note how IBM servers will out-perform much larger servers from HP and Sun purely because they have faster RAM. (Don't get me started on the E10000).

Disk access is another problem as the system needs to access large amounts of data to rate events and then has to write the details back. This usually gets solved by elaborate cache methods but these eventually get blown by the sheer amounts of data to be processed.

The most successful systems usually use C/C++ or Cobol. I have seen attempts done with Java and other 4gl languages but it simply does not work. Java's memory management is a problem here as it does slowly 'leak'. After running constantly for a few hours it will eventually leach all the memory out of the system (and all the file-handles as well).

Julian Bullock
Monday, December 17, 2001

Could you explain further what you mean by Java slowly leaks ? Do you mean that the gc doesn't work properly ?
Which are your experiences there ? I am really interested in feedback in that area.

Philippe Back
Tuesday, December 18, 2001

Any Java application that leaks memory and file handles is mis-written. Simple as that. I'm no fan of Java, but surely any memory leaks or file handle leaks that were introduced into the JVM itself would be caught and fixed, and would hardly be a case against the language itself.

There are many servers that run Java software for weeks or months at a time without needing to be stopped. Particularly if you leak enough in a few HOURS to bring a server class machine to its knees, then the code is BADLY broken.

Brad Wilson
Tuesday, December 18, 2001

I think the problem with Java is just that the gc algorithms tend not to scale well with simplistic code - try writing some code which repeatedly creates & destroys millions of objects - most JVMs become increasingly sluggish and hog memory.

It's not a leak (the memory is eventually returned to the system and the system will stabilize at some [bloated] level which [hopefully] is less than the amount of system memory) but rather an example of programmers being told they don't have to worry about things which they really do need to worry about.

Server JVMs may or may not be more intelligent about this and you can play tricks like increasing some of the java parameters for things like initial memory limits but these are at best delaying actions.

The answer is reusing objects but that's not what most Java programmers learn. In fact, most of the introductory tutorials & courses are almost gratuitously wasteful on this count. For example, how many examples you have seen which create a new Connection, Statement, ResultSet for each SQL query? This works fine until that code is in the middle of a loop, one of 3 queries executed for each of the 25,000,000 records in the billing database.

(Note that this isn't a java-specific problem - traditional mallocs don't handle churns like this as well as they could, particularly not with multiple threads on multiple processors)

Chris Adams
Tuesday, December 18, 2001

You (Julian) may wish to look at this, about Java's GC scalability:
[ ]

In any large application, the GC must be considered a factor to think about in design, just as you would any other aspect of hardware/software.  For small problems it allows you not to think about memory alloc; for large ones, it is another feature that makes one life nicer but must be understood.

Java has been around for about 6 years, and I believe these projects you mention MUST have started years ago, if they are as complex as you mention.  Therefore we can see that you're talking about immature Java implementations.  Past some point, implementation tells more than the concept itself.

Richard Jenkins
Wednesday, December 19, 2001

Ah the dangers of mis-used words.

I am fully aware that the Java gc does not leak as such, Rather it is a lagging garbage collection, as stated by Chris. I have got into the habit of talking about this as a 'leak' of late as it does appear to have many of the same characteristics, and it is easier to talk to managers about it in this way. My apologies for this.

Note that the instance I was talking about was allocating huge numbers of large objects over a sustained period of time. This is a characteristic of the billing/rating world. It could be avoided by reusing objects (as Chris mentions) however third part libraries are some of the most common violators (esp JDBC implementations). There is only so much you can do.

Web applications (for comparison) are typically more bursty in nature. This means that the gc does get time, every now and again, for it to catch up. That being said, I have seen more than a few Java web applications bought low, running out of file-handles after a large burst of traffic.

Also note that the scalability/effeciency of the gc does differ amongst JVM implementations. Solaris and Windows are good, Linux JVM gc is not so good. I do expect this area to improve, maybe it already has.

Julian Bullock
Wednesday, December 19, 2001

I also was not clear.  The link I gave showed the difficulty of scaling up Sun's recent GCs, which agrees with what you just said.

Richard Jenkins
Wednesday, December 19, 2001

> So I want to know, in your opinion, what kind of software requires the most speed (+optimizations) and how current CPU architectures and language technologies affect you.

You should look at what applications they run on supercomputers.

Telco/data transmission/media servers need speed: the slower the software, the more hardware (more money) you need for a given throughput or volume.

Multi-CPU machines, multi-threading, running from RAM instead of disk, these things are good; LANs, backplanes, Moore's; specialized chips (e.g. image processing), firmware or silicon.

Layering: x data servers and y media servers and z transmission line handlers.

So at the other end from supercomputers, digital signal processors.

Protocol stacks like TCP. Databases. BIOS and OS bootstrap. File systems. Version control systems, and other such end-user applications. Web servers. Email servers. Google. VR. Libraries. Enterprise (airline, banking, car, ...) systems. GUIs <g>. Everything. <g>

Re. languages I haven't programmed in assembler for years; I use C/C++.

Christopher Wells
Sunday, December 23, 2001

*  Recent Topics

*  Fog Creek Home