|
Development has become insane complex
Software development has become insane complex.
It has become insane complex and therefore it is less and less fun
for me.
Oh my dear companions let me cry out loudly,
let me have a few shots from the hip with hail and
bore you a little with my 2 cents.
Out of college I got my first professional programming job.
I created applications with RPG (later also ILE RPG) on an
AS/400.
After I had few years programming experience I was asked to
create a great dataware-house collecting *all*
medicine prescriptions in my country.
My boss bought me a brand new AS/400 that costed about 250 K
dollars (nowadays i-series only a fraction of that)
and gave me all the freedom to generate a solution.
Then I started programming. Green characters,
on a "Russian television set".
After four months I was ready to let the data roll in from 1500
data contributors.
In all I created a few hundreds of RPG and CL programs.
During the development I had one thing in mind:
make it as simple as possible, never
try to be the clever boy by implementing tricky stuff.
From the very first moment the medicine dataware house and the reports
I generated with it, were a great success. Our government was willing to pay
a tremendous sum of money to have data covering about 95 percent
of all the prescripted medicine.
I did the programming and a very smart guy sold the project
to the outside world, a job in which he was very successful.
That my friends were the happy days, the days that I felt
tremendously productive and was able to create quite big
systems just on my own. From start to finish.
Untill today every government decision is still based
on data from the system I built.
Then the internet and OO happened.
Making web-applications.
I was clueless. I started by reading a book on
Java, read the gang of four book on design patterns,
developed programs, tried to get grip on
the new tools and design principles.
Of course I also got more involved in the world
of Microsoft software, since all projects I did used
Microsoft as server and also as client.
The first big smash for me was the instability
of the Microsoft software. (I won't be religious on this)
Even after minor events (install some software)
we had to reboot the NT server.
Oh silly me, I was laughing at the
guy who said to me that we had to reboot the server.
Reboot the server?
I thought he was joking.
In six years of using an IBM AS/400 system
we never ever had to reboot because of some failure.
That did not happen.
All the nice things that were handled by OS/400
I had taken it for granted and I have to admit,
I did not enough appreciated it at that time.
What also struck me from the beginning when starting with
web applications:
the insane complexity of even the most simple programs.
I studied the J2EE pet shop from SUN.
Every line I read I think: you don't
want this, it's just too much.
When I tried to develop my own java window
applications I got more respect for the developers
at Microsoft.
Now I can understand a little bit why for example an
editor as Word takes hundreds of man-years development
time. It is just tremendously complex.
When you open MS Word you have more buttons
and switches then in the cockpit of a Boeing 747.
When you consider the complexity of their
software products you can say that
the MS guys did quite an impressive programming job.
But at MS they don't have the ten years Joel assumes
it takes to get it right.
Bet let me get on with some moaning...
In web application development you need to
have an awful lot of knowledge:
you have to know the ins and outs of the database you use,
the application server, the web server, the
GUI components Swing and AWT, the different design patterns,
XML, connection pooling, caching algorithms,
proxy servers, Struts, communication protocols,
virtual machine configuration, IDE characteristics,
ant, servlets, JSP's, HTML, several SQL variants,
NT basic administrator tasks etc. etc.
Then we have the leaky abstraction law from Joel
meaning: you really have to understand it to make a corrections.
More often then not you are in tata without tutu (also Joel)
Look at the java API. Who on this world
is ever capable of mastering such a tremendous
amount of information and apply it successfully?
I know the answer, nobody.
I will just cost you thousands of dollars to
develop something useful.
I am only involved in the Java swamp, I reckon with
.NET you got the same shit.
For each new project lots of parameters change.
"We don't use DB2, we use Oracle 9i here.
But we have some performance problems"
All ins and outs of Oracle database tuning,
there are specialists who do that for a living.
But nobody stands up and says: enough is enough.
So what?
An IBM AS/400 rpg program can hardly be compared
to a java web based program.
It is complicated but what is your point?
Eh.. well let me end with a few statements:
I think it is hard to be happy when you are
a web developer.
The tremendous complexity can never be controlled.
When you can not control your stuff, you are
going to be unhappy.
The tons of knowledge you need is outdated in two years.
Keeping you an the run forever.
When you are a big company,
make strategic choices about technologies you
want to use. Every technologie wave you let in
will cost you a tremendous amount of
money.
My advice: buy a mainframe, use Cobol.
Have one DTP package installed on the companies
secretaries desk.
For presentations for people who can not live without pictures.
Don't let manager types burning 200 dollars per hours going wild
in a 100.000 buttons 747 Word cockpit.
Buy a Mercedes (I did), drive 438 thousand kilometres in it (I did)
and get a feel of what quality of products can do to your live.
Kiss every citizen in Rochester just for the quality of the AS/400
technicians that live there and developed the system.
That's of my chest I'm feeling already a bit better.
Hope I'm not the only one sometimes sick of technology..
Wish you all the best
John Fisher
Wednesday, October 29, 2003
> .. make it as simple as possible, never try to be the clever boy by
> implementing tricky stuff.
> ... I did the programming and a very smart guy sold the project to
> the outside world, a job in which he was very successful.
Great job! :-)
I have a similar story in my protfolio.
A small note: "do right things from the early beginning. Then complex project will be more manageable/flexible in the future."
Evgeny Gesin /Javadesk/
Wednesday, October 29, 2003
Can you be more specific about what in the RPG
environment made things simpler? This goes
a long with a question i've had, what was really
wrong with dbase/fox pro/power builder?
son of parnas
Wednesday, October 29, 2003
It probably is insane complex (sic) for someone who can't tell the difference between an adjective and an adverb.
Ouch!
Wednesday, October 29, 2003
dbase/fox pro/power builder....
amen brother. Never tried power builder, but dbase & fox pro made data processing a freakin' snap.
Wednesday, October 29, 2003
Sick of Technology!!! I have barely started technology. I am so encouraged by your words as a fledgling IT person. In addition to the gazillions of acronyms and "witty sound good" talk programmers use, there is every type of software with numerous versions. And every person has a different opinion, this ones better than this one, no your stupid that one's better and so on. I'm SCCCAAARREED!
However, as I am part of a mismanaged mobile software team, and feel astoundingly stupid at every progress meeting we have, I have decided not to fight them, but join them. I begin training next semester.
Maybe the answer is that we don't need to know everything, just the bits we can use.
So my thanks to you John, I feel a little less lonely in the vast technological universe.
TES
Wednesday, October 29, 2003
You sound like a lot of COBOL programmers I know. Maybe you should think of a new career or just retire?
GenX'er
Wednesday, October 29, 2003
I'm not saying RPG makes things simpler.
In fact it is quite an old (position based)
dinosaur, simple but effective.
Never used Dbase, but the code I saw looked pretty straigthforward.
About PowerBuilder I hear my collegues say that they
were completely happy and referring to it as the good old days.
John Fisher
Wednesday, October 29, 2003
Hey ouch, you need to reference the error, I am 38 and still don't know the difference. You can say I sucked at english. I am always amazed by the grammar police, they always dismiss the content by attacking the how it is presented.
hjm
Wednesday, October 29, 2003
RPG doesn't really make things easier. I think it's only the percieved notion that this guy is getting because that is what he learned first. I code VB, C++ and RPG/ILE RPG all day long. In fact RPG I, II, III are extremely cryptic to people who are not used to using Indicators to turn lines of code on and off. SDA (Screen Design Aid) isn't exactly like drawing on VB form with a mouse you know.
The argument of system complexity usually comes from people seeking empathy for their percieved problems.
I realize that IBM prides itself in the stability of it's AS/400 and OS/400, but I don't think you can compare apples to oranges (MS/IBM).
Wednesday, October 29, 2003
I will try to make a defense for how it is. The issue is one
of choice. It would be simpler if we could say everyone
should use RPG and a josh9000, have one religion,
have one political party, have one set of clothes etc.
Once you introduce choice you get complexity
because you immediately need to wrap choices
with layers so your choices will work anywhere.
Java is just another language. OO is a way of programming
and if you don't do it already you have to learn. The
GOF patterns are programming techniques as a
professional you should have known or recognize.
We have html to and css to protect us from a particular
vendors. The same for application servers.
Other os's aren't as stable as AS/4000 because they
are cheaper and work on more types of machines
and nobody really cares about reliability that much.
As for the cost, you can do it all for free. Not sure where
the 1000s of $ is comming from.
Freedom ain't easy.
son of parnas
Wednesday, October 29, 2003
John,
Check out Rebol [1].
Rebol/View shows that GUI apps don't have to be so complicated. One man can made an application.
It still has a way to go, but it does show that there is an alternative.
[1] http://www.rebol.com
Ged Byrne
Wednesday, October 29, 2003
As for the costs Son of Parnas, I just meant that dealing with complexity is very time consuming.
Somebody is going to pay for that.
I agree with you, freedom ain't easy.
Giving everybody a Lada as the tried in the USSR, did not work out that good..
regards
John Fisher
Wednesday, October 29, 2003
What's a "Russian Television Set?" Old-fashioned character-based display?
azazello
Wednesday, October 29, 2003
The AS-400 and RPG and Cobol on anything are the best BUSINESS platform/languages ever designed. All the relatives and decendants of C are less capable. That is what is so stupid about computer technology. We switch just because something is newer. Guess which one takes up more bandwidth over frame relay and is less productive for order takers? GUI or Green Screen. Second question: What are vendors pushing?
It must be something we are drinking.
Mike
Wednesday, October 29, 2003
What makes Cobol the best? Is it better than increasingly common high level languages like Python or Lisp, which have (or are building) better support for numerical calculations without arbitrary holes such as tiny integers?
Or is it more the platform?
anonymous
Wednesday, October 29, 2003
COBOLs great strength is that it was designed for the problem domain. It was designed to be easily understood by accountants and for coding financial transactions.
Most programming languages seem to be obcessed with the implementation space, whether it is shovelling bytes or declaring objects.
Ged Byrne
Wednesday, October 29, 2003
Progress 4GL has excellent build-in DB support. Way more convinient than SQL. Everything else with the 4GL is a disaster.
The WebSpeed Man
Wednesday, October 29, 2003
John Fisher,
I understand your frustration, and I agree it is insanely
complex. I can't really compare with how it used to be, as I didn't start programming for real until about 6 years ago.
However, the first program I wrote I believe was in Pilot for the Atari 400. I liked programming because it was simple and easy to create something cool. Real world programming, nowadays at least, just isn't like that though.
When I began to learn this first-hand was in college- in CS 101. We had been writing programs in Lisp for the first month, and then we were introduced to "C". While teaching us Lisp, the prof had a habit of referring to the "little man" in the computer, who did all sorts of things in order to perform the instructions we coded.
The day he introduced us to "C", he opened the class by saying, "The little man... is dead." Starting that day, programming became a lot harder. Eventually, however, I learned to swim in the sea of complexity. Developing software is about creating order from complexity, reducing entropy. It's up to us, as the creators of software products, to harness the right things in that sea, and fit them into a neat little package that is simple enough for a "regular person" to use.
The other thing I've learned is that you just can't "know" everything. The more I learn about software development, the more I realize I don't know. All I can do is learn what I need to know, and apply that knowledge as well as I can. Every now and again, I get that same feeling I got when I wrote Pilot programs on my Atari, of creating something cool and simple.
Ken
Wednesday, October 29, 2003
"Hey ouch, you need to reference the error, I am 38 and still don't know the difference. You can say I sucked at english. I am always amazed by the grammar police, they always dismiss the content by attacking the how it is presented."
It should be "Development has become insanely complex". I always worry about the code that people who write such things write, as the attention to detail seems poor. And no, I'm not including non-native English speakers in this because I think he is a native English speaker.
I read the rest of the original post and it seemed very odd to me that he should get this one word wrong when the rest of it is fine. And it's not a typo because he repeats the misake three times.
Ouch!
Wednesday, October 29, 2003
"COBOLs great strength is that it was designed for the problem domain. It was designed to be easily understood by accountants and for coding financial transactions."
I've programmed in COBOL for a large company and I can tell you that COBOL is total crap. It's like programming in BASIC; virtually no structure. All modern (post structured-programming) language features are non-existent. Accounts might be able to understand a 30 line COBOL program; but I've seen 10,000 line COBOL programs (all in one file, of course). And most COBOL programs are 10,000 lines or more; if they do anything meaningful.
Basically any language could do that work and do it better. COBOL is still used because it gets the job done and it's entrenched. I saw code that was older than myself when I was there. Code reviews sometimes involved calling up senor VP's who wrote code when they were lowly IT grunts.
I fully expect that Java will eventually replace COBOL everywhere -- I just might not live to see it. ;)
Almost Anonymous
Wednesday, October 29, 2003
The OS/400 never had to deal with running on every 2$ piece of crappy hardware under the sun with a "the one with the sortest straw has to write it" driver . That makes a HUGE difference.
Was your Windows system at least on the HCL using WHQL-certified drivers?
Just me (Sir to you)
Wednesday, October 29, 2003
Euh .. that came out sounding a bit rude. Sorry for that.
I agree with your main point: the environment in which we work has become extremely complex. Why? Because we want to do complex stuff way before everything has matured to the point of sanity.
Just me (Sir to you)
Wednesday, October 29, 2003
My personal definition of intelligence is knowing what to ignore. True, there's far more information about programming out there than any one person can take in, but for any given application almost all of it is irrelevant.
Being able to spot what's interesting and relevant in any new technology, methodology, etc., and making quick decisions whether it's worth going into deeper or not is a critical skill for our profession.
The next problem, of course, is convincing the HR moat keepers of this. For the most part, how shall I say, they do not share my notion of an intelligent developer, I think.
Jim Rankin
Wednesday, October 29, 2003
Bet let me get on with some moaning...
Holy crap!
nat ersoz
Wednesday, October 29, 2003
The original author *does* have some good points.
How come, in the days of yore (you know, DOS, OS/3x0, VAX/VMS, others) we could create programs which are not functionally any different than the window'd/web'd equivalents of today, and yet *are* simpler to develop. What happened? Is what we have to do today really any more complex than what we had to do back then? Note that I'm distinguishing between what we have to do vs. how we decide to do it (which certainly is more complicated).
AnMFCAndJavaProgrammer
Wednesday, October 29, 2003
Great article John!
I have to agree the the issue of reliability. I have the chance to use some really nice operating systems that were very reliable. Many years ago I used a apple II with the UCSD Pascal system. It was amazing how reliable, and how clean the system worked. I can’t EVER remember the apple crashing when using that system. Just incredible (I wrote a payroll system on that apple II). Who would have said that a AppleII is rock solid? (with the right OS...it is!).
Also, having working in the Pick environment on a lot of Mini computers (68000 based, and a good many Intel boxes also), again, I found the environments to be rock solid.
By rock solid, you can think a design in your head, write the code, and it just works. The programming environments are so clear, so clean that you can master the whole system, and they just work! They are like clear spring mountain water. Pure and clean.
You spend little time “avoiding” things.
This is very important point:
"Avoiding things."
In many new platforms, much of the skill required is learning what works, and what does not work! In these old systems, VERY LITTLE avoidance was required. Today, avoiding doing wrong things is much more difficult!
A few mentioned some PC products list FoxPro. Those systems where again quite clean, but did not have to deal with a GUI. They were also non event driven systems. I did good stint in FoxPro. And, it was reliable, but there was some kludge creep starting to occur (the stacking, and popping of function keys is a good example).
However, in all of the above systems, they were p-code, and they were non event driven. My best programming experiences have all included p-code driven systems. They do seem to be the most reliable systems I have used (hum...this might be a vote for the .net CLR!).
Now, back to this avoidance problem. It was just yesterday that I wrote some code (happened to be in ms-access). However, what was MOST surprising is that after writing some code, building the form, it did NOT behave as expected (of course, forms in ms-access are far more complex then just plan VB forms..which are simple!).
Anyway, in the OLD days, after building a form, and writing some code, I NEVER used to have a “surprise”.
In other words, using my design skills, and building a form and some code, the end result DID NOT work as expected. This never used to happen to me! I am a Microsoft access MVP, and I write TONS of code in ms-access. I would say that I have mastered the product quite well. Yet, I can still come up with designs in my head that DON’T work!
There was no coding bugs in a logic sense, but application did not behave as expected. (there was some interaction occuring that I could NOT have predicted). Once I disconved what was happening, I built a work around!
As a result, a task took about 2 hours instead of 45 minutes.
The answer it seems is that some of the interactions in code today does NOT behave in as MUCH as a consistent manor as it used to many years ago. I think some of the new systems (due to complexity) don’t behave in a consistent manor, and thus waste valuable developer time.
Albert D. Kallal
Edmonton, Alberta Canada
kallal@msn.com
http://www.attcanada.net/~kallal.msn
Albert D. Kallal
Wednesday, October 29, 2003
I lost 2 20 page reports in school on an apple II.
They did crash :-)
son of parnas
Wednesday, October 29, 2003
Ken,
You sound like a person who has a lot of advice to offer to someone trying to learn "what she needs to know". Do you have any tips for a brand new software developer.
TES
Wednesday, October 29, 2003
For that matter, does anybody have any advice to offer?
I'd be eternally grateful :-)
TES
Wednesday, October 29, 2003
Sure. Never bet on a certainty.
How's that?
:o)
Dave Hallett
Wednesday, October 29, 2003
I can state from personal experience that my Apple IIs were *not* stable. They were just as buggy as today's computers. There's signifigant psychological research that states that people view the past with rosy colored glasses. ;)
The only thing to be said is that the older minicomputers and mainframes might have had much better system stability because they used quality parts with built-in emphasis on stability.
AS/400s are incredibly well designed for business applications and being a server. There's a lot of dumb luck there. At the time they developed them, IBM was banking on magnetic bubble memory taking over after a few more years. Thankfully some of those design decisions also made a lot of sense with RAM paged to disk. When I had the way they worked explained to me, I was in awe. Although I've never actually used most of the layered products, so I'm not sure about RPG and such.
Flamebait Sr.
Wednesday, October 29, 2003
>> Maybe the answer is that we don't need to
>> know everything, just the bits we can use.
TES, that answer sound like it'll work. :)
The bits I use have got me through an awfully large variety of tasks, and my list of core useful things is:
- Perl
- some kind of GUI system (I use tcl/tk, but there are a LOT out there, and Java works for this too)
- Java, C/C++ (not sure how ascendant C# is yet)
- HTML, XML, Javascript, all things webbish; PHP is up-and-coming, IMO.
- SQL, as much database info as you can handle
These aren't in any particular order, really, but they've all served me in good stead in some way or another.
Good luck, HTH.
van pelt
Wednesday, October 29, 2003
>I can state from personal experience that my Apple IIs were *not* stable. They were just as buggy as today's computers. There's signifigant psychological research that states that people view the past with rosy colored glasses. ;)
LOL!...I am laughing as I read this!
The other poster:
>>I lost 2 20 page reports in school on an apple II.
They did crash :-)
Yes, in fact, when you ran the Apple DOS 3.2 (or whatever it was called back then), the little 8 bit box with 64K ram was not what you call a stable platform. Games and applications kill the box quite easy!
However, when you dumped apple DOS, and ran the UCSD Pascal system as the OS, the computer was complete transformed into to a VERY reliable system. Not only that, you got extra goodies like type-ahead keyboard (that was a big feature back then!). I remember how great it was...since even when very young I was a fast typist
So, much of my point was that every p-code system I have used (and that includes both the OS on systems like PICK and UCSD) tended to be VERY reliable systems.
A program crash was limited to the p-code interpreter, and I can’t remember a program crash brining down the OS in this regards. You could break any program, and gracefully shut the program down.
So, the AppleII when running a p-code operating system like UCSD tented to be very reliable. And, hey, the resulting p-code could run on a IBM pc also! The p-code java idea is not new....we had this back in 1981 with the Apple II!
The platform was rock solid when running UCSD....and that an’t some old war memory!
Albert D. Kallal
Edmonton, Alberta Canada
kallal@msn.com
http://www.attcanada.net/~kallal.msn
Albert D. Kallal
Wednesday, October 29, 2003
Ouch, sorry for the misspelling, I'm not a native English speaker. (I just keep on trying to improve it)
Almost Anonymous,
Every language gives you the opportunity to make a complete mess of your programs.
COBOL is well known for that. I think this might be a reason: with no formal IT education you can quite easily start coding. Of course this will lead to accidents: 10.000
lines in one source file.
Of course there is no silver bullet when writing whatever application.
The price of the total freedom of every component in your
software development process makes it to a very hard to control it. To my experience boardroom members very much like the words: Generic Solution. Sound like intelligent and
good investment.
This boils down to the developers, where very often the
accidents happen and disasters are born.
My unasked advice:
When you hear a developer use the words "generic solution" press the fire alarm, call security and have the person removed before it affects others.
It is fun to read all your replies,
kind regards
John Fisher
Thursday, October 30, 2003
TES, The best advice I can offer is to learn on demand when it comes to specific technologies. Don't waste a lot of time learing the ins and outs of a speicific language/platform/protocol unless you know you will need it. That being said, I think it is definitely worth learning general skills- coding practices, design, etc. as much as possible. Unfortunately, the best way to learn this stuff is through experience, ie- you need to screw up a few times. Of course there untold numbers of books, periodicals, web sites and stuff that can catalyze the learning process. Oh, and last tip, don't be afraid to challenge all of the stuff you read and are told. Occasionally I do it the "wrong" way on purpose. You can learn a _lot_ by doing this. Hope this helps.
Ken
Thursday, October 30, 2003
"Look at the java API. Who on this world
is ever capable of mastering such a tremendous
amount of information and apply it successfully?
I know the answer, nobody."
This was exactly the reason I developed http://www.LingoLanguage.com - development has become too complex and it's not necessary.
You have to remember why Microsoft and Sun keep releasing new versions of development tools. It's *not* to make our job easier, it's to ensure their business survival by keeping us locked in. Microsoft compete by a continual upgrade cycle where you have to run to stay still. Joel wrote a good article about this, with some military analogy about always moving forwards.
As soon as you realize the goal is lock-in and not developer productivity, Microsoft's product range makes a lot more sense.
Bill Rayer
Thursday, October 30, 2003
Many thanks for everyone's advice. It's good to hear some candid appraisals of the software industry from experienced programmers. I look forward to reading more of your comments throughout this site.
TES
Friday, October 31, 2003
I like FORTRAN ;)
Ros
Friday, October 31, 2003
I know FORTRAN well enough to hate it. I know COBOL well enough to hate it -- which takes exactly one keyword -- EQUALS.
Flamebait Sr.
Friday, October 31, 2003
SW development is harder today because our programs are CLOSER to the non-standard, untrained human USERS.
It's harder to interface with a human because the human side of that interface is very VARIABLE. Each human interprets things a bit differently and has different preferences. Essentially, you have a floating specification.
So, in the old days, when only a small number of trained users were using your program, your program could be simpler and (perhaps) not have as good a UI. Today, if you're writing software for "grandma" (as I do), the UI challenge is much harder.
The good news?
There are a more grandma's in the world than there are trained computer operators. So, while programs are harder to write, they can reach (and help) more people. Which can make you more money.
It's also more satisfying (to *me* at least) to help 1,000 stroke survivors relearn how to speak than it is to help a company automate it's billing records.
Entrepreneur
Wednesday, November 5, 2003
BTW, the Mac folks predicted this all decades ago. They called it "democratizing technology" bringing what computers to the average person.
Little did they (or we) know that the challenge would be the average person, not the technology.
Entrepreneur
Wednesday, November 5, 2003
 Recent Topics
Fog Creek Home
|
