Fog Creek Software
Discussion Board




Programmers won't automate their own tasks - why?

Maybe I'm not working at the best companies, in fact I'm sure of that, but I am always disappointed by unwillingness of my colleagues to even think about automating their own work.

I have been at a new job for three months and am beginning to understand the work we do enough to start in.

1) Simple shell script to automate the creation of debugging builds.

2) A lot of data creation depends on the relationship of requested events to wall-clock time. I made some scripts to modify data for repetitive testing (Yes I have to test more than once).

3) We do DBA style work on customer sites. I talked with my tech lead about trying to create scripts for the more common tasks. He told me it's a waste of time, he can do them by hand faster. Yeah, sure until the day comes when he makes a mistake and screws up a customer database. Or worse, when I make a mistake and then get hammered for it.

Automated set ups have less chance of error, go faster, and can be reviewed and of course passed around the shop. Someone has an insight, we all benefit.

So why the resistance?

frustrated
Thursday, June 17, 2004

job security

seriously, often the tasks CAN be done by hand more quickly, and automating, then debugging your automation, re-automating when specs change, etc takes up a lot more time than just DOING whatever it is.

I've automated lots of my tasks in many previous jobs and now this one as well, but you have to automate selectively.

muppet is now from madebymonkeys.net
Thursday, June 17, 2004

> job security

That is the joke about our tech lead, that he won't explain. He just sits at his keyboard doing his "magic" all day.

Well 90% of that magic could be done in 20 seconds with automated scripts.

frustrated
Thursday, June 17, 2004

In my organisation, people automate their work all the time. We wouldn't survive otherwise.

Doing it by hand is fine but there are two issues. First, people make mistakes, well tested scripts don't. Second, critically, scripts scale well, manual processes don't.

If you have ever tried to write screeds of software in a company which is growing very fast, faster than it can recruit new staff, you will understand why automation is important. Not least because it releases you from chores so that you can pick from the piles of interesting new stuff coming in.

On the other hand, if your organisation is static or shrinking, then chores make useful trenches.

WoodenTongue
Thursday, June 17, 2004

Write your scripts.

Keep them to yourself.

You will then be, (and more importantly be *seen* to be), quicker, more reliable, more efficient and less error prone than your co-workers.

Might come in handy that.

David B. Wildgoose
Thursday, June 17, 2004

It's just where you're at.  Good programmers automate mechanical tasks because they have low tolerance for drudgery.  I'd say that's a trustworthy marker for the quality of the programmer.

The topic title is asked like the prototypical leading question of "have you stopped beating your wife?"  It's asserting something that might be false, and then asking a question, the mere answer of which implies support of the assertion.  A very lawyerly rhetorical flourish.

Cabby
Thursday, June 17, 2004

I didn't get the same impression from the subject that you did.  I don't think it's lawyerly at all.  It reads like a headline.  You have to be brief in that little subject box, there's not much room to qualify statements :)

muppet is now from madebymonkeys.net
Thursday, June 17, 2004

Terrence Parr (creator of ANTLR) on life and writing parser writers:

"Why program by hand in five days what you can spend five years of your life automating?"

Cabby
Thursday, June 17, 2004

A couple of reasons:

1. Many times programmers simply don't know the tools that are available to do simple automation.  This is especially true with more "hardcore" programmers that use C++, raw Win32 API calls, and so on.  Web guys and so-called lightweight programmers quick to embrace Perl and PHP and Python, but in my experience this is unusual among more hardcore types.

2. Programmers sometimes don't see automating tasks as part of their job.  They like to do the down-and-dirty hard stuff, but see automation as more mundane.

3. It seems to take a certain kind of personality and experience to push for heavy automation.  People who have been around tend to do it more.  People who are new or very tech-centered often don't want to.

Junkster
Thursday, June 17, 2004

The judgement about whether to automate or not is made all the time. If I'm doing something to 10 files and don't expect to ever do the same thing again, I would not automate it. It is not necessarily a sign of stupidity or stubbornness if a programmer does a task by hand. Sometimes the risk of automating is too great because scripts are as dangerous as they are helpful. Sometimes it takes longer to write and test the script, having to foresee and consider all possible scenarios, than to go ahead and suffer through the drudgery.

Dr. Real PC
Thursday, June 17, 2004

See Stephen R. Covey's explanation of quadrant based time management.

QI is everything that is important *and* urgent.

QII is everything that is important, but without any corresponding urgency.

QIII and QIV don't matter for this discussion.

Most "busy" people spend a lot of time in QI, which is cranking out the code, meeting deadlines, etc. -- bailing water out of a boat with a hole in it.  Activities like automating tasks are QII -- fixing the hole in the boat.  To make the transition to spending most of one's time in QII takes discipline and guts.  One has to literally take a leap of faith that taking time away from QI activities will pay off instead of sinking the ship.

I hate to use the buzzword, but it really requires a paradigm shift, and it sounds like frustrated is experiencing that shift.  Good luck helping others see it.

MacSqueeb
Thursday, June 17, 2004

Sometimes, programmers are just plain stupid and lacking in common sense and proper judgement, especially in smaller, less formally run hack shops.

I worked for awhile as an independent contractor with a commercial software vendor that has 70 or so different editions of the same software that must be released concurrently every year.

They do not use configuration management tools such as source code control. They literally ZIP source files and squirrel away copies of source on their server.  They have no way to rebuild identical versions of code that they release.

Also, they use Delphi and nobody there has any idea how you use the command line version of Delphi and batch files to automate builds.

Also... the alsos keep coming... the owner is infatuated with the idea of making his developers share the same source file, and having two (or sometimes more) developers modify one file concurrently. Then the multiple versions are handed to one "senior" guy who eyeballs all the modifications and "merges" them into a single result file.

Also, you can't suggest that they could do anything more effectively. If you mention source code control they think you're pushing off academic concepts on them. They think the non reproducibility of their builds is not a problem.

The owner spews this lean and mean, "we just write simple software" bullshit all the time. It's a sweatshop, too.

So, stupidity and being in over one's head often explains a lot.

Bored Bystander
Thursday, June 17, 2004

At one point I was so pissed I wrote this:
http://www.komkon.org/~junta/techsts/monkeys.html
:)

GG
Thursday, June 17, 2004

GG,
It's very easy to get aggravated and fee superior to anyone who does things differently from you. There will always be someone with less experience than you in some little area. You will always be able to find stupid examples written in a hurry or when someone was having a bad day and thinking about other things. No one thinks of or remembers everything, not even you.
I bet we could all find moronic examples in your code.

Dr. Real PC
Thursday, June 17, 2004

I don't see what the furor is over this idiom:

if( isOK( ) ) {
return( true );
}
else {
return( false );
}

Yea, it's more verbose, but who cares?  The net effect is precisely 0; the compiler should optimize away any extraneous jumps.

It's just stylistic; and more so, it's useful if, for instance, you suspect you'll be adding error handling code in the false case later.

(ok, done hijacking the thread now)

monkey
Thursday, June 17, 2004

CG  - I know how you feel, I worked in a company with a lot of monkeys. Myself and another developer had to fight with departments filled with them. Programmers, Managers, Lan Guys & DBAs.

Dr. Real PC, I agree with you that sometimes we may over react but CG wasn't saying that he never makes any mistakes or that his code is bug free. He is accusing certain programmers of being monkeys because they don't think for themselves. They listen to whatever managers, clients and other programmers say even if it doesn't make sense. These monkeys don't understand basic concepts that should have been picked up in comp sci 101. These types in IT become a major hindrance to pragmatic programers who want something better than just collecting a paycheck and patching crappy software all day long.

Sometimes I think that certain programmers came into the field because of the $$$$ not because they really like it. These are the types that don't keep up with technology. Programming and computers are not their hobby or passion. Software development is just a job - it's not really fun to them - they have no desire to "build things right."

I worked at a company like that and eventually I had to get out of that environment. I am much much happier now...

Gen'xer
Thursday, June 17, 2004

>On the other hand, if your organisation is static or shrinking,
>then chores make useful trenches.

This is one explanation, our lead is busy all the time, and seen as typing away 10 hours a day. The company is at serious risk of shrinking, and I suspect that this is one way for him to ensure he is last one out the door.

frustrated
Thursday, June 17, 2004

Many-in-one reply...

Dr.Real PC:

> It's very easy to get aggravated and fee superior to
> anyone who does things differently from you. There will

Yes it is :). There are many arguments where I have
agreed to disagree, because the other position was
at least ARGUED. But where in the particular examples
I've llisted have you seen actual thinking process?

> always be someone with less experience than you in

Lack of experience/education is not a problem or a reason
to disrespect someone professionally. However, lack of
desire to learn and think is.

> some little area. You will always be able to find stupid
> examples written in a hurry or when someone was

Did you see the dialogues I was having with these people?
It was not written in a hurry, it was persistent.

> I bet we could all find moronic examples in your code.

And after you point it out to me I will be the first to
call myself a moron. But please separate mistakes from
persistent lack of thinking.

monkey:

> I don't see what the furor is over this idiom:
...
> Yea, it's more verbose, but who cares?

I think this actually is harder to read (slightly). I am not
against it for reasons of optimization.

GG
Thursday, June 17, 2004

>Sometimes I think that certain programmers came into
>the field because of the $$$$ not because they really like
>it. These are the types that don't keep up with
>technology. Programming and computers are not their
>hobby or passion.

I think that depends on what you mean by keeping up with technology.  I'm very interested in programming and computation, but I don't keep up with technology in the sense of learning all the latest libraries and software dev vendor tools.  Much of that stuff represents the same concepts repackaged in a different form.  I think that the concepts from which technical implementations are derived are the most interesting things.

I guess my perspective on 'bad programmers' has been that they're not aware enough of past work in this field to not get hoodwinked by 'new technology'.

Kalani
Thursday, June 17, 2004

Monkey, the problem is it has spaces were you don't
need them and no white space where you do need
them. Compactness means zip if you can't scan it.
It's like writing headlines in an article or web page.
People don't read dense blocks of text. Give it some
breathing room and drop the useless parens. "return"
isn't a function so don't use parens.

if ( isOK( ) )
{
  return true;
}
else
{
  return false;
}

son of parnas
Thursday, June 17, 2004

son of parnas, maybe you're being sarcastic or clever or whatever, but the obvious solution is to simply write:
return isOK( );

Captain McFly
Thursday, June 17, 2004

Captain McFly, that's the thing that sparked this discussion in the first place, read above :)

GG
Thursday, June 17, 2004

> maybe you're being sarcastic or clever or whatever

I don't have the prototype for isOk so i don't know what
it returns. In anycase it doesn't matter, it's the idea
that counts.

son of parnas
Thursday, June 17, 2004

> I don't have the prototype for isOk so i don't know what

In this case, fair enough. I should have mentioned that it's Java, so it actually can only return a boolean, since you're already using it like that in if statement. Of course, in C it could return int, but then you'd still use int instead of true/false, so that point is kinda moot too. :)

GG
Thursday, June 17, 2004

""return"
isn't a function so don't use parens."

It should require parens.  if isn't a function either.  Neither is while.  How about for?

Just because the language is inconsistent, doesn't mean I am.

monkey
Thursday, June 17, 2004

Very often it is not programmers but it's management that stands in the way, especially for things that require more sophisticated automation.

Say there is a task that uses up 2 hours every week, but it would take 30 hours to write something to automate it.  It is going to necessary to keep doing it for at least a year.  With 30 hours to write it, there would be payback in 15 weeks, so for 1+ years it is definitely worth it.

The trouble is that at any given point in time, it is always going to be faster to spend the 2 hours right now to do the task by hand instead of working to automate it, and management won't approve 30 hours to automate it.

NoName
Thursday, June 17, 2004

I think a real, red-blooded programmer looks to automate anything he/she possibly can.  It's the nature of the job.

pickle
Thursday, June 17, 2004

Why not automate:

1. Organisations which simply won't let you have the tools. You can't use GNU make (it's "free" and therefore "unsupported"). You can't use Perl, because it's "insecure". Bang - you're down to shell and Sun's broken make.

2. Developers who know exactly one language because that's what they need to do their job. To them, "make" is a mysterious tool that they don't understand. So is shell, perl and awk. They don't understand regexps, they don't understand shell variables.

2.5 This is because people are taught C++. They don't get taught "how to build C++ projects". They don't even get taught how to write large C++ projects, they just read example programs that are 40 lines long and have two classes and then get let lose building things like stock exchanges...

3. Typing, even boring, repetitive typing makes you look productive in some organisations.

4. Job security is damn right. If you don't tell someone how to do the builds, you're hard to fire.

5. Failure to notice it's even possible. UNIX development is *SO* dire across the industry that people can spend their entire career and never see a completely working makefile. They can't even imagine it's possible to do one-step builds.

5.5 A belief that this is how it happens. On windows it's harder to automate anything outside a dev studio tool... people end up just having to drag files around "by hand". So they do it on UNIX as well...

6. Too many staff. Frankly, there are too many people in the software industry. We could easily fire the bottom 90% of the software engineers and we'd not notice. Most of those people are doing drudge work; writing stupid header notices on files. Doing delegation layers in Java that should be multiply-inherited in. Typing commands that should be automated. Watching builds from clean happen because their makefiles don't do dependencies better. All those dumb developers who can't be given anything real to work on because they'll screw it up need to be given something to do. Mindless typing is perfect. This is why large organisations are like this. Small ones can't be, because they don't have enough spare people. Or rather small organisations that try to be like that will soon not exist.

Katie Lucas
Thursday, June 17, 2004

> Just because the language is inconsistent, doesn't
> mean I am.

I think that's a good impulse, but i think readablity
is more important.

If and while require parens. A return doesn't. So
my not using a parens you are making returns
visually distinc and easier to scan.

son of parnas
Thursday, June 17, 2004

I agree with NoName. Monkey Management is the problem.

In the company I worked for, every line of code we wrote had to be approved and tracked in a project management software package.

If we wanted to write some automated tools then an official project request had to be submitted to the Imperial Senate for review. Most of the times Senator Binks from Naboo would deem such projects as not being necessary and veto them.

My non-monkey friend and myself wrote some rogue scripts anyway but the other monkies refused to take part and did not see the value in automating something they have done manually for years.

I can hear the monkeys now....

"Guys we don't do that here..."

Gen'xer
Thursday, June 17, 2004

"Most of the times Senator Binks from Naboo would deem such projects as not being necessary and veto them."

Sounds like you need to start using some Jedi mind tricks on the good senator. :-P

Wisea**
Thursday, June 17, 2004

pickle:

> I think a real, red-blooded programmer looks to
> automate anything he/she possibly can.  It's the nature

I think this also is a conceptual problem. Monkeys are
perhaps unable to realize that their work is really
making it possible to automate other people's tasks,
because it's easier and more efficient. So -- why not
start with yourself? :)

GG
Thursday, June 17, 2004

Oh yes. The "Them" that veto things. Oh, I hate those discussions.

"Look, we'll just fix the makefiles."

"Why?"

"Because they're broken."

"Why does that matter?"

"Because everyone is doing builds from scratch because the dependencies are broken, it takes ages over our duff network."

"Why is that a problem?"

"Look you're the ones who keep telling us we're too slow at this development thing."

"That's why I think you should be writing code faster and not messing with these 'makefile' things..."

It's like having a battle of wits with a rock.

Katie Lucas
Thursday, June 17, 2004

Automating is a lot more fun than typing the same darn thing over and over. You don't have to be so passionate about keeping up with technology that you never sleep to consider programming more interesting than mere typing.
Nevertheless, I will not automate every little thing. You wind up with scripts all over the place that were used once. I automate a lot of things, whenever it makes sense to me. But somebody glaring over my shoulder might think "why the heck doesn't he automate that, he already typed the same thing twice."
I also do some things by hand because a mistake would cause many hours of boring miserable work trying to put something back together.

Dr. Real PC
Thursday, June 17, 2004

To the guys complaining about the "them" out there. I'd highly suggest trying to develop a sort of friendship or mutual respect with the evil vetoing senator. As soon as they trust your opinions, they'll take your suggestions a lot more seriously.

Even better, get yourself another senator to back it.

Useless Mgmt
Thursday, June 17, 2004

When you work at a company which requires about a dozen manual steps in order to check in your work then you will know what hell is.


Friday, June 18, 2004

> Doing delegation layers in Java that should be multiply-inherited in

You can't do multiple inheritance in Java :-(


Friday, June 18, 2004

> You can't do multiple inheritance in Java :-(

Yeah, that, in fact, does make for a lot of redundant typing... There should be some nice sugar for that.

GG
Friday, June 18, 2004

*  Recent Topics

*  Fog Creek Home