|
What to make of "hand optimized code"?
Its review time again...and one of the peer developers whom I am reviewing has written in his accomplishments "wrote hand optimized code".
I took issue with the statement because to me "hand optimization" means someone took the assembly object code generated by a high level language compiler and optimized it by modifying the assembly code directly.
This is not what happened in the case...the developer claims he optimized the source code itself...almost 10% of which was old old and 90% of which he newly wrote. He does not have performance metrics of his "non-optimized" v/s "hand-optimized code".
Is it wrong to assume that one has to attempt to write optimized code in the first place...that it is a job requirement for a developer rather than an accomplishment...or am I wrong in assuming this and the world has changed under me?
Anyway I let the issue slide in the review but still curious enough to know what other developers think about this.
Code Monkey
Friday, January 9, 2004
If he created code using a RAD tool and then optimized the output of that I would credit him with "hand optimization".
When I used to live out east I recall seeing a brand of potato chip on which the label bragged "Hand Cooked". That really impressed me. Ouch!
Name withheld out of cowardice
Friday, January 9, 2004
Nothing wrong with optimizing code. I might have a minor issue that his workload is so light that made it to the list.
The big problem, and what I'd thwack him on the nose for, is not being able to provide metrics. Did the performance gain/memory improvement justify the time he spent "hand tweaking"?
Philo
Philo
Friday, January 9, 2004
>If he created code using a RAD tool and then optimized the output of that I would credit him with "hand optimization".
Interesting...because the developer claimed he took the code generated by a "wizard" and changed / added to it. I dunno if that qualifies as "hand optimization" since wixards by their very nature generate templatized code and one always has to modify them.
More the more bigger issue is would you call something which was written 90% newly as "hand optimzed" since there was no real basis to benchmark it against anything similar before.
The developer also claimed that optimization which is done by a compiler automatically e.g like remove invariants out of a loop is "machine optimization" and anything else is "hand optimization"
Code Monkey
Friday, January 9, 2004
>Did the performance gain/memory improvement justify the time he spent "hand tweaking"?
Not really..he did not give any performance metrics but he did say (and it is true) that the number of bugs discovered in his code were lesser by around 12% of average.
But my point was that the average is just that...an average and it is meaningless to simply compare bug counts in found in different code modules against each other even if normalizes them by dividing them by LOC since this does not take into account the differing complexity of code fragments.
I guess in today's job market one has to put everything one does into "accomplishments"!
Code Monkey
Friday, January 9, 2004
Bleh, in most of the applications I've worked on, the only thing to optimize that was worthwhile were database queries and possibly one or two loops. Everything else, when profiled, was pointless. I suspect that this is very often the case in other applications.
Saruman
Friday, January 9, 2004
Ditto what Philo said. It's neat that he can optimize code. But what kind of performance gain did he get? Does he know when it's not worth it to tweak the code?
One of my accomplishments is in almost doubling the performance of the core loop in a bytecode interpreter VM. I didn't write any assembly, but knowing how to force the compiler to assign certain variables to registers, and figuring out that branch prediction misses on the x86 are atrociously bad, were important in getting that sort of optimization.
Alyosha`
Friday, January 9, 2004
Like Alyosha` said, hand optimized doesn't have to mean getting down to assembly. And like Philo said, the lack of metrics is troublesome.
But what I really don't understand is what led this developer to decide to optimize it in the first place. Was it buggy? The 12% reduction is bugs is nebulous. Was it slow at a critical process? Did they doument there reason for spending this time?
If someone decides on their own to allocate time toward optimizing code, when it might not even need it, than that should count against them on their review, not for them.
Nick
Friday, January 9, 2004
>>doument there reason
should read:
document their reason
Nick
Friday, January 9, 2004
Why do you say that the fact he didn't provide metrics is bad?
Sometimes an app runs slowly.
You look at it, examine the loops, modify them, replace some function calls with other function calls, and then the apps runs significantly faster.
You don't need metrics if you simply notice that the new version is faster.
I don't know about other companies, but arround here, we optimize for developer speed.
That is, developers have to write a good, bug-free program, and they have to write it as quickly as possible.
It doesn't matter how fast the program runs.
Then, after a testing phase, we decide together which parts run slowly, and have to be optimized.
MX
Friday, January 9, 2004
MX, I agree with you in general. However, if it's significant enough to put down on your performance review, then it implies that it took a significant amount of effort. The greater the amount of time you spend working on something, then the more you need to justify how you allocated time.
Nick
Friday, January 9, 2004
>But what I really don't understand is what led this developer to decide to optimize it in the first place. Was it buggy? The 12% reduction is bugs is nebulous. Was it slow at a critical process?
Well the 12% fewer bugs means that the number of bugs logged against this developer for this particular "hand optimized" module were 12% less than the average of all the bugs logged against all modules. This is true I was sufficiently curious to cross check and confirm this. I still feel it is not right to say that this metric alone is sufficient to say that the code is hand optimized or not.
Whether it was slow in the first place is undecidable since only 10% of the code was from somewhere else and 90% was rewritten. It is possible he is comparing the performance of the first iteration of his code with the final one...but to me that is not kosher.
>Did they doument there reason for spending this time?
This is an interesting aside....I found out that the developer took more time than allocated to finish the module and one of the reasons he gave for the delay is that the code he produced was of higher quality (again giving the bug metrics) and hence it took him longer to write.
I hate doing peer to peer reviews because of such situations!
Code Monkey
Friday, January 9, 2004
MX - come on, "you don't need metrics if you simply notice that the new version is faster."
I got to work in 15 minutes today. Is that faster than yesterday. It seemed like it.
"It doesn't matter how fast the program runs. Then, after a testing phase, we decide together which parts run slowly, and have to be optimized. "
Ok, this must be one of the "before" companies when they talk about cost of corrective action. Bad design give bad results. Figuring it out is a bad design AFTER it it is coded, is far more expensive than a good design would have been.
AnonAnonAnon
Friday, January 9, 2004
When you compare the number of useless statistics against the percentage of cars painted blue in Iowa, the useless statistics have a whopping 45% lead.
Alyosha`
Friday, January 9, 2004
By any chance would those cars have been hand painted? I heard its slower at first, but it makes the car run faster.
Andrew Hurst
Friday, January 9, 2004
>By any chance would those cars have been hand painted? I heard its slower at first, but it makes the car run faster
Good one Andrew :-)
Although the guys who malke Rolls Royce Phantom proudly claim their cars are better because they are hand assembled :-)
http://www.seriouswheels.com/top-2004-Rolls-Royce-Phantom.htm
Code Monkey
Friday, January 9, 2004
1. See Joel on peer reviews. They guy had to write *something*. He picked 'hand optimization' because he remembered it, most probably as a pleasant experience. He's a good programmer.
2. Anyway, why are you concerned? His only sin is being too devoted to his work; if sometimes this means he over-optimizes and therefore turns work in a bit late, just explain to him the rules of early optimization: Don’t do it, Don't do it yet. He's a good programmer.
I think you're better off with him than with someone who never optimizes and turns out big Schlemiel monsters.
Alex.ro
Friday, January 9, 2004
>> "It doesn't matter how fast the program runs.
>> Then, after a testing phase, we decide together
>> which parts run slowly, and have to be optimized."
> Ok, this must be one of the "before" companies
> when they talk about cost of corrective action.
> Bad design give bad results. Figuring it out is a
> bad design AFTER it it is coded, is far more
> expensive than a good design would have been.
People are more expensive than machines.
It makes sense, from an economic point of view, to optimize for development time, and not for run time.
We try to reduce the development time, and not the run time.
One of the ways we reduce development time is by having a good high-level design.
Another way we reduce development time is to have programmers work with the following mindset:
"My goal is to produce quality code, which solves the problem without bugs and is as modular and simple as possible. I usually don't sacrifice simplicity for speed."
The idea is that we don't write code for compilers. We write code for other humans, who later have to maintain the code.
When the client requests a modification, we want to be able to understand the code as fast as possible, and modify it as fast as possible.
Of course, this approach produces, sometimes, code that is too slow.
We then isolate that parts of code, and optimize them individually.
The other programmer mindset, which you seem to like, would be "I strive to write fast code, and optimize my code all the time."
Both approaches can produce good results.
But, in my opinion, if we take into account all costs, including hardware costs, development costs, etc - this approach is a lot more expensive than the approach we use.
MX
Friday, January 9, 2004
"Why do you say that the fact he didn't provide metrics is bad?"
Because without understanding the context of this "hand optimized code", it's meaningless to discuss it in a peer review.
Programmers are notorious for gold-plating and for spending an inordinate amount of time on the things that don't really matter. That's why having good metrics can go along ways towards really justifying why someone needs to spend 3 days optimizing code.
In the absence of good metrics, a programmer can spend days increasing the speed of code that is rarely used by 5% and then say "Hey! Look at me! Aren't I special!?"
Mark Hoffman
Friday, January 9, 2004
So he engaged in a process that resulted in fewer bugs in his modules than in the modules of other programmers. And he was proud enough that he did so to call attention to it.
And you guys are slamming him for it. Probably jealous. Regardless I'm glad I don't work where you guys do. I'd probably kill myself before long.
Tony Chang
Friday, January 9, 2004
"One of the ways we reduce development time"
One of the ways we reduce development time is to spend a little bit extra time to reduce bugs by 12% before it gets so far along in the development process that it ends up costing us ten times as much in developer time, effort and missed opportunity due to delays.
Tony Chang
Friday, January 9, 2004
"And you guys are slamming him for it. Probably jealous. Regardless I'm glad I don't work where you guys do. I'd probably kill myself before long. "
Oh the blissful ignorance of the young, naive, apprentice programmer!
Bah
Friday, January 9, 2004
--
And you guys are slamming him for it. Probably jealous. Regardless I'm glad I don't work where you guys do. I'd probably kill myself before long.
--
I agree with this. You guys complaining about it are just a bunch of whiners.
First people complain it was bad to optimize because there are no metrics. Then you complain that a 12% reduction isn't a valid metric. I'd say 12% is a decent reduction.
Do you guys do cost-benefit analysis of every single task during the day? Do you involve other people with every decision or do you sometimes take the initiative and do something because you think it's a good idea?
NathanJ
Friday, January 9, 2004
Seems to me anyone who writes "hand-optimized code" is a bit of a wanker.
First, the code should be good anyway. Does he normally write slop?
Second, "optimize" is perfectly sufficient to describe the process. Adding "hand-" suggests someone not skilled or experienced in development, but seeking to add pretentious labels. Which affirms point 1.
Me and the view out the window
Saturday, January 10, 2004
>>Do you guys do cost-benefit analysis of every single task during the day?
I knew an intern (paid hourly) who calculated how much money he earned every time he took a dump. Does that count?
Nick
Saturday, January 10, 2004
I'm just trying to envision a developer turning his list of major accomplishments that read:
- Coded some module
- Delivered [x] project on time
- Hand-optimized code for [y]
If we're talking a significant amount of time (quarter, semiannual, annual), then my initial reaction is "in 3 months you consider 'hand optimization of code' a significant accomplishment?"
I mean, I've optimized code. I think the longest I ever spent on it was four hours (to take a page load from 40 seconds to 3). I can't imagine putting a four-hour task on a quarterly review.
And even so, I know the metric on that page improvement.
Admittedly, there's a lot we don't know here. But from what's been said, if *I* were the manager, I'd use it as a teaching opportunity about "effort vs. reward"
My $.02
Philo
Philo
Saturday, January 10, 2004
Sure, he spent a little bit effort and lowered bugs by 12%. Probably it was a signifigant overall gain since the bugs would have had to have been fixed anyway and the longer you go the more it costs.
But then I am repeating myself.
I wonder how many extra hours he spent on it compared to the number of hours his detractors spend posting on JoS? Let's do a cost benefit analysis of that.
"My primary achievement this quarter was posting long winded puff pieces about my self on JoS while slamming developers who used their time to optimize code, resulting in code with a lower defect rate."
Tony Chang
Saturday, January 10, 2004
<I>
in 3 months you consider 'hand optimization of code' a significant accomplishment
</i>
Wow, you people must just spend all day every day discovering entire new algorithms that have previously been unknown to CS.
Me? I've had a fair amount of 3 month stretches where I've done absolutely nothing but refactoring and maintenence work. Kind of hard to pick out specific accomplishments there. And if I wrote down something like "made legacy code vastly more usable/readable" you bastards might ding me for not giving you metrics on just HOW "usable" and "readable" the software is now.
Anyway, self-evaluations are a joke. If you're managing people and you really have so little clue as to what they are accomplishing that you need them to tell you every 3 months, you shouldn't be managing people. I'm absolutely not advocating micromanagement, but how the hell can you not know what your developers are up to? Don't you ever swing by and talk to them? Aren't you planning up front and then assigning them work? Aren't you working with QA and listening to other developers enough that you get a good sense of the quality of the developer's work without asking him/her for their own opinion on it?
I just... don't... get it.
Mister Fancypants
Saturday, January 10, 2004
Damn, forgot about the no-HTML thing here.
Wanted to comment on this too:
"I mean, I've optimized code. I think the longest I ever spent on it was four hours (to take a page load from 40 seconds to 3). I can't imagine putting a four-hour task on a quarterly review. "
I think that statement is somewhat naive in assuming the optimizing developer works on a certain type of software that he or she may not work on (eg, web services app or business frontend or such). I have no idea what industry the unfortunate target of this thread works in, but as someone who has done a lot of professional graphics programming in the past, I've sometimes spent WEEKS optimizing things algorithmically. In some programming tasks you have to have things running so ridiculously fast that you really do need to worry about low-level optimization, pipeline stalls, cache misses, etc and you also have to worrk about high level data structure and algorithmic optimization, which isn't a minutes/hours job.
I'm not saying this as a means of dick waving or anything, and I'm not saying that people who do graphics/3D work are in any way 'better programmers', I just wanted to point out that there is a wide range of software tasks where optimization is a lot more effort than just making sure your inner loops aren't making too many function calls, etc.
Mister Fancypants
Saturday, January 10, 2004
Couldn't it simply be that the poor guy was actually proud he'd managed to optimize some code?
Stephen Jones
Saturday, January 10, 2004
Wow, someone takes self-evaluations seriously...
(I always considered them something of a curse someone way up in upper-management plagues us with, and everyone knows they are BS...)
SG
Saturday, January 10, 2004
"I'm just trying to envision a developer turning his list of major accomplishments that read:
- Coded some module
- Delivered [x] project on time
- Hand-optimized code for [y]"
Hm. Well compare to:
- got fired from contract job
- posted 1800 messages to online forum
- quit 2nd contract job 1.5 months into contract
- got hired as a salesperson for a microsoft satellite office
I dunno, I kind of like the "hand optimized" guy better...
...
Saturday, January 10, 2004
Ah, it always warms my heart when someone makes it personal.
And the second contract job was 20 months, not 1.5 ( I worked it concurrently with Camel, which was part of the problem at Camel).
And if it makes you feel better, I hand-optimized code in there, too. ;)
Philo
Philo
Saturday, January 10, 2004
golly, all we need now is to start attacking performance reviews too! I've never seen them as useful myself..
i like i
Sunday, January 11, 2004
No, we just need to start whining about people engaged in the useful and productive process of whining about whiners. Any further whining would just be getting silly, though.
"You're wrong because of blah blah and blah" is fine, but "you're just whining and I'ld kill myself if I was within 83 light years of your office" is good for a laugh, at least.
Sunday, January 11, 2004
Onlee ze Party membahs must bee ullofed to speek!
Adolf
Sunday, January 11, 2004
"Premature optimization is the root of all evil" - Tony Hoare
No metrics on optimization is a bad thing. You need to know where the time is being lost before you start, it's almost never where you think.
Peter Ibbotson
Tuesday, January 13, 2004
 Recent Topics
Fog Creek Home
| 
