Fog Creek Software
Discussion Board




Eureka Moments

Is it normal to have sudden flashes of insight in software development?

For instance:
I was called into debug some code which was segfaulting (memory access violation for MS folks) recently. The problem was intermittent, so it took me a while (including deploying an instrumented version of the product) to track down the cause.

I left work one evening having identified a pattern in the corrupted memory, but I didn't really have a handle on how the code would cause that.

I woke up earlier than usual the next morning, and the first thought to enter my head was exactly why the code was failing. I just *knew* it was the answer, without checking the code. Needless to say, I couldn't get back to sleep.

Some of my work colleagues found this strange. I thought it was normal. Am I wrong? Has anyone else got war-stories like this?

Finally, if you were a consultant, how would you bill for the "extra" time you spent debugging the problem?
Is there a real ethical dilemma here -- would you bill for the whole time you were asleep, just your REM sleep, or what?
I'm not a consultant, but my employer does requies us to keep accurate time sheets...

MugsGame
Sunday, August 17, 2003

Like my mom says, "when you start to get frustrated, walk away from the problem." The brain does a lot of work on our behalf including problem solving. I would imagine this happens a lot and has happened to me as well.

As far as billing for this subconscious time, I would say not. I am not hourly, but if I were, I doubt my employer would let me bill for the work-stress I carry home with me. It is unrealistic to expect humans to turn off their problem solving thoughts when they step onto the parking lot.

m
Sunday, August 17, 2003

This happens to me all the time. Sometimes when I go out and take a walk, or when I'm driving home. But often the hard work is done while dreaming. I wish I could get paid for it!

The Real PC
Sunday, August 17, 2003

Yup, from time on time it happens to me, too. The most "memorable" flash I had was back in my first year doing real work. I designed a complete graphical windowing system over DOS, and one of my goals was to be able to using some now common event (mouseover and the such) in a queue system. But performance was a serious issue when I had many windows on screen at the same time. A complete day was spent trying to optimise this, but my pillow told me that I had to move the last invoker to the top to the queue and that's it.

Now it looks like such a simple solution, but back then it was driving me nuts.

Okay, I feel old. Shoot me now.

Leonardo Herrera
Sunday, August 17, 2003

I believe (in the UK at least) that it's fairly common for solicitors/lawyers to bill for time they spend 'thinking' about a case ...

blargle
Sunday, August 17, 2003

I solve no end of problems by walking away from them to do something else or take a break, and then "Eureka!".

Theres no right or wrong or strange or normal way of exercising your brain, theres simply what works good for you.

Robert Moir
Sunday, August 17, 2003

Not strange at all.

The server cupboard was being rebuilt, so everything was shutdown and pulled out over the weekend (and I mean every cable, because we decided to take the oppurtunity to straighten everything out). I spent Sunday afternoon plugging everything back in, and when I turned it all on none of the workstations could access either server. I had been there for hours, decided I would ask some colleagues about this one, and went home.

About 3am that morning I sat up with a start....I had forgotten to plug the hubs back into the powerpoints. I couldnt wait to get back to work and fix it (I was tempted to get up and get dressed right then and there).

I spoke to my husband about it, I was amazed at the capacity of the human brain. I mean I was asleep, supposedely dreaming, but instead my brain was working over the problems of the day. Anyhow my husband (an electrician) said that he recalled the same thing happening often to one of his coworkers.

The human brain has alot of untapped potential I think....on the other hand, I would miss the crazy dreams I have if my brain spent the entire night solving problems every night....

An Aussie Chick
Sunday, August 17, 2003

I go for a walk. Preferably during the evening when the temperature is (relatively) low. Clears my head; keeps my cholesterol down; and let's me listen to music all at once.

Works like a charm.

Mickey Petersen
Sunday, August 17, 2003

As for the "should I bill it" question -- yes, you should. Not the hours you were asleep, but the time you spent after you woke up thinking about it and making sure it would work.

If I'm even thinking about a client's (application(s), database(s), issue(s), bug(s), whatever), I bill it as "analysis" time. Just because I'm not in front of a computer looking at the source code, doesn't mean that I'm not actively debugging the problem. In 6 years I've not had a single client complain about this (as long as the problem was solved in a reasonable amount of time). YMMV -- about 6 years ago I did have a client complain because he knew from the building's keycard-access logs that during the timeframe I billed as "analysis" I was not in the building anywhere. We sat down, I stated my case, and he paid the bill. Depends on the client, but I'd bill it anyway.

Sgt. Sausage
Sunday, August 17, 2003

Perfectly normal behaviour.

My morning bike ride is often the most productive part of my day.

Andrew Lighten
Sunday, August 17, 2003

Happens to me all the time but my inspiration usually strikes in the shower.

My most recent experience involved a tough nut I was trying to crack.  I spent the whole day approaching it from different angles to no avail.  I finally left the office at 8 pm (usual quitting time is 4 pm) and hopped in my truck for the drive home.

My mind was in overdrive during the drive home when the eureka moment suddenly hit me.  Next thing I know I'm plowing through traffic cones that were marking off the beginning of a construction area.  Fortunately the cones extended far back from the actual work area and I hit at very low speed, I only knocked over a couple of cones.

As luck would have it there was a Saudi policeman right near my "accident" site.  He came over and asked me what the hell happened.  No doubt he could see the excitement in my face as I gushed about the difficult problem I was facing at work, and how I'd suddenly figured out the solution.  The weirdest part is that he then congratulated me and said I must be very happy, but please do drive carefully.  (Stephen Jones can attest to how weird that is.)

Anyway, I turned around and hauled ass back to the office to implement my solution.  Within an hour I had finished the very thing I'd spent over 12 hours trying to do.  As usual the solution was to get away from the problem for awhile.

I was so excited I could barely sleep that night.  I love eureka moments.

Matt Foley
Monday, August 18, 2003

Several months ago I was listening to an NPR interview with a TV screenwriter. He would think about his storylines before going to bed, dream about them, then get up in the morning and write down his dreams. Most of his screenplays were direct products of his dreams. He also kept a notepad by his bed for middle of the night inspirations.

I've had these experiences too. The most memorable time was when I was trying to solve a hairy statics (engineering) problem as an undergrad. I banged my head on that problem for 4 hours. Finally I had to give up and go to sleep. I dreamt about the problem, solved it in my head, and woke up in the middle of the night with the answer. I walked straight to my desk and solved the problem. Perfect!

Nick
Monday, August 18, 2003

I think this is why it's a good idea to design your stuff (on paper!) before implementing it.

Sitting in front of the computer (or where ever you have a nut to crack) will "prompt" you for an answer every microsecond, and your brain never gets to use its full potential because of the constant flow of interrupts.

But when you are away from the keyboard, in a situation where time doesn't exist or matters, the problem will bounce around in your brain and eventually hit the solution.

Martin A. Boegelund
Monday, August 18, 2003

When you put a problem aside for a while (say at least an hour to overnight), you come back to it fresh... you drop any assumptions you may have been making. 

anony125
Monday, August 18, 2003

I've had eurekas come to me while I was in the shower (isn't that like how "eureka" got started?).  More commonly, I've walked into work fresh in the morning and quickly solved a problem that was frustrating me for hours the previous evening. 

I once near-*ordered* a coworker to go home.  He was uselessly grinding his gears against a problem, so stressed (deadlines) and frustrated he was practically in tears.  I politely but firmly let him know he wasn't going to achieve anything that night, and volunteered to meet him in the early morning to help him finish.  The next morning, he walked in all smiles -- he'd figured out the solution during his morning commute.

anonymous
Monday, August 18, 2003

Hot tubs, saunas and showers work best for me.

Just me (Sir to you)
Monday, August 18, 2003

Thomas Edison used this extensively.

He would think about a problem then, while holding marbles (or something) fall asleep in his chair.

As soon as he fell asleep, he'd drop the marbles and and wake himself up. 

Supposedly, he came up with a lot of good solutions that way.

Entrepreneur
Monday, August 18, 2003

Matt,

If is possible could you let me know which company you work for?

Thanks,

Prakash S
Monday, August 18, 2003

It's a very large oil company.  You guess the name  :-)

Matt Foley
Tuesday, August 19, 2003

got it:-)

Prakash S
Tuesday, August 19, 2003

I've had the same sort of reaction, with waking up with the answer to exam questions.  And then feeling really annoyed, because of the lost marks. 

Sigh

Daniel Schwartz-Narbonne
Wednesday, August 27, 2003

*  Recent Topics

*  Fog Creek Home