Fog Creek Software
Discussion Board




A-ha! interview questions

Regarding a-ha questions; the general consensus (on these boards, anyway) seems to be that they don't give you any valuable information, that they only show if you've seen the question before.  I just wanted to offer this for consideration.

I work for a defense company, in engineering support (the programming equivalent of "freelance hit team").  We work with several languages and platforms, as well as a variety of different problems and tasks that crop up.  Having a keen and curious mind is a definite requirement for the job.

We hired two interns for the summer of '02, and, during my interviewing, I showed each a few logic puzzles, then the gold chain question from Joel's archive, and another 'puzzler' problem.  The two hirees had very different approaches to solving the problems, as well as very different responses.  The one who asked for pen & paper ended up getting both after some wrestling.  The one who simply contorted his face and hmm-ed a lot ended up just guessing wrong.  The first got a lot out of the internship, as well as giving a lot back.  He grasped the issues & ramifications quickly, did solid work, and knew when and where he needed assistance.  The second ended up needing a lot of coddling with every aspect of his work, particularly debugging his own code.  We've invited the first one back for next summer; we will not hire the second again.

I'd like to submit that the people asking questions that require the intuitive leap are using it because they know the kind of minds that they want.  Not that it has to be the sole criterion, nor is it necessarily always a desired trait; but I think it's a valid and valuable type of question to ask, particularly in the programming world.

van pelt
Monday, September 09, 2002


I think people on this board like aha:! questions, but think that aha:!!!!! questions are pretty much useless.

http://www.techinterview.org/aha.html

But be careful to remember that this is an interview.  Some of the best thinkers are not necessarily quick on their feet.

semi
Monday, September 09, 2002

How much dirt is in a 1X2X3 meter hole, measured in cubic feet? hint: 1m=39.25 inches

XXXXXXXXXXXXXXXXXXXXXXXX

answer:
!!!!!trid on sniatnoc eloh A

Robin Debreuil
Monday, September 09, 2002

I like aha! interview questions very much. Not only they show how smart interviewees are, but also how well they perform under stress conditions.

However, it's possible, that interviewers are chosing the method they personally feel comfortable with... For example, being interviewed, I will take puzzle-type questions over API-type questions any day. Of course, I'd like to see the same qualities in people I'm hiring.

The problem with it is that you need a lot of fresh puzzles. Interviewers should not use puzzles from usenet FAQs or popular web-sites... like this one: http://www.ocf.berkeley.edu/~wwu/riddles/intro.shtml (check it if you're into puzzles - excellent collection)

Igor K.
Monday, September 09, 2002

Semi -

I think there's some misunderstanding about what's aha! or aha!!!? question. From the site you linked to:

"what's the sum of the numbers from 1 to 1,000"

IMHO, it's not aha! question at all. Sum of arithmetic series is high school math question. Where's that aha! factor here?

Igor K.
Monday, September 09, 2002

Igor,

You're right.. hence that's why there is only one exclamation point and not 4 or 5.

Michael H. Pryor
Monday, September 09, 2002

techinterview is not my site, it's Michael's! Give credit where credit is due...

Joel Spolsky
Monday, September 09, 2002

The aha for the ol' "adding 1 to 1000" is the notion of adding pairs of numbers that each add up to 1000. 

eg:
1+999
2+998
etc

Bella
Monday, September 09, 2002

Bella, yes, pairing - this is exactly how the N*(N+1)/2 formula is derived. Every math class should cover this.
It might be an aha! question for high management positions, but definitely not for engineers ;)

Igor K.
Monday, September 09, 2002

This stuff is all freaking me out!

In the US, are questions like this common in job interviews? If so, how on earth does anyone who is not a puzzle freak ever get a job?

Les C.
Monday, September 09, 2002

Les, I've worked several programmer jobs and I've never had these kind of questions. From what I gather, they're asked by egotistical interviewers that think they're clever.

Troy King
Monday, September 09, 2002

Igor, I think you're just being difficult.

The sum problem is a perfect example of an Aha:! problem for someone that hasn't had that math class you had.  Or someone like me who can't remember a formula he learned over a decade ago and has had no need to recall since then.

those who know me have no need of my name
Monday, September 09, 2002

"pairing - this is exactly how the N*(N+1)/2 formula is derived."

Personally, when I was asked this in an interview, I simply guessed the formula and then proved it by induction - if it holds for some positive integer N then it also holds for N+1, and since it obviously holds for N=1 it holds for all positive integers. I got turned down for being a smartarse.

Carl Gauss
Monday, September 09, 2002

LOL :)

Igor K.
Monday, September 09, 2002

What kind of programming questions can be considered "A-ha! questions?"

S.C.
Tuesday, September 10, 2002

I guess what it's about is not the answer to a problem, but more the way in which it is solved. That includes trying to understand. To speak with the words of Stephen Covey:
Seek first to understand, then to be understood (habit 5 from the 7 habits of highly effective people)

In my job as software architect, my mission is to find the RIGHT solutions; to solve problems. Solutions are not explained in C, C#, C++, java or even classes, but in plain natural language, possibly enhanced with pictures!

Adriaan van den Brand
Tuesday, September 10, 2002

"What kind of programming questions can be considered "A-ha! questions?"

What about "How would you swap the values of two integer variables without using a temporary variable?", the answer involving the behaviour of the exclusive-or operator:

a^=b; b^=a; a^=b;

But would you seriously want to employ some one who wrote code like this on a regular basis?

Andrew Simmons
Tuesday, September 10, 2002

Les,

are there software developers who are not puzzle freaks?

My whole work is solving puzzles in a way. I do not remember many 'puzzle' or 'aha' interview questions, though.

Have fun,

Jutta Jordans
Tuesday, September 10, 2002

Sadly all too many interviewers ask questions utterly irrelevant to the job they are actually trying to hire somebody to do.

As part of a technical test I once found a question that had a multiple choice answer, but none of them were correct. I pointed out the flaw in the code and why none of the answers worked, to be told (after a pause and a confused re-reading of the question by the interviewer!) it was a "trick question". Ha ha. Later on in the same test there was another question asking for the benefits of a particular routine. When I pointed out there weren't any unless you particularly like memory leaks, I got withering looks and had 'failed' the test at that point. So if I parroted the manual's entry on that routine, I'd have passed, but real world experience of its bugs... no good.

The great thing about this sort of experience, of course, is that you wouldn't WANT to work for people like this!

DB
Tuesday, September 10, 2002

"Personally, when I was asked this in an interview, I simply guessed the formula and then proved it by induction - if it holds for some positive integer N then it also holds for N+1, and since it obviously holds for N=1 it holds for all positive integers. I got turned down for being a smartarse."

Cool. I guess they didn't like the idea that you would analyze and try to prove all algorithms they used. :-) Tells you a lot about the company.

Frederik Slijkerman
Tuesday, September 10, 2002

I hate people who use these sorts of questions. From experience, they have no idea how to find out if you're suitable for the role. If you get the job, you find out after a while they actually wanted someone else.

I'm sorry, but I REALLY dislike this macho atmosphere that pervades IT. That everything is hard work, sweat, blood and tears. It's all driven by needless pressure, a lack of thought and an inability to think things through.

The best developers I know tend to be thoughtful people with vast, wide ranging interests. They have an opinion on a huge range of subjects and have read enough to have made that opinion sensibly. And they understand people, not just technology.
I recently applied for a couple of jobs. Job 1, (which I got) interviewed me as a person, about how I'd relate to the team. About how interested I was in learning things. Then did a technical interview. And finally a long rambling chat with a director to see if I was a "people-person" and not just a geek.

Job 2 went all HR on me. First interview was two hours of running IQ tests at me: what's the missing number in this series. Reading comprehension exercises. I was rude to them in the end, because I considered this a rude way to treat a professional. And the gotcha questions are as well.

It's rude. And it's not useful to know if people can come up with answers under pressure. Because you rarely want answers under pressure. That's how you get people guessing at schedules and costs and screwing up projects.

Software is chaotic enough without deliberately going out and hiring people who shoot first and ask questions if there's enough time later.

Katie Lucas
Tuesday, September 10, 2002

I have to strongly second Katie's response.

Even direct software related questions don't tell you that much. Some people don't do well under "answer in 5 minutes" pressure but can do very well on the job.

I nearly didn't get my first job out of college because I stumbled on a fairly simple circuit question due to nerves. However all three people who interviewed me later said it would have been a big mistake not to hire me. I got very good performance reviews.

Pressure and quiz questions have no place in an interview. An in depth discussion of the candidates previous projects and experience is a much batter way to find out what they can do.

sgf
Tuesday, September 10, 2002


  Great post, Katie.  You said it all.

Ricardo Antunes da Costa
Tuesday, September 10, 2002

"I was rude to them in the end, because I considered this a rude way to treat a professional."

Hm. I went through a process like this once. The salary was good, so I kept going.

The problem was, at the end -after passing every single IQ test, tech questions, a-ha! questions, lots of meetings, for about two weeks!- they wanted to do a psy test on me. And I agreed.

Guess what happened :-)

Leonardo Herrera
Tuesday, September 10, 2002

I agree with the last few posts, but how do you counter the "hey we've got 100 people to interview and we're busy and at least the tests help us weed people out" argument?

Adrian Gilby
Tuesday, September 10, 2002

If you disagree that strongly with the assessment methods chosen by a prospective employer, then you should be grateful for the red flag and early indication of the corporate culture.

Interviews are a two-way street; you should be evaluating the company and the interviewers at the same time they are evaluating you.

Dunno Wair
Tuesday, September 10, 2002

"hey we've got 100 people to interview and we're busy and at least the tests help us weed people out"

Maybe ask them how you tell a weed from a non-weed, and while they answer go around the room pulling plants out of their pots.

Robin Debreuil
Tuesday, September 10, 2002

"I agree with the last few posts, but how do you counter the 'hey we've got 100 people to interview and we're busy and at least the tests help us weed people out' argument?"

If you're stipulating the fact that the A-ha! interview questions are not a good indicator of a good or bad prospective employee, then time taken to administer the A-ha tests would be the counter to that argument.  You might as well take your stack of apps, flip a coin on each one.  Heads stay in consideration, Tails get a rejection letter.  Repeat until you have a sufficiently "weeded out" stack.  Helluva lot faster, and just as accurate. ;)

Dignified
Tuesday, September 10, 2002

Some organisations want fast, creative thinkers, and 'Aha' tests will probably find them (if they haven't seen the question before).

I've been interviewing a few years now, and
we have several good discriminating tests.

The best one is a design exercise. Give the candidate an hour in a room, and a nice simple OO design problem (OO is important to us) and see what they come up with. We are only loking for an inheritance tree and prototypes for a few key methods, so they should be able to complete with no pressure.

This really weeds out the ones who don't understand design principles, or who have learned OOD by rote. It also says a lot about people's style. The fussy details guys produce reams of code (although they weren't asked to) but have flaws in the design. The 'read one book' guys start looking for common data instead of thinking about 'is-a' and 'has-a'.

Incidentally the worst solution we've seen yet was from someone who said he had lectured an Object Oriented Programming course.

Our next best discriminator is to ask about the Software Engineering Process. We are usually recruting relatively junior people,
but it's surprising how many people don't even understand that you should design before you code. I would rather take the ones who fail the C++ technical knowledge questions than ones who don't understand that.

We also ask OO concepts questions ("What is Polymorphism?") and syntax questions ("What does a static method mean?") for junior people.

So far we've recruited only one lemon.

Interviewer
Tuesday, September 10, 2002

I agree completely with Katie that those who throw a test at you right off the bat are those who I'd rather not work for in the long run; the tests don't tell you anything.

I didn't get tested at my current job, and the interns knew it wasn't a test for them, either.  We had been discussing algorithm analysis classes and the weird assignments that seem to collect around them, and segued into 'puzzler'-type questions like the towers of Hanoi (they'd both seen that one), and the 7 gold links on the techinterview page came to mind.  As far as what kind of questions should be used, well, the "how much dirt is in the hole" question above is one I wouldn't use.  I'd have to look for something that was part intuition, part analysis, part approach.  The 7 gold links question is about right on all counts.

I'm extremely reluctant to administer formal tests as part of an interview, but the results of the puzzle questions were informative, and have since caused me to reconsider a few things.  I don't know if I'll pursue this kind of "informal" testing or not when interviewing... food for thought, is all.

van pelt
Tuesday, September 10, 2002

The problem with A-ha! questions is that they depend upon a "blinding flash of insight" for producing the best (and sometimes the correct) answer.

What's the sum of the first 1000 integers?

Sum = 0
FOR I = 1 TO 1000
  Sum = Sum + I
NEXT I

Do it in my head? I thought I was interviewing for a programmer position, not as a mental mathematician.

If you want to find out if I know some specific technology, ask me about it.

If you want to find out how I approach problems, give me one, or ask about something I did  previously.

If you want to find out how I handle fools, ask me an A-ha! question.

Steve Wheeler
Tuesday, September 10, 2002

I don't even ask the "what is polymorphism". Mostly because I get so bored of answering it myself that I've memorised a book description of it and I can trot that out without having to think. I can't help but think other people can do that but not actually do OO.

So... I pinched a question from an interview someone asked me. It's fantastic. Goes like this:

"You know the board game `Monopoly' -- tell me about how you'd implement it in OO...."

[Ahhh... doesn't always work. One chap I interviewed said "What's Monopoly". He was Eygptian, so I figured he might not have played it...]

There are two important things to look for: Good OO people will firstly mutter about "properties", "ownable properties", "buildable properties" and will secondly ask for a pen and paper to draw some sort of inheritance diagram on.

Structured programmers who've read a book on C++ and the rest of them will start muttering about arrays of strings to hold property names...

Anyone who says "you can have the board as a linked list... no you can't -- there are multiple ways into one of the squares... you could have them have successor pointers.... no you can't -- you have to be able to move backwards..." gets bonus points for attention to detail...


But seriously. Asking people "what is polymorphism" is asking for a book passage to be recited to you from memory...

You should have seen the looks on people's faces recently when I got asked "what're the differences between pointers and references in C++" and I replied "Your answer probably comes from a book and says something about not being able to have null references, but that's not true, you can... <I get a skeptical expression in reply> -- here I'll demonstrate..."

Katie Lucas
Tuesday, September 10, 2002

Does anyone here ever ask to see a code sample? I've found it very useful in weeding out superficially plausible candidates who turn out to write shambolic code. You can get some idea by asking a person to solve a problem in an interview - Joel has some good examples - but some really good people might not do themselves justice under the stress of an interview. If you were interviewing a graphic designer, I presume you'd want to see a portfolio of their work - why not try to apply the same idea to programmers?

Andrew Simmons
Tuesday, September 10, 2002

Its usually pointless to ask for a code sample.

1. they may not own anything they can show.  So  you have a loaded dice against them, unfair interviewing process.

2. they may require too much context to even bother looking and understanding.

3. it may be a collaborative effort

Simon Lucy
Tuesday, September 10, 2002

"If you were interviewing a graphic designer, I presume you'd want to see a portfolio of their work - why not try to apply the same idea to programmers?"

I've often wondered this.

When I was interviewing for my first job, I brought the code I had written for an inventory program along with.  They seemed to like that I had it, but didn't look at it too closely.  With the job I have currently, I don't know how to do it, as giving out any of my code would violate "we've got your a$$ in a sling" documents I signed just to get the job.

I like this idea, but for people who signed NDA's, I think you'd have to code a suitable example in your free time.  I suppose that's another good reason not to sign NDA's, though in today's market, I think a lot of prospective employers will just say "OK, thanks for your time then."

Dignified
Tuesday, September 10, 2002

Dignified, you're supposed to spend all your time outside of work also programming.  That's where you get all the samples.

semi
Tuesday, September 10, 2002

I agree with Katie Lucas on this.  Rarely do you want a developer that thinks they can come up with a solution to a complex problem in under 5 minutes.  Quick decisions like this can be time-consuming in the end and expensive.  Usually you want someone that is able to take into account time, budget, effects of the change on other components of the system, effects of the change on testing, effects of the change on documentaion, internal/external customers, and if the change somehow breaks the design/architecture of the system.  Code does not exist in a vacuum and especially in todays environment where disparate systems have to interact, quick fix changes have a devastating effect on the system.  What you DO want in a developer is someone who is thorough in coding, yet understands how the overall system is put together, why it was put together that way and what changes can be made without affecting the overall system in a major way. 

Ask these Q's:

"What was the architecture/design of your last project?  What were some pro/cons of the design?  Did it scale well?"

"How did you design your unit tests?  What about integration tests?"

"Did you use OOA/OOD methodologies?  Did you use design patterns?"

"Give me one example of a nasty bug that took some time to fix in (C++/Java/VB)"

"What was the most complex thing that you worked on?"

"Do you just jump in and start coding, or do you sit down with a few colleague and create a design, then quickly prototype it?"

You could also throw in a few of these coding questions that they wouldn't HAVE to code, but they could tell you how to do it, or give you some pseudocode.  Better yet, you can set up a laptop with the development environment, hook it up to the projector and ask them to do some simple coding techniques.  Tell them that it doesn't have to be perfect, but that you want to see in general how they approach a problem.  They can use help, the web, and any other resource but if you find that they start cut/paste code that isn't a good sign.  Also if they rely on looking up and reading about how virtual functions work, that isn't a good sign.  Now, if they forget about how whether strstr uses const* or not, they can look that up.  Ask them these:

"Reverse a string in place"
"Write a class CFoo that is derived from CBar, add some member functions to both that allocates an int[] using new, the destructor of both classes should delete the int[], and create the code to call these member functions, compile, set some breakpoints and step through the code."

"Write a class CFoo that is derived from CBar, add a virtual funtion to CBar and override this function in CFoo. compile and step through the code"

"Add a function that takes a pointer to a CBar class and deletes the pointer. compile and step.  Which destructors run?"

"Now declare the ~CBar as virtual.  Which destructors run?"

If this is a C++ position, this should cover inheritance, virtual functions, memory leaks, debugging, pointers, memory allocation, polymorphism.  Stuff you'd see in every day C++ development.

Tim
Tuesday, September 10, 2002

"Your answer probably comes from a book and says something about not being able to have null references, but that's not true, you can... <I get a skeptical expression in reply> -- here I'll demonstrate..."

How can you get a null reference, Katie? I ask because one of Joel's older articles about debugging Citydesk mentioned that the C++ spec states that references can't be null, but he had one machine crash with a null reference error.

"I like this idea, but for people who signed NDA's, I think you'd have to code a suitable example in your free time."

If I recall correctly, my contract states that ANY work I do (even in free time) belongs to the company.

Anonymous just in case
Tuesday, September 10, 2002

Null reference = "Evil, pure and simple." - Scott Meyers, MEC++
char *pc = 0;    // null pointer
char &rc = *pc;  // null reference

David Blume
Tuesday, September 10, 2002

That'll do it...

And, what's even worse, once you've created your null reference, you can happily call member functions through it. They crash the first time they try to access member data inside the function, not on the call. (Unless it's a virtual function).

Bah..

Katie Lucas
Wednesday, September 11, 2002

Katie, you are Verity Stob and I claim my £5

Simon Lucy
Wednesday, September 11, 2002

Pop quiz: why did UK national newspapers run the promotion about finding their representative and demanding your five pounds? Hint: They used to run it in the summer...

Katie Lucas
Wednesday, September 11, 2002

I throw questions that elicit individual responses. For a technical job I asked "what was your first solo break/fix that you were proud of?". You would be surprised how many dad's have had their home PC's broken and fixed by errant sons! The job went to the candidate that had on paper the least skills available. It was the route to the fix that mattered, not the speed of the journey.

I was on the other end and got caught out with what now seems a trivial code test in the first telephone interview. The first question on the face-to-face interview was the same trivial code test,  and I didn't impress when I admitted that I hadn't searched out the answer meantime.

Lawrence Attrill
Wednesday, September 11, 2002

"Pop quiz: why did UK national newspapers..."

Well umm you had to be holding a copy of their newspaper when you said it if I remember rightly so naturally it was to sell newspapers when people were on holiday. 

It always seemed to be in Margate as well.

Simon Lucy
Wednesday, September 11, 2002

I wrote an article on stupid interview questions.
It's at http://madman.weblogs.com/interview

MadMan
Thursday, September 12, 2002

My HR manager asked me to technically qualify people for 'analyst' positions after he'd seen them, and I admit that I was at bit of a loss. I tried the 'Design a public garbage can/a house' as joel suggests but these bombed out.
I've had more success with
- an aha! question to see if they kept on the subject at hand instead of spouting terminolgy, asked questions, were persistent but knew when to give up
- discussing previous projects (what are the tradeoffs of doing this in batch/nonbatch, etc)
- describing a business and asking them to model it (see if they could figure out the is-a, has-a from the description, seeing what questions they ask in return)

The 'design monopoly in 00' above seems another good question.  Any other good suggestions for interviewing analysts (not programmers)?

Yves
Friday, September 13, 2002

Yves -
I 100% agree with you. All the point of aha! questions is to see how they react, what questions they ask, etc. It's not about right vs. wrong answer.

MadMan -
great article! I always thought - how strange that so many interviewer ask those clichéd "personal" questions ("what is your greatest weakness?", etc.)... I think, it's laziness. Interviewer and interviewee always play certain game. When they both want to spend minimum amount of effort, they reduce the game to conventional answers to conventional questions. Both parties understand that the game is artificial and dull, but they still play it. Interviewee has a little choise, of course.

"Can you work under pressure and deadlines?"
"Yes"

Yuck.

Igor K.
Friday, September 13, 2002

Well the only trick to interviewing, and this relates to any kind of interview, job or otherwise , is to not ask closed questions and actually listen to the answers.

Asking questions which themselves contain a bias or can be satisfied with either a yes or no answer will get exactly what they deserve.

If you want to find out about how they handle pressure (dubious anyway, since they are already under pressure in the interview), then you can ask them non personal questions throughout the interview which gives you an idea as to how they react.  And, no you don't have to threaten them, frighten or bully them.

Simon Lucy
Saturday, September 14, 2002

I don't understand this "aha!" question business. Why would anyone use it and what good does it do? Having just been through a few interviews and started a new job, I've seen nothing like this. But maybe things are done differently when interviewing for engineering positions at the Ph.D. level.

The sorts of questions I was asked were:
- Give a presentation on my research
- Explain how you analyze, measure, or design system X for us
- A guy at a party said his company made product Y that did some voodoo. Is that voodoo real or is he just blowing smoke?
- What's your work style? (group vs. indifividal efforts? Project leader experiences? Hands-on or theory? etc)
- What would you like to know about us and our team?

Those types of questions seemed work well in both helping them assess me and vice versa. (Indeed, I had equal rejections and offers when all was finished.)

David Fischer
Sunday, September 22, 2002

sadfsadf

sadfsadf
Sunday, September 29, 2002

"And, what's even worse, once you've created your null reference, you can happily call member functions through it."

This is false.  A reference must always be bound to an object, and null is not an object.  More explicity, initializing a reference to null would require dereferencing a null pointer.  This gives undefined behavior in which case you can make no conclusions about what you can do.  Joel was correct, the standard does say this in 8.3.2/4.

If I asked a question in an interview, and instead of answering the question the interviewee instead told me what *my* answer was going to be and and that it is wrong, I would not be pleased.  I would be even less pleased if the interviewee then proceeded to tell me something that was in fact incorrect.

pedant
Friday, February 27, 2004

*  Recent Topics

*  Fog Creek Home