Fog Creek Software
Discussion Board




writing coding tests at interviews

Gave a couple of interviews recently all had half hour coding tests, in C++ and Java.I am pretty okay withthem, asking algorithmic and programming questions, but sometimes I find they ask arcane language features, which you have to be a total expert to know.How do people design these tests? and how do they weigh the tests overall in giving job offers?

Simon
Sunday, March 16, 2003

>  I find they ask arcane language features,
> which you have to be a total expert to know...

Maybe they are looking for a "total expert"?  Do you have some examples?

The Fake PC
Sunday, March 16, 2003

Well, something like why you can't pass a method in the parameter list of a method in Java( you could do it with func pointers in C++).

Simon
Sunday, March 16, 2003

  The problem with coding tests is that they test the specifics of what you remember, not what you know. Over the past year, I've done projects in C++, C, Delphi and C#, and debugged PowerBuilder code. There's a lot that I "know" that I may not remember exactly.

Mike Swaim
Sunday, March 16, 2003

Can you give more details about the format of the interview? Were you writing code responsively with an interviewer, or did they park you in a corner for a half hour with a computer?

That said, the question you cite sounds completely appropriate. I would expect someone who has designed & implemented large systems in either Java or C++ to be able to discuss when you might want to pass a method to another method, and how you might be able to do that.

Besides, even if the question was more esoteric, if you don't ask a candidate a question he or she can't answer, then you don't have a good feel for how much they know. I've sat through a ton of interviews (on the -ee side of the table) where the interviewer failed to ask any questions which would indicate whether my resume accurately reflected my knowledge and experience. I usually try to avoid working at such places, since they can't get lucky all the time. =)

Bryan
Sunday, March 16, 2003

I was writing code with pen and paper by myself.I know the method question is pretty legitimate, but are you supposed to get around 35-40 of these type of questions in 30 minutes?
Besides there was no interactivity with the test, just straight marking of the stuff written out, at least in one case. A bit more interactivity would certainly have helped, as I could explain where I was coming from in the answers.

simon
Sunday, March 16, 2003

That does sound like a lot of questions. Do you know how it fit into the rest of their interview process? That is, are they using the test as a significant factor, or are they just getting a gauge for you rough skill level? I could sort of understand it as an alternative to those "How do you rate your Java knowledge from 1-10" questions that I never know exactly how to answer.

Either way, it does sound like a lot of pressure, in a rather unnatural situation.

Bryan
Sunday, March 16, 2003

We use coding tests to "screen" applicants. We don't really care if our lead java programmer knows how map works, or the internals of a a delegate, but if someone puts 4 years advanced perl and 2 years C#, they better know both of those.  The best way to interview someone would probably be to go through code with them, and have them explain it.

Vincent Marquez
Sunday, March 16, 2003

There are no pat answers.

"How do people design these tests?"

Some employers purchase written tests, some employers "roll their own" tests, and then there are some employers who place you in front of a PC and have you take tests that were created by a testing service such as TekChek.

"and how do they weigh the tests overall in giving job offers?"

Depends on the type of employer you are interviewing with and the type of work you will be required to do. Most staffing/consulting firms place a lot of weight on coding and technology knowledge tests.  While some non I.T. businesses (i.e. a large insurance company) may care more about your soft skills or business knowledge.

One Programmer's Opinion
Sunday, March 16, 2003

That probably wasn't the most well-designed test, but it's still probably better than the typical BS interview questions that reveal nothing about your abilities or experience, like "what are your strengths and weaknesses?" and "do you work well under pressure?"

T. Norman
Sunday, March 16, 2003

I've seen a couple of these technical tests, and personally I find them to often be absurd.

While it is important to understand that generally they aren't used to rate the selected prospects, but rather is just to filter out a gross lack of knowledge, they still can really goad a consumate professional. When I first started in the computer industry, I made it a personal mission to memorize every element of the language (at the time C), every standard library call, etc. As it progressed, soon I was remembering Windows API calls, then the MFC, then the ATL, etc. During this time I regularly used C, C++, Pascal (and the Borland Delphi variant Object Pascal), along with some scripting languages. There came a point, around the time that online help because speedy, that it just became ABSURD to try to continually memorize every aspect of programming. Now I quite honestly memorize very little: I hit the MSDN, Deja (er Google Groups), and other resources daily for things that I've done a million times, often because I'm jumping between a half dozen languages in an average day, using a huge swath of technologies. I don't chastize myself at all, though, as computer science has progressed to the point that it's a lot like the medical sciences: Ones job isn't to remember the contents of every book, but rather to have the analytical skills to call upon the experiences they do have to narrow down to the answer. In other words rating candidates by whether they know every parameter to printf off by heart is just silly, and is more prone to get you an idiot savant than a practical advanced programmer.

Jimmy Chonga
Sunday, March 16, 2003

Non sequitor, but this reminded me of a story a friend of mine told me a few years back when he was interviewing for jobs:

He had gone through a number of "social" interviews (questions and talking mostly), and had one last 30 minute technical interview with some lead developer. The guy asked him to write out, on a white board, how my friend would design a hierarchy of classes to represent a binary tree.

My friend wrote out his tree setup, and made it such that a terminating leaf had two "EmptyNode" children (instead of using NULL or something). He even used the singleton pattern so that it wasn't more than a constant increase in space.

The guy who was interviewing him didn't believe it would work and basically threw my friend out of his office. Because he didn't understand the idea behind it or hadn't see something like it or ... whatever.

Needless to say, my friend wasn't made an offer -- not that he cared at that point.

P.S. The best technical interview _I_ ever had was one where they gave me a written out specification, a computer, a few C / C++ language reference books, a computer with a net connection, and told me to have at. Simple maze world type program. But *shrug* I work well under that sort of pressure.

Steve C.
Monday, March 17, 2003

Just to be clear, they only gave me one computer, with a net connection. My age-old foe, proof reading, we meet again!

Steve C.
Monday, March 17, 2003

I don't ask people to write code. Every member of my family can write code. They can read the manuals, they can write code that will go through the interpreter or compiler and the program will run. This is from a developer who left the industry in the 70s, a mechanical engineer and an accountant.

From that I'm extrapolating ANYONE can write code. Anyone can write a million lines of code.

I want to find out if they can ORGANISE those million lines into something useful, because that's the deciding point.

So I ask design based questions. And I want to know how coherent the candidate is, so I ask them the questions verbally. If they can't explain the stuff face to face they won't be able to write the documentation.

It's the single thing that angers me most about technical interviews. They're usually not about how technically competent one is, they're about "gotcha!!" questions. Quick, what parameters does OBSCURE_FUNCTION take?

The correct reply should be "I dunno, pass me a manual"

It's like the difference between remembering the time in the morning the first shots of Waterloo were fired and having an understanding of the political background behind why it happened...

Katie Lucas
Monday, March 17, 2003

"From that I'm extrapolating ANYONE can write code. Anyone can write a million lines of code."

But in reality, there are many people who call themselves programmers who CAN'T write code in the languages they put on their resume.  A coding test can be very effective at filtering out pretenders, as long as you look at their answers with the goal of evaluating whether they have actually worked with the language and understand the common concepts, and you're forgiving about the nitty gritty details of syntax and function names.

T. Norman
Monday, March 17, 2003

One thing I wouldn't like is sitting there trying to remember, "Was it def, defun, define?"  (Especially if it were Java... which has nothing like that.)  Maybe that means it's good to bring a portfolio of stuff in different languages, to quickly glance at and remember the surface syntax.

Or bring a little util on a notebook that launches apps you've written and shows their sourcecode.

Sounds important to feel out interviewers on whether they have some macho BS with coding.  My normal mode is to work on paper (high bandwidth) and then do data entry.  But sometimes it's best to switch modes to one that looks more like LOC is being pumped out.

Tj
Monday, March 17, 2003

I think I've mentioned before that my preferred method of dealing with "would you please sit down and take our coding test?" is to reply "No problem. Would you please sit down and take my management quiz?"

1) Who is Deming?
2) What is "Code Complete"?
3) Define the Mythical Man Month
4) Why is micromanagement bad?
5) Why is "lines of code" a poor metric for measuring programming productivity?

Philo

Philo
Monday, March 17, 2003

Philo,

Haha...Mind if I borrow that little quiz to my next interview?
Sad thing is I think most managers I've ever had would fail pretty miserably on that quiz.

Patrik
Monday, March 17, 2003

Philo, those would come under the section "Do you have any questions for us?"

Asking difficult questions really depends upon how desperate one is for a job (times is hard, etc).

If you want to be a bastard, try asking about the company's strategic medium term business objectives and how they intend to achieve them. What is expected of someone in your potential role to help them attain these objectives? What incentives do they offer someone prepared to exceed the company’s set targets? These are both a take on the standard “where do you see yourself in 2 years time” and “how would you show commitment to the company and produce results” interview questions.

If the company intends to float or sell itself and is offering options, find out how the value of the company is to be determined. Ask “how do you intend to improve the P/E ratio prior to IPO?”

It is equally important to understand these questions and the answers yourself, or you’ll end up looking stupid.

Like I said, do you really want the job?

Justin
Monday, March 17, 2003

Justin - agreed that it boils down to "how badly do you want the job" balanced with "do you feel the need to assert your authority?"

And I prefer the "please take my quiz" method to asking directly because I want to make the point of how silly it is to make me burn my time to take their test, especially if they're not willing to make the same commitment.

And this wouldn't apply in a Joel-style interview of "write a routine on the whiteboard"; just the silly "what is the value of adVarChar" type quizzes.

FWIW, I haven't had an opportunity to try this yet, though I do have the quiz printed out and in my "interview folder"

Philo

Philo
Monday, March 17, 2003

Philo,

so I guess your answer to the question "how badly do you want this job" obviously would be "you can stuff it up your ars for all I care".
While I would even welcome serious questions about the state or direction of the department or even management style, pulling  a smartass "here's my quiz" to me would only indicate poor people skills and arrogance on your part.

Just me (Sir to you)
Monday, March 17, 2003

It all comes down to how one perceives the employer/employee relationship. In days of yore this relationship was akin to a mother/child relationship, where the employer was the employer for life (or a large portion thereof), and the employee was loyal and subservient: Employee churn was virtually unheard of. In today's world though this obsolete impression of employers isn't valid, nor are the trappings that go along with it (such as "how many hoops will you jump through to work for us?"). Are there people who are willing to submit to a endless barrage of inane technical tests and long winded interviews? Absolutely: They're called the desparate (and this may come as a big surprize to employers, which I am myself, licking their chops eager to take advantage of the down market: Competent employees are as in demand as they ever were, and if you treat them like they need to beg they'll be at my door and you'll get the bottom feeders. Enjoy your demise). Professionals though just find this insulting. I treat empoyees (actually contractors) as little individual businesses, not as children.

A technically proficient interviewer can easily gauge the technical abilities and intelligence of a prospective hire via even casual conservation about past projects, how they were implemented, and the technologies used, etc: Trying to trick them with absurd nuances of syntax or obscure SQL extended functions is just ridiculous (especially when the tester is generally clueless, and pulled the questions via a surface glance, not realizing that there are other, often superior, methods of doing the same thing).

Jimmy Chonga
Monday, March 17, 2003

I have a standard question when asked by an interviewer if there is anything I want to know:

"What is your staff turnover?"

- Then watch the reaction, and examine the answer carefully.

David B. Wildgoose
Monday, March 17, 2003

Sir:
"While I would even welcome serious questions about the state or direction of the department or even management style, pulling  a smartass "here's my quiz" to me would only indicate poor people skills and arrogance on your part. "

Well then you got the message. ;-)
Because I think calling me in for an interview then giving me a test and sitting me in a room to take it indicate poor people skills and arrogance on the part of the interviewer. Again, let me reiterate - this isn't a shotgun reply; it depends on the situation. It most definitely would NOT apply in Joel's "here's a white board, show me how you'd invert a string" interview style. It's only for the case I mentioned above.

And I can do it because in that kind of situation I've probably already decided not to take the job. Now I'm just wasting your time as you wasted mine. [grin]

Philo

Philo
Monday, March 17, 2003

If you need the job badly enough, it doesn't matter what the answers to the questions are anyway. You'll go work for a bad manager to cover your mortgage payment until you can find something else. In that case, there's no point in asking. If, however, you are interviewing from a position of strength, and the interviewer has allowed you to keep the confidence you had when you walked in, then I can see how it would be possible to ask whether she's read Mythical Man Month without being confrontational and snotty.

Pete
Monday, March 17, 2003

What I think is funny, is that a lot of people are saying its insulting to be given a programming test, yet I'm pretty sure every one of you would test me just because I look young.  Even my current boss said I was the only one he hired that had to take a number of aptitude and programming tests. 
    For one, I dont' think you can judge someone by the way they look or whats on their resume.  A 35 year old guy in a nice suite and "8 years j2EE" on their resume might not be able to code near as well as the h1b with 2 years experience. 

Vincent Marquez
Monday, March 17, 2003

I dislike tests. If you need a short-term CYA codemonkey, then tests are requisite, without a doubt. If you need a programmer, I'd say you're probably better off with an XP pair-programming session - it proves that the applicant has skills and that personalities mesh at the same time.

Pleeeeeassssse, next time I go on an interview, and you want to give me a "coding" test, give me a real text editing application or IDE instead of DreamWeaverMX (I believe that a crackhead designed that UI, but that's beyond the point). Right tool for the job, or notpad.exe is my motto. The only thing I knew at that point was "this project is lost, with or without me". Remeber, if the applicant is worth a damn, then they will be unit-testing the tester.

-j
Monday, March 17, 2003

I sat a C++ interview  test recently for which you had to know  the following:

- virtual functions called in base class constructors aren't called polymorphically.
- const data members can only be initialised in the constructor
- static data members can't be initalised in the constructor
- sizeof(a virtual derived class) is greater than the sum of its data members
- if you don't declare destructors virtual they mightn't all get called.

to get an interview you had to get a perfect score on the test.

Too obscure, or good questions?

goosbane
Monday, March 17, 2003

This stuff is on my mind, since I'll be inteviewing someone tomorrow, for the first time in months.

I have a few programming questions in mind, which are desgined test a person's programming thinking skills instead of their explicit knowledge. Verbal pseudo-code and pictures will be sufficient, since it's so unnatural to write code with pen and paper.

It's good to have some objective or semi-objective questions, to fairly evaluate people and compare them to other interviewees. Plus, you can argue your impression more forcefully when you have evidence to back them up.

Julian
Tuesday, March 18, 2003

The answers you posted should be minimum requirements for anyone who claims to know OOP, whatever the language. I guess it depends how the questions were phrased, and if it was multiple guess or not. Kinda cool that they checked early if you knew about defensive programming techiques for memory leaks too. I guess I should expect security questions in the near future too as MS is getting management on the secure coding bandwagon.

On other occasions, I've had "Architects" question me about OOP, and they don't understand the difference between transitive and polymorphic, much less what reflexive is, thanks to their fine VB upbringing. The only possible "correct" answer is the only way they know how to expain it, and trying to figure out their limits of OOP to explain it to them in their terms is difficult.

Establishing good communication skills is also important (in some cases). Has anyone ever had to take a written communication test? Or is just taken for granted that if you know exactly the order in which a COM object's VTABLE is loaded, you must know how to communicate.

-j
Tuesday, March 18, 2003

>Verbal pseudo-code and pictures will be sufficient, since it's so unnatural to write code with pen and paper.

Mon dieu, c'est faux pas!!!!

Try UML. It accepts everything from cave drawings to pencil sketch to vase painiting to super-mega IDE's from our friends at Rational. Works well with all mediums (except for Inkblot's and vanishing ink). And not only is natural after you get the hang of it, it's pretty much "universal" and faster than coding or trying anything "verbal" with code.

codito
Tuesday, March 18, 2003

I suspect all of those C++ questions are answered in Marshall Cline's C++ FAQ.  Is it reasonable that a candidate, for that position, be familiar with the FAQ?

That said, I'd not be surprised if the questions were unfair; where a candidate, not knowing which answer is expected, cannot work out which question is being asked.

(My own personal example: "What are the three ways a copy constructor can be called?"  Doesn't sound too bad, until you realize there are more than three.  Oops....)

Danil
Tuesday, March 18, 2003

I just had an interview where they asked if I had any questions and I responded, "I don't have any questions for you pertaining to the position and its responsibilities or about the organization in general as I believe, during the interview process, all of my questions have been answered."

At first the 6 person interview committee was a tad bit quiet, then one programmer piped up and said, "That's a fair answer."  Then everyone chuckled and nodded and verbally agreed.


Tuesday, March 18, 2003

Danil, it's when passing by value as a parameter, returning by value in a method, and when doing something like-

Object a=b;

Simon
Tuesday, March 18, 2003

Simon - there are a more than that, if you look for them. 

Though you are very close to what I remember the "correct" answer being.  One of the keys to reaching that answer is recognizing that pass by value is a candidate, but if it is, there are two others which are similar that also qualify.

Danil
Tuesday, March 18, 2003

Don't forget explicit instantiation.

Class1 c1(...);
Class2 c2(c1);  // or Class2* c2 = new Class2(c1);

Brad (dotnetguy.techieswithcats.com)
Saturday, March 22, 2003


I think some questions that are related to basic knowledge  must be put in an interview. For example: What is the main diffrenece between new and malloc? .. That is because the employers dont want to hire ppl with university degree, "experience" and so on  (a wise - guy type) that cant really work because of major lacks. The practical side is to be revealed with specific code - related problems and with basic design idioms like designing a small program arhitecture. The virtual functions matter is a must in interviews as well in my oppinion.
I dont like when the guys put questions that they dont understand at large, but want to stress the candidate, or to fill their lacks of knowledge from the candidates answers. For example, "When do an application needs multithreading?". That is a bad question because this is a problem that isnt clarified yet in all aspects at a theoretical and practical level. Many experts are arguing on that so, a candidate for a job cant really answer completely and satisfactory, and the employer knows that, but is a fucking pervert.

Cristian Ionescu
Wednesday, August 25, 2004

*  Recent Topics

*  Fog Creek Home