|
Requiring pre-interview homework
Recent topics have discussed ways to "watch how the juggler juggles" during a job interview. The time constraints of an interview force us to assign trivial problems, however, which have only limited use in evaluating candidates. In the past, I've emailed candidates before their interviews and asked them to bring a non-trivial sample of code they had written. This turned out to be problematic because there were legal issues with code they had written at work, and not all of them had non-work code. Even when they did have something they could show me, typically the project had been written by a team so it was hard to point to code they "owned" completely.
How about this approach: set up a web page with a description of a problem to solve using the target language (C, Java, etc.). Send emails to all the candidates whose resumes passed muster and tell them to logon to the web page, complete the problem, and submit it. This should be a program that takes at least a few hours to write. Evaluate the responses at your leisure and choose whom to invite for an interview.
Pros:
1. The required program will be large enough to provide some real insight to the candidate's abilities. For example, do they know how to design objects? Is their code clear? is it efficient? Etc.
2. For those candidates who arrive for an interview, you already have a non-trivial project you can grill, er, discuss with them.
3. It weeds out candidates who have no real chance of meeting your requirements.
Cons:
1. Might turn off some good people that would have come in for an interview otherwise.
2. Need to make sure that all candidates can submit a solution. Perhaps it will be necessary to provide them with the development tool (gcc?).
I think this approach allows finding the (small) group of good candidates in a mountain of resumes. It doesn't replace an interview, of course. What do you think?
Oren
Monday, December 1, 2003
Are you going to compensate the candidate for the:
"... This should be a program that takes at
least a few hours to write. ..."
I would think that if you go beyond the bounds of a "normal" interview -- and I think you are here, that it would be "the right thing to do" to compensate the candidate for the time invested. You don't work for free, why should they?
You've also got the ever present issue of: Was it actually the candidate that completed and submitted the work? Or was it a buddy, relative, hired help, etc. that completed it, submitted it, and got the interview for the candidate? How are you going to police this?
Sgt. Sausage
Monday, December 1, 2003
The problem is, do you really know if they wrote it.
Maybe schedule the interview at 9:00am and give them until lunch to do the project. And spend the rest of the day talking about the project, and other interview stuff.
Michael Sica
Monday, December 1, 2003
Here's one way to do it. They advertise on slashdot.
http://www.itasoftware.com/careers/programmers.php
The key to this is that these are puzzles, not "work." And they made the interesting choice to not require any specific language/compiler, which is a sign of confidence and attracts the kind of programmer they want. Not every company can do this, which they are aware of.
Tayssir John Gabbour
Monday, December 1, 2003
To: Sgt. Sausage
Why should he pay for the guys? If they want to go to the interview they should do this. There is no contract between them, why do you want to pay for them?
name not available
Monday, December 1, 2003
Actually, this is precisely what I've been doing while trying to hire a programming security architect and a software engineer. We have four questions:
1. Design a C++ string concatenation algorithm that optimises concatenation speed given a standard desktop environment (reasonable amounts of RAM and quick processors). Explain the choice of algorithm and its limitations.
2. Write a C++ clone function that will clone any given C++ object passed as a pointer parameter. The only requirement is that the class to clone implement a copy constructor.
3. Write a Java helper class to copy the properties from one object to another, regardless of class. This question was inspired by the Jakarta Bean Utils; and I will accept "Use the Jakarta Bean Utils" as a valid answer.
4. Write a Windows/C++ class that will load and display an image with alpha blending. It is acceptable to use operating system supplied functions/classes.
5. For extra credit, write a C++ method that returns the number of elements in an array.
One of the big draw backs I've seen is that most people who respond to the advertisement, don't complete the quiz. Either it scares them off or it pisses them off. Either way, that's fine by me.
I'm actually very lenient when scoring the quiz. If it looks like the candidate understands the question even if the answer wasn't entirely correct, he gets full marks and an offer for an interview.
Jeff
Monday, December 1, 2003
Jeff -
I am curious how you accomplish #5. Please correct me if I am wrong, but an array (assuming we are not using vector<> or equivalent) doesn't store the size. Are you accomplishing this through some sort of RTTI trick?
Mike Swieton
Monday, December 1, 2003
==>Why should he pay for the guys? If they want to go to the interview they should do this. There is no contract between them, why do you want to pay for them?
Contract, schmontract. Of course there's no contract. It's an interview for $DIETY's sake.
If you're willing to work for free, it's your prerogative.
If you're wanting/needing the job bad enough, then sure, by all means, do whatever it takes, including working for free. I probably wouldn't.
I simply wouldn't put in any extra effort unless at least an offer to pay me was made. It provides a clear, unambiguous signal from the company that they acknowledge:
(a) My time is valuable. To them and to me.
(b) As such, from (a) above, my time has an inherent
worth, and this pegs an actual value to what
they think my time is worth. If it's too low, then
I can skip the effort of working half a day toiling
on the "busywork" of their interview process and
move on to the next interview.
(c) Let me put it this way. If you were about to hire, say
a lawn maintenance crew, you would in no
way even think about saying to them: " Hey,
why don't you come out to my place and mow
the lawn for 3 or 4 hours and when you get done
I'll tell you if you've got the job. O yeah, and by the
way, I expect that 3 or 4 hours of work you're
doing to be free of charge!" Yeah right. That
works.
NOT! Try it and let us know how it went.
If you don't like it in terms of a lawn-care crew --
then put it in terms of some other profession.
Say your accountant, or your attorney, or
your physician. " Sure doc, before I hire you
for my brain tumor-ectomy, I'd like you to
lobotomize me first. <grin> But there's one catch.
You've got to do it for free!"
Put it in terms of any other professional
service that you use. Hell put it into terms
of general unskilled labor: "Yeah, we might
hire you (to some factory worker), but first
you've gotta do a half a shift on the line
plugging part A into part B for 4 hours. We
might hire you after that, but for that half
a day you'd be working for free! We'll let
you know after you've completed the shift."
Ask your plumber: " I might hire you for
this job, but first, can you spend 4 hours
soldering random pipes together. I don't
really need them, I just want to check if
you know what you're doing first. After
I check them to my satisfaction, I might
put you on the short-list for consideration
for the job. The specs are here <random URL>.
Let me know when you've got the pipes all
soldered up and assembled. We'll get back
to ya and let you know. Don't call us, we'll
call you!"
In any context I can think of, it makes no
sense to ask a candidate to work for free.
Why on earth should I be willing to work
for free when my accountant, my attorney,
my doctor, my lawn guy, my plumber (etc.
Insert skill/profession of your choice) won't
do it.
Why are you so willing to do it just because
you slog code for a living?
That being said, if I wanted the job bad enough, I'd probably refuse it (I'm not stupid)-- but the offer itself would go a long way toward my trusting the company I'm about to give several hours of my life to on the chance that, maybe, possibly, I might get hired.
Sgt. Sausage
Monday, December 1, 2003
While I can't be *certain* that the candidate didn't have "help", I don't think this would be a big problem. First, I would require them to sign a statement that this was their own work. This will eliminate all but the really bad apples. Second, I would make it clear that during the interview we will discuss their design decisions and the tradeoffs they had made. I don't think a complete cheater could pass such an interview.
I can't have people solve the problem while at the office because I intend to use this technique in order to decide whom to invite in the first place.
I like the itasoftware link. I would require that the candidates solve one of these problems, however. As I've mentioned in my original post, sample code that the candidates choose is often not useful to me.
I'm now leaning towards the view that setting up a web page might not be cost effective; it's probably useful only if you're IBM and you get a thousand resumes for an opening. For a smaller shop, asking candidates to email their solutions would be fine.
The task that will be requested of candidates can be tailored to the job. For example, this might apply to technical writers by asking them to document a very small program that will be provided by the company. For SysAdmins, have them write a detailed list of steps for creating a cron job that backs-up some hypothetical network. For moose trainers, have them explain how not to get bitten by a moose. The possibilities are endless...
Oren
Monday, December 1, 2003
Sgt. Sausage has raised a valid point: candidates may not be willing to spend several hours on this problem. He suggests that I should pay them to do it, but I don't think that would work. I know *I* wouldn't solve some problem for the sort of payment that would be reasonable here, and in any case the practical obstacles are formidable.
I'm more concerned with the possibility that a good candidate might decide not to spend the required effort, and will move on. In order to have a high degree of certainty that strong candidates will complete the required task, one of the following should be true:
1. My company should be well-known and candidates will want to work in it due to previous knowledge. A good example of such a company is id software; perhaps Fog Creek also qualifies. This is the best case.
2. The job is especially interesting or lucrative.
3. The job market is terrible and people will do anything for a job.
I don't want to rely on #3 being true. Besides the fact that this situation won't last forever, I want people to join my company because they're eager to do so. The unfortunate conclusion may be that I must already have a great reputation before I can put into effect my dastardly plan.
Oren
Monday, December 1, 2003
Regarding the problem of who actually did the work: I don't really care, because I plan to discuss the problems with the candidate during the interview. And I'm confident this wouldn't get past me.
Mike:
template <typename T, size_t N>
size_t arraySize( T (&array)[N] )
{
return N;
}
Because it borders on trivia, I've made it extra credit. But it isn't that hard; and can be found in several C++ primer books. Oddly enough, it's also incredibly useful.
Jeff
Monday, December 1, 2003
Jeff -
Interesting code bit. A bit inefficient, having the compiler create a new instance of the function for every different type or size of array you do that for.
A question though: how does that handle dynamically allocated arrays? Being a templated function, the compiler creates the function at compile-time, so how can that work on an array whose size is determined at runtime?
Please correct me if I'm totally off-base here, my c++ knowledge isn't perfect (still need to buy that copy of Stroustrup's book).
Mike Swieton
Monday, December 1, 2003
Basically it won’t even compile unless N can be deduced at compile time.
coresi
Monday, December 1, 2003
The arraySize function shouldn’t be at all inefficient with a decent compiler since it would be a great candidate for inlining resulting in calls to arraySize being replaced with the equivalent of a hardcoded constant (I just verified that Visual C++ 7 works this way). You don’t get much more efficient than that.
SomeBody
Monday, December 1, 2003
Here's the problem. The talented people have no problem getting jobs, so you are simply giving them reasons to ignore your company.
I would never do several hours work for free just to get an interview. Totally pointless.
Monday, December 1, 2003
I agree with Blank, from personal experience. The superstars you really want already have a table full of standing job offers from friends and former associates and aren't going to waste time writing string concatenation functions for you or thinking up arcane template tricks or mind-compiling printf statements.
Doesn't the (verifiable) MIT degree speak for itself? Or the (verifiable) book and article publications? Or the (verifiable) long-time position as Sr. Scientist at MajorSoftwareCorp?
Superstar Bob (MIT '89)
Monday, December 1, 2003
1. Keep the test, but I would suggest the test after the interview, if you are serious. Then no one's time is wasted. Tell them what you expect. (Full doco, just the code, etc.)
2. You don't pay. This is a win-win. They get a new job, which they want. You see what skills they have.
3. Make it a sample question. Don't have them solve your foobar, you have been struggling with since version 3.2
4. Grade the curve. If it works, it passed. Then bring them in to talk about it. The discussion is more important than the code itself.
5. You will not know they actually wrote it until the discussion. However, if they are going to lie does it matter where and when?
HangUp
Monday, December 1, 2003
Sgt.,
The world is not adrift in MIT graduates who bring value. Most businesses need to decide from the available pool. High ideals candidates generally think more of themselves than the rest of us would of them.
As for the lawn crew analogy, it is a herring. I am spending $16/year for a lawn guy, 86k+ a year for developer. If you cannot see the difference, college was probably a disappointment.
The same is true for a plummer. They will provide you with good references and you can check their current work. Feel free to give me your current employer if you want me to ask them how you do. Again, a herring as the loss on a plummer is minimal as I watch them work.
Lawyers, accountants and Doctors are all certified by a government agencies as to their having met minimum requirements. We have no such thing for software. Again a herring.
As for your time being worth something, so is the company's. Would you expect them to pay you to fill out applications, paperwork, your time in the interview? You seem to be stuck on coding like it is some extraordinary burden.
GETREAL
Monday, December 1, 2003
What's wrong with my favorite interview method, from both sides of the table:
Describe a programming problem you faced, and how you solved it. Ask drill-down questions as necessary.
IMHO a good candidate will be able to communicate that they know what they're doing, and a good interviewer will be able to tell.
From interviewers - what problems have you run into with this question?
Philo
Wisdom trumps knowledge
Monday, December 1, 2003
I agree with Sausage and blank. I would not do several hours of work in advance on the off chance of maybe possibly getting an interview. I do think that someone who is *desparate* would do so, however. So whether this tactic works depends on whether you think that there are many desparate out of work developers who have the skills you need and who will be responding to your ad.
An offer to pay, or some other method might improve the draw. You'll always get the desparate ones (every one advertising a job hears from them), but you'll also get the non-desparate ones as well.
You really need to make the interview interesting for the candidate as well. If they are going to be there for a while, you should have someone show them around, have one of the other developers take them out to a *nice* place for lunch, and if they are coming in from out of town, you should pay for their travel expenses and offer to put them up in a modest hotel for the night.
You don't have to do any of this of course, but like I said, the sort of people you will attract will depend on the bait you use to attract them.
If I was giving up a weekend or vacation to go interview someplace, I would certainly want to know that I wasn't throwing pearls before swines. How my treatment was being presented would give me a clue to this.
Dennis Atkins
Monday, December 1, 2003
Also, regarding your #3 Oren, the job market for developers is no longer in the can. Stuff has really picked up in the last 6 months and I regularly turn work down, as do many people I know. Just FYI.
Dennis Atkins
Monday, December 1, 2003
Sausage,
It's pretty unlikely that the company is going to take your "interview code" and profit from it in any way, form or fashion. The only reason that they would want you to demonstrate code or solve a problem is to see if you're worth hiring. In that case, it's laughable to suggest that they pay you for your time.
Mark Hoffman
Tuesday, December 2, 2003
One more thing...
It would only make sense for a company to request such code from a candidate only if they candidate has already passed a number of screening interviews.
I agree with Denis, that I wouldn't agree to spend several hours writing code for an interview unless I felt that the company was a good fit.
Mark Hoffman
Tuesday, December 2, 2003
==>It's pretty unlikely that the company is going to take your "interview code" and profit from it in any way, form or fashion.
I couldn't care less if they made a profit from the code or if the code made a trip straight into the bit bucket never again to see the light of day. Them making (or not making) money/profit is not the point.
==>The only reason that they would want you to demonstrate code or solve a problem is to see if you're worth hiring.
The point is that it's several hours of my life and that I'll never get back. Period. We have a finite number of hours on this earth and I'm not going to waste my time with someone who thinks it's worthless. That time is valuable time, whether they want to make a profit or whether they just want to give me a test-drive and check me out.
==>In that case, it's laughable to suggest that they pay you for your time.
I think it's quite laughable that you place so little a value your time. To each his own.
Sgt. Sausage
Tuesday, December 2, 2003
"We have four questions:
1. ...
2. ...
3. ...
4. ...
5. ..."
Uh. Yeah. Ok. Forgive me if I would say that I wouldn't want to work for a boss who can't count/specify properly.
Or maybe I would -
"We'll five you 4K per month."
"The same way that four was counted in the interview?"
Tuesday, December 2, 2003
It's really interesting that many people seem stuck on the figure of "several hours". One of the things we measure with the four question (or five if you count the extra credit question) quiz is how much time elapses between when I email the candidate the quiz and when he responds.
Every single candidate who has completed the quiz reports that it took about an hour, total. Given that most candidates respond several days later, I expect the questions have percolated in the back of their minds for some time. And when they finally sit down to work the quiz, their subconscious provides most of the answers.
That's good. Frankly, I wouldn't hire anyone who claimed the quiz took them hours. We're looking to fill a senior staff position, although I suspect my boss has set the maximum salary too low.
Jeff
Tuesday, December 2, 2003
Speaking of red herrings - the examples given re: lawyers, accountants, doctors, plumbers and even lawn care are flawed because they are one time/irregular events versus full time employment.
I could also point out that plumbers have apprenticeship periods. Doctors must first complete a residency. etc
And there are also "counter-examples" (equally flawed). For instance, before you hire a caterer, it is normal to receive free food samples.
Phibian
Tuesday, December 2, 2003
Philo's method is the only one which gives you a chance of hiring someone that makes sense on all levels.
Talk to people, listen, respond.
Hypothetical or academic questions and quizzes are just going to set up barriers. Sure someone will leap them better than someone else but if they're inarticulate in doing so what use are they?
Don't hire compilers, hire people.
Simon Lucy
Tuesday, December 2, 2003
"The point is that it's several hours of my life and that I'll never get back. Period. We have a finite number of hours on this earth and I'm not going to waste my time with someone who thinks it's worthless."
So it's about time, and not the work you produce right? So it would be fair to say that if you were to interview with a company halfway across the country, you would expect them to pay you for your time during the interview? Paying travel costs is normal, but I would expect that you would require them to compensate you for the time it took to interview, right?
Do you invoice the companies that grant you an interview?
<g>
Mark Hoffman
Tuesday, December 2, 2003
If a company is willing to fly you in for an interview, it is likely you are one of a set of ten or less candidates still in consideration for the position. Thus there is a reasonable possibility of getting the job.
But when given hours of work before an interview, you could be just one of hundreds. If you already have a job or you have offers, it makes little sense to invest that time for the remote possibility of even landing an interview, unless the company has demonstrated that the job is unusually lucrative or interesting.
Requiring pre-interview homework is just a way to narrow your candidates down to the most desperate bunch.
T. Norman
Tuesday, December 2, 2003
"We're looking to fill a senior staff position, although I suspect my boss has set the maximum salary too low. "
Ah.
Dennis Atkins
Wednesday, December 3, 2003
Dennis, yes: Ah. If you've followed my thread on "Recruiting Ethical Problem", you probably have some sense of what's wrong with my current employer.
Oddly enough, I have an interview tomorrow. Haven't been given any homework. Haven't even had a technical screening yet.
What this tells me about this prospective company:
Either my resume completely impressed them (I can always hope) or, they don't really have enough technical depth to screen candidates.
Still, it could be a good gig; and will probably be better than my current position.
Jeff
Wednesday, December 3, 2003
 Recent Topics
Fog Creek Home
|
