Fog Creek Software
Discussion Board




software developers needed

We are a small company looking to expand with a lot of projects in hand.Unfortunately the problem has been getting the right people.
After interviewing 20 candidates over the last 4 months,
only 2 or 3 seemed good enough, but they left for doing grad school.
We are a Linux based company so we are looking for people with good Linux, C++, Java and system programming skills and XML/XSL knowledge
Our interviews involve a 2 hour technical interview with different parts - including OO design,system programming,database knowledge, unix tools knowledge,
and crytpography knowledge(we do some crypto work)
with a variety of knowledge based questions and problems to solve.
I will say it is very rigerous, and I found out of the last
20 people interviewed only 3 fitted our needs, and 2 left for grad school with 1 getting a better offer, we could not persuade him.
Is the level of knowledge among programmers so bad, they can't answer basic OO and system level questions?

People_wanted
Monday, July 28, 2003

So out of 20 people you only found 1 that was available, but you couldn't give him the amount of $ he was being offered elsewhere.  Sounds like you're searching at the bottom of the barrel (low salary) for a gem.  Get someone with lots of experience and pay him/her.

Alternatively, rethink what level of developer you're looking for if you can't pay a reasonable salary for your current target.  In other words, if you can only afford a developer with 3 years of experience, don't try to find one with 10 years of experience at the same price.

GiorgioG
Monday, July 28, 2003

I recall during one interview, a peer asking if the person knew what instruction would compare two packed values in IBM Assembler.  No offense to my team mate, but he was looking for a specific answer to a very specific question (of which more than one answer is possible)

My point?  If something is "obvious" or "simple" would not the fact that only 0.5% of the people being interviewed indicate otherwise?  Maybe you could give us an example of the questions and what you are expecting?  Perhaps, it is the questions not the people being interviewed. 

BigRoy
Monday, July 28, 2003

Some questions asked were -

1. Difference between stack and heap
2. what is auto -ptr, when to use it in c++?
3. Why should you not throw and exception in a destructor?
4. should you call a virtual method in a constructor?
5. difference between pass by reference and pass by value
6. questions on pre-emption,threads, pipes etc...
7. forking of processes and process id's
8. difference between overloading and overriding

etc..
I was expecting a developer with 2-3 years experience to know these. as well as some simple 2-3 line programs
to write.
Was I wrong?

People_wanted
Monday, July 28, 2003

I have to agree with GeorgioG, and it is a common mistake I see made again and again. 

<rant>
I believe companies have too great a reliance on the Human Resources cost containment myth.  This is creating an environment where being very good has no advantage.  Instead of hiring the best, they take the median, subtract 10% (because the economy is bad) and then tell us this is the best they can find. 
</rant>
That off my chest, spend more, buy fewer. 

Three very good developers are better than 5 good ones, and probably 7+ average ones.  And do not kid yourself when it comes to the output.  Top tier developers save you uncounted hours long term.  If you simply need coders, where just LOC matter, go to the local 2 year college.  If you are looking for results, without the need to monitor every activity going on, look for seasoned, top tier, professionals.  They do not come cheap, because they are not interchangeable, with "any" developer.

Mike Gamerland
Monday, July 28, 2003

5% is about right. I think it's because the other 19 guys never get hired, so they are interviewing everywhere.

xyz
Monday, July 28, 2003

If you *want* to hire someone then consider relaxing some requirement, and hire for ability more than prior knowledge. For example you might find an OO/cryptography expert who know Windows but not Linux, and be willing to give them Linux support for a few weeks; or find a Linux expert who is capable of bit-twiddling but has no previous programming experience as a cryptographer.

When I was hiring I used to feel that, not matter how much a candidate knew, they couldn't be expected to know *our* code, or even our industry, and that they would be 'junior' for at least their first 3 months with us and maybe 6 months ... from your description, you *might* find it difficult to keep a candidate long-term even if you succeed in hiring one that you like.

Another idea might be to consider the people who do relatively well (not absolutely well) on the tests. I've taken three tests so far for different companies ... my test results were in my own estimation imperfect on every test, yet to my surprise all three companies have wanted to interview me further.

You might relax your test standards while still protecting yourself against a 'bad hire' by having a 'probabation' period after hiring.

As for a 2-hour test, perhaps the 'real' difference between a good and a bad candidate is not that a good candidate can do it two hours without reference materials, but that a bad candidate can't do it any finite time even with reference materials.

Christopher Wells
Monday, July 28, 2003

My roommate once told me about a question he asked all the new hires at his company during the interview loop:

"How can you tell whether a particular problem is solvable by a computer?"

Well, that was a vague question.  I hemmed and hawed about halting problems and O(n) complexities and problems with subjectively-evaluated solutions, like AI.  All good answers -- shows how I think, right?  Well, I could tell my roommate had a particular answer in mind, that I just wasn't "getting it".  "This is such a simple question, I don't see why people take so long on it ..."

After a while, I finally said.  "Okay, I give up.  What's the magic word you're looking for?"

"Algorithm.  All you needed to say is that if you can find an an algorithm for it, then it's solvable".

I was nonplussed.  "Well, the definition of 'algorithm' is 'a series of steps that solve a problem', right"?

"Right ..."

"So really the answer to your question is: 'if I can find a way to solve the problem, then it's solvable'.  How tautological an answer is that?  Moreover, an answer like that tells you nothing about problems that can't be solved".

Tautological or not, I have the impression that I failed his question anyways.

That's not the only time I've had to answer a "guess my magic word" interview question, either.

Alyosha`
Monday, July 28, 2003

So what I am gathering from this forum is -

1. testing for knowledge -not that relevant?

2. Go for people with good previous work experience, even
  if they don't do that well?

People_wanted
Monday, July 28, 2003

I think you're wrong in your thinking.  It's not that the topics themselves are difficult, but expecting someone with 2-3 years of experience to have worked with those topics is unrealistic.  The guy with 3 years of experience is most likely still an underling to a more senior developer, so it's hard for this person to know the ins and outs of C++ and systems development.  Personally I could only answer 1,3,5,8 - but I'm not a Linux/C++ developer...

GiorgioG
Monday, July 28, 2003

It is not quite clear what you expect to find.  What is the criteria for fitting your needs?  Do candidates have to be knowledgable in all the areas covered in the technical interview?

If your criteria for an acceptable candidate is someone who knows C++ AND Java AND XML/XSL AND OO design AND system programming AND database knowledge AND UNIX tools AND crypto, then I think you did quite well finding 3 people out of only 20.  OTOH,  you'd think they should all know some basic OO and system concepts.

mackinac
Monday, July 28, 2003

> Some questions asked were [...] Was I wrong?

Speaking for myself, I could answer questions 1 through 5, and 8; I don't know how well I could answer 6 and 7: I know the language well 'off the top of my head', but I invariably use the 'man' pages when I code a system call.

Christopher Wells
Monday, July 28, 2003

This is my first time interviewing, so if anybody can give me pointers on how to structure it,  would appreciate it

People_wanted
Monday, July 28, 2003

I don't expect tham to know everything,
for each topic, I assign a rough grade from A to F,
then average them out,  to get a rough idea where they stand, so long as there are not a whole lot of D's and F's, it
does not matter that much how much they get.

People_wanted
Monday, July 28, 2003

If your budget doesn't allow for a top notch senior developer - find a top notch developer with 3 years of experience who can solve problems and work with them to become that great senior developer.  "How Would You Move Mount Fuji?" is a good read. 

GiorgioG
Monday, July 28, 2003

Our organization can afford top notch developers,
of the other 3 we have, one of them is a major open source contributor, and lead dev for an open source C++ project, respected by his peers worldwide.He goes and talks at various dev. conferences on Linux.
The other 2 are just as good with 5+ years experience.
Our compensation is top drawer, compared with others in the industry, in the top 20%.
It's not lack of money preventing us from hiring.

People_wanted
Monday, July 28, 2003

I don't think it's unreasonable that a person with 5 years of experience in C++ to know the answers to your questions..

One of the things that I've seen techies do when interviewing is to let it become a contest of egos. I've seen guys that just love to ask questions about some irrelevant topic that they are experts on. Don't be that guy.

Make sure your technical questions are relevant and pertinent to what you are looking for and not just there to boost the ego of the person asking the questions. (The questions you posted seem perfectly appropriate.)

Mark Hoffman
Monday, July 28, 2003

" 1 getting a better offer, we could not persuade him."

"It's not lack of money preventing us from hiring."

Hmm...
So, Is there another reason you could not persuade the 1 developer you wanted ?

Just curious...
Monday, July 28, 2003

People---  I would ask your questions, but consider them the minority score (25%).  Look for what the person has done and check the references. [I found people will put references on their CV/resume, they have not worked with the person in 5 years.]  I also like to see code.  Just to see what type of work they do. I ask for samples, not coding in front of me.  I kind of look at is like the resume.  If given time you cannot produce a nice product, how can I expect good results on the job?

Go back to what is required by the person doing this job.  Then divide the tasks into "must know", "good to know", "not expected".  Be realistic with what you are willing to pay and the number of years experience you are seeking.  Three years C# experience is going to be difficult unless they work for Microsoft. If crypto is most important, Philip Zimmermann is going to score very high.  So high in fact, I probably would let him fail most of the other sections.  If  a well rounded person, who can be a "jack of all trades" is what you need, admit it and look for that.  They will score average on most things, better on some, worse on others. 

While defended strongly by people, such as HR (see Mike's rant above -- it's a hoot), tests are a questionable indicator of performance.  One that even testing companies indicate in their disclaimers when you use them.

BigRoy
Monday, July 28, 2003

You say you want Java, C++, and XSL, yet the questions you list all look to me like hard-core C++ questions. I'll bet you're generally getting Java and/or XML devs who have dabbled in C++, figuring that's a fluff requirement, and the "real" C++ coders aren't applying either because they don't know Java or figure it's not a C++ job.

You need to figure out what you want and word the listing appropriately - don't just have a laundry list, put it out there:
"Seeking a seriously experienced C++ coder who's worked in Java. Your job will be slinging pointers and templates, but you'll also expected to be able to read calling code to help debug..."
or
"We need someone who can code Java with his eyes closed, but can help out with C++ device drivers when there's a need for it..."

I think more companies should work on job descriptions that actually describe the job, instead of trying to present a punchlist of what they want in a candidate...

Philo

Philo
Monday, July 28, 2003

Your questions may be testing specific knowledge, when you really want to be testing general knowledge.

For example, odds are, you don't need someone who knows the specifics of the threading API for Linux, you just need someone who knows enough about threading that they can quickly adapt. Someone who knows why you must always grab mutexes in the same order will probably be good enough -- they'll adapt to penguin_threads or whatever wacky threading model Linux has.  So very general questions are probably the right thing here.

Same with auto_ptr -- lots of organizations don't make use of it because it doesn't do what they need, so lot's of programmers haven't been exposed to it although they understand the constructor-as-resource-allocator/destructor-as-resource-releaser  idiom, smart pointers and the like.

You probably do need someone who knows the specifics of cryptography, however, so it would make sense to ask for specifics there.

Gustavo W.
Monday, July 28, 2003

Have you browsed around the fogcreek web site and read some of Joel's writing about hiring developers?  Are you following his criteria of looking for people who are "smart and get thing done"?

I have 20+ years development experience and have used C++ for about the past 7 years.  I can understand your questions and have answers for most of them.  Some would require a little thinking about and for things like system calls I generally take a look in the man pages.  Even with several years experience, I have found that there are features of C++ that I have never used and so don't have a detailed knowledge of.  So, your C++ questions don't look very difficult as long as you don't expect a guru level response to all of them.  If you expect candidates to answer similar questions on Java and database, etc., then you may be expecting too much.

As I think more about this, it seems to me that maybe you aren't really doing all that badly with finding people.  Although you have interviewed a lot of people who can't answer some simple technical questions, you still managed to find 3 people out of 20 who were adequate for your needs. That is better than 10% success rate.  It is a bit surprising that you couldn't hire any of those, but with all the out of work developers out there it shouldn't be too long before you find people to hire.

As an aside, please tell me where you are located.  As of the end of last month I became one of those many out of work developers.  So far I have no response to the resumes sent out.  There are a lot of jobs around here - if you have a top secret security clearance.  I need to find out where the jobs are.  Especially where they are so plentiful that people can pass up one offer for another.

mackinac
Monday, July 28, 2003

If you want to drum up interest from more qualified readers and help solve your hiring problem, why not confess where this job is located?

Steven E. Harris
Monday, July 28, 2003

Let me get this straight you are looking for people who have the following technical skills/knowledge:

* Linux OS programming experience
* Unix OS programming experience
* C++ programming experience
* Java  programming experience
* System programming experience (i.e. writing device drivers)
* Business database programming experience
* XML/XSL knowledge
* Crytpography knowledge/experience
* Prior OO design experience


Wow!  You Open Source guys really are hard core techies.

Do you really believe you would have no problem finding lots of developers with only 3-5 years of work experience under their belts who could meet all of those requirements?

One Programmer's Opinion
Monday, July 28, 2003

I agree with "People_wanted".  I think that it is surprising and disappointing that people who claim to be experienced C++ programmers often fail to answer basic questions about the language.  I would hope that *anyone* claiming to be a programmer would know the difference between a stack and a heap.  But I realize that many people now working as programmers really don't know the difference, and so I'm not surprised that "People_wanted" is having trouble finding qualified people.

old coot
Monday, July 28, 2003

There is a problem of unrealistic expectations here.  This company is interviewing only 1 or 2 people a week.  After interviewing 20 people they have already found 3 who meet their rather long list of qualifications.  Perhaps they should advertize a bit more and get their interview rate up, but it shouldn't take them very long at all to get all the qualified people they need.

Z
Monday, July 28, 2003

He has the right idea - I wouldn't hire anyone who couldn't answer those questions.

And from my experience if you're getting 1 in 20, that's pretty good. Unfortunately it seems most developers interviewing today couldn't find their arse with both hands and a map.

punter
Monday, July 28, 2003

I agree with Philo.  You say the test is about OO design but all of the examples you listed were more along the lines of peculiarities of C++ rather than theoretical OO.

Also, it is a good thing your company doesn't also have a "rigerous" spelling test in the interview process.

Mister Fancypants
Monday, July 28, 2003

"I agree with "People_wanted".  I think that it is surprising and disappointing that people who claim to be experienced C++ programmers often fail to answer basic questions about the language. "

I will agree that most of his questions are pretty basic for a C++ programmer.  However,  I'd never hold it against someone if they didn't know much about auto_ptr as long as they seemed to grasp the idea of it if I explained it to them. 

On the other hand, I've seen a lot of C++ interviewers(including here at the company I work for) ask questions that I think are totally whack, eg: what does the auto keyword do, what does the mutable keyword do, explain why I'd use the export keyword in such and such a case, etc.

Sure, if they are interviewing for the position of C++ compiler programmer, by all means ask them these questions... If they aren't, then don't.  These are the dirty corners of C++ that are virtually never used, and in many cases (export) are barely even supported by compilers. 

I get the feeling these interviewers are more interested in showing how smart [they think] they are rather than trying to learn if the people being interviewed can do the job. 

Mister Fancypants
Monday, July 28, 2003

>"Our compensation is top drawer, compared with others in the industry, in the top 20%.
It's not lack of money preventing us from hiring."

You are looking for a top 2% developer, but only willing to pay in the top 20%? The developers who can pass your tests are going to have multiple offers.

Those already working for your company may have taken the job for personal reasons such as being able to work close to their home or their spouse's workplace or something.  Their acceptance of their compensation levels is not necessarily an indicator of the salary that most developers of that caliber will accept.

T. Norman
Monday, July 28, 2003

There seems to be some fuzzy reading going on here.  People_... only said the test included OO design stuff, not that it was exclusively theoretical design stuff.  The questions listed were only a few examples of a two hour interview.  Let's not be too picky.

To provide my own opinion about what developers with 2-3 years experience should know about People_...'s questions:

1. Difference between stack and heap
    -> Basic CS stuff.  Everyone should know.

2. what is auto -ptr, when to use it in c++?
    -> This is standard template library stuff.  It's in Stroustrop's third edition, but not in the second back when I learned C++.  If I hadn't used it on a project, I might not have heard of it.

3. Why should you not throw and exception in a destructor?
    -> Seems like basic C++ that most everyone should know.

4. should you call a virtual method in a constructor?
    -> More basic C++ stuff, but I had to think about it a couple of minutes because I haven't had to worry about this problem for a while.

5. difference between pass by reference and pass by value
    -> More basic CS stuff.

6. questions on pre-emption,threads, pipes etc...
    -> Sounds like basic CS.  Are you asking theoretical questions or ones dependent on a specific OS?

7. forking of processes and process id's
  -> This is rather OS dependent and a lot of developers might not really have need to spawn subprocesses.

8. difference between overloading and overriding
    -> Basic C++ stuff, but this is the sort of feature that I use without thinking being too concerned about the labels.  I'd probably miss this question even though the concepts are quite clear.

The problem with "Basic CS stuff" is that people who get in to software development without a CS degree (or self-taught for us physics majors) might never have had any need to learn the concepts.

Another issue to consider is how to people learn C++?  It didn't exist when I started programming.  I learned it by reading and on the job experience.  I try to get a basic understanding of the entire language, but it is easy to miss details if you don't use them.

mackinac
Monday, July 28, 2003

people_wanted, someone who has successfully developed at least one and preferably a few systems will be able to do or learn whatever you're doing. You can confirm their depth by discussing development with them.

Expecting specific answers to specific questions is a throwback to university.


Monday, July 28, 2003

I am a programmer, and have been for about 20 years now. I know C, C++, Pascal, Fortran, Cobol, some Java and so on. I haven't done any C++ in about 10 years, so I couldn't answer to many of the questions posted here.

But, during the last two years I've learned a completely new programming language (Hal - Hansa Application Language), and become very good in it. So good, that currently I am the lead developer for a system that is used in several countries.

So what's my point? It's that these types of questions only tell what a person knows now, not what he or she is capable of. I wouldn't hesitate to take on a new programming language any day, because I believe that the basics of programming remain roughly the same no matter what language. Also, I wouldn't hesitate to take on cryptography even though I've never really done it, because I believe I could learn it and become good at it.

It's all in the mind, and the mind is a great thing if used properly. Not all programmers in a company should be "artists", but it doesn't hurt to have different levels of experience, knowlegde and common sense.

And everything is documented, be in man pages or the internet or some books, or even a discussion forum. A person who knows how to find things out is in my opinion worth the same if not more than one who knows things already.

But all that of course depends on what you are looking for. All interviews should be designed with the goal in mind: what kind of a person do we want, and what sort of things do we need to ask in order to find out if the interwieved person is like that?

Antti Kurenniemi
Tuesday, July 29, 2003

> We are a small company looking to expand with a lot of projects in
> hand. Unfortunately the problem has been getting the right people.

Please review javadesk.com. We may outsource your Java-based projects.

Evgeny /Javadesk/
Tuesday, July 29, 2003

People_wanted,

I know the answer to your questions with no problem. I am a seasoned C++ developer, Unix/Linux (although I do only Windows nowdays), CS grad, OOP designer etc. However, I never worked with Java (I could obviously pick it in a few days, as I did with C#) and more importantly I don't know much of crypto stuff neither (although I did get the highest mark at my crypto course in school, but I haven't done  anything with it since nor I am too interested in it). Would I qualify for your job?

DP
Tuesday, July 29, 2003

*  Recent Topics

*  Fog Creek Home