I was recently contacted by a head hunter for a job I thought was worth investigating. I wanted to talk to the company directly, but the head hunter made it clear that Step 1 was always a web-based programming aptitude test, no matter who you were. 20 questions. 18 correct was considered passing. They would only talk to candidates who got 19 or 20 correct.
So I took the test and got 20 correct (as I imagine many people here would do as well). I thought it was very easy. The headhunter later told me that in 9 months, she had sent 52 people to the test, only 2 of us got 19, and I was the only one who got 20.
I'm not really sure what this means. That there are a lot of posers out there? That she wasn't very good at screening talent? That the companies seeking the best talent gladly pay $100 50 times to save their time?
I guess my biggest feeling is one of disappointment. It's just not that hard to become a good enough programmer to get 20 right every time. All it takes is study, passion, a lot of dedication, and a lot of hard work. I wish more people would do that. There aren't enough of us.
Wish I could find the source, but the math is simple. If you're great (at programming and marketing yourself), you pick where you want to work and apply once (or more likely, are convinced to leave one good job for another). If you're good, you pick a few places and apply a handful of times. If you're bad, you end up applying constantly. The average unscreened resume is the average person in the job pool, who is well below the average programmer.
I decided this seems like something one should be able to calculate with conditional probability.
Let P(A) be the probability that someone is a pretty good programmer. (I said top ten percent, so .1).
P(A') is 1-P(A) = .9
Let P(E) be the probability someone is employed. I guessed and put that at 1-P(E'), guessing programmer unemployment rate at 6%.
P(E) = .94
P(E') = .06
Here is a more wild assumption. Let's say that given that someone is a top 10% programmer, there is a 98% probability that he or she is employed.
P(E | A) = .98, and conversely, a .02 probability they aren not.
With these assumptions in place, I ran the calculations. I was intending to type them up, but it would be difficult with just a text area.
Here is the punch line (with those symbols, and assuming I did this correctly, which I did it on the bus on the way home, so probably not)...
Given that someone is unemployed, there is a .033 probability that they are in the top 10% and a .96 probability that they are somewhere among the rest. (The 'rest' includes a lot of pretty decent programmers though!)
In other words, 3.3% of applicants would be top 10% candidates, and that's assuming equal probability of them sending in an application in the first place.
I might rework it with different assumptions. Anyway, you said the math was pretty simple. If someone wants to see how I screwed it up I can type it up into latex and upload it somewhere.
> I wish more people would do that. There aren't enough of us.
Ehh... I'm pretty glad we're in short supply. Makes it very easy to just walk into (or out of) a job even in a poor economy and raises my wages to levels I feel guilty for charging :-|
"It's just not that hard to become a good enough programmer to get 20 right every time. All it takes is study, passion, a lot of dedication, and a lot of hard work."
As someone working for a big company with a lot of passion, the four items that you listed is hard. Dedication and hard work requires significant amount of time commitment. My work for 8 hours typically involves so much paper work, process and discussion that only 20% of that time is really spent in programming. Between family & kids and a 8 hour job, I try to spend 1 hour each day doing new stuff, but with that time, it takes a long time to be really good.
The entire test was about a fictitious language, I forget what they named it. Every question built upon the previous.
The first question was something like, "These are the definitions of variables: A string is... An integer is... A date is..." Then the question would be something like, "What can '15232' be? a. an integer b. a string c. a date d. any of the above e. a and c".
Another question might be "Here are the rules of precedence... What is (1 + 2) * (3 + 4) / 3 a. 4 b. 2 c. 7 d. 12 e. Can't tell for sure."
It got more and more complex, all in their own pseudocode. They covered branching, iteration, arithmetic operations, Boolean algebra, etc. Things most people here have seen a million times. Kinda like SATs for programmers.
That's sorta cool idea though; much better than questions like "do you know this obscure feature X of some programming language Y, or what does some class EJBServletContainerFoobar do?"
The US military has a standardized test called the DLAB (Defense Language Aptitude Battery) that tests not your ability to use a particular language, but to learn a new, fictitious language - much like this programming test.
success in the first stage of an introductory programming course is predictable, by noting consistency in use of the mental models which students apply to a basic programming problem even before they have had any contact with programming notation
definitely better than having to relearn java or C++ every time they ask me to take one of these tests... which tests for your knowledge of a particular language rather than programming concepts in general. Way cool test.
You might like the "labs" (exercises) that come with J (http://jsoftware.com/). It's a real language, albeit a very unconventional one - it's a descendant of APL.
In 1996-7 I worked for a firm that had retained one of the two best headhunters in the D.C. area and our experience was that 1/2 of the people they referred to us couldn't program their way out of a paper bag.
My boss mentioned some time after I was hired from an ad in the paper that I was the only one who passed his programming test. Which was sufficiently generic and easy that I couldn't rememer it in detail, e.g. it was something like reverse a doubly linked list and find errors in some C++ code on the white board, none of them hard.
This is why I think that Codility (covered on HN earlier here: http://news.ycombinator.com/item?id=1039140 ) is such an amazing idea. You can administer a web-based company-appropriate FizzBuzz for rounding error next to the cost of doing an interview or (worse) hiring someone who can't do FizzBuzz.
Incidentally, if you think Java programmers who can't figure out how to count the number of occurrences of the letter "a" in the string "Java java java" are bad, you should see how many interpretations of "advanced oral and written Japanese proficiency" we've had. Oh boy. Let's hypothetically say you're a recruiting company and you get a candidate as far as our door without disclosing that their exposure to Japanese is, I kid you not, "I loved watching Kenshin in high school" -- we will not be doing business with you again. Hopefully ever.
See, honestly I think something like Codility misses the point of these coding exercises.
For me, a coding exercise is more than just a way for me to see if the person can produce something that works. It's also a kick-off point for other questions. Often there are things the person will do along the way to a solution that reveals some lack of knowledge, so I keep my eyes peeled as they talk through their solution for these potential gaps.
One of the biggest problems now is that you have people who can even pull of FizzBuzz but don't know much of anything beyond how to get something like that working. They don't understand data structures, algorithms, or how a computer really works.
A simple coding exercise like FizzBuzz can reveal these gaps too, even if they get the problem right. The trick is to use their implementation as a way to probe their thinking.
For example, if they are just using a "sort" function, like in Ruby, ask them how it contributes to the runtime of their algorithm. Or, if they are loading a file into RAM instead of just iterating over reads on a file handle, ask them about it. Or, if they are iterating over the keys of a hash (h.keys), ask them how they think that works. (This recently revealed to me the candidate did not understand that a hash function was irreversible.)
None of the major gaps in knowledge I've found that separate mediocre from good candidates were found by just looking at their code and giving them a thumbs up or thumbs down. That's good for separating the mediocre from the morons. To find out if they're good, you have to probe.
To find out if they're great, well, that's another story entirely that is far afield from FizzBuzz :)
"That's good for separating the mediocre from the morons."
That's a lot, if you ask me - the morons usually make up the vast majority of candidates, so by filtering them out effortlessly at an early stage, you get to spend your time more effectively - identifying the great ones among those who at least hold a promise.
Yeah a good point, actually. I actually am just subconsciously responding to the implicit meme that seems to be spreading that if they get the coding question right, they win. I think for screening out morons, that is probably a good bar. But don't forget to do the next coding question in person, and drill into areas where they gloss over things that they may not understand.
That casts the old "I'll just pick up the language as I go... I'm a fast learner" in an interesting light. And I'd take it as a good counter argument to "smart and gets things done". (edit: maybe not "counter arguement since SAGTD is also important. Maybe "corollary" would be a better word choice?)
I've seen a few "will pick up [perl | c++ | java | whatever] as a I go" type guys. (Come to think of it... I've been that guy on occasion)
I've done that a few times myself - my first coding experiences with a number of languages were paid gigs [C, shell, Modula-2, Data General Nova assembly, PL/I].
We were once hiring programmers for a small consulting firm.
We asked the prospective employees if the knew how to program for Windows, VB, etc...
They almost all answered 'yes', when we asked them to write a sample program the couldn't, so we asked them to describe how the would program something for windows...
Their answer: 'You put the CD in the drive and run the setup.exe'
They all thought that programming meant 'installing' software... Needless to say we ended up not hiring anyone.
Whenever the FizzBuzz topic pops up on Hacker News in the back of my mind there is a nagging anxiety: am I one of these programmers that can't program? There is always some comment in the ensuing discussion that makes me question my own competence.
...am I one of these programmers that can't program?
Wrong question.
Right question: "Where do I stand and what to I have to do to get better?"
Clue: The answer has nothing to do with comparing yourself to anyone else and everything to do with what your building vs. what you need to be building.
How you describe yourself is one aspect of Dunning-Kruger: highly competent people often do not think that what they do is anything special, even though in reality, their skills are very rare and highly sought-after.
Yes, but the inverse is not true. There are plenty of incompetent people who probably also think that they're incompetent. Thinking you're incompetent does not imply that you are competent.
Ditto, and for me it's a more general thing. (In this case, I tested myself quickly on Patio's suggested question from somewhere in this thread. I just had to open a terminal and see if I could quickly do it. Stupid.)
I always feel like a fake - not about programming in particular, about whatever I'm doing and supposed to know about. I like to tell myself that I'm being 'Socratic' and worrying about what I know I don't know.
You should read this recent submission if you haven't yet:
http://news.ycombinator.com/item?id=1121775
[No One Knows What the F* They're Doing (or "The 3 Types of Knowledge") (jangosteve.com)]
It's very easy for someone to misinterpret 3. as "I could tell you if I think you can ever code, so I would amend it to "I could tell you if I think you can code now".
"Can't code now" could become "Is a great coder" later.
Everyone is at their own place on their path. We've all traveled that path. I look at some of my code as recently as a year or two ago and I'm embarrassed.
Nobody here should get discouraged. You should just get better.
ASmartBear covers this topic from a related angle and it's well worth a read. Esp if, like me, you frequently have to battle the "I'm not good enough" demon.
I've done several interviews where I was asked to code and I really don't think it's easy, because you're under intense pressure of doing your task right the first time and under a very tight time frame. Sometimes even with your interviewer sitting next to you.
The whole point is to use a task so trivial that even moderate amounts of stress won't prevent your fingers from taking over. That's why people who criticize "FizzBuzz" as not testing the true requirements for a job are simply wrong. All you want is some real code that produces a correct answer.
My programmers wrote FizzBuzz in under 60 seconds. If you can't write FizzBuzz in 15 minutes in an interview then you are either unable to program, or any level of stress will prevent you from doing your job. Either way, you couldn't work for me. I don't routinely stress my programmers, but stress is a matter of observable fact.
Yeah, I think it's a good test for that reason. I personally would probably do badly on a complicated programming test--- I tend to program best when not in a room with other people, not being put on the spot, allowed to think while walking around outside and then return to a burst of coding, etc. But I could write a few lines of obvious code in pretty much any situation; there's a pretty big difference between peak performance and absolute minimal competence.
It's hard to make analogies to other fields, but it feels sort of like a writing job not asking you to write an insightful philosophy essay on the spot (legitimately difficult), but just asking you to write a paragraph of grammatical English (anyone even remotely fit for the job should be able to do it).
I don't think it's just stress. Few programmers actually write correct code on a whiteboard / pen and paper so expecting them to accurately program outside of their preferred environment can be vary hard for some people. It's something like asking an author to spell out an essay, plenty of people can do it but it's a separate skill from being a good writer.
PS: Try composing your response by saying one letter out loud at a time without typing it before you type it.
Your comment about generating my response by saying one letter at a time before typing it made me stop and think - thank you for the insight. However, I still don't think it's a sufficient response.
I agree completely that writing code on a whiteboard is utterly alien to some people, and is so completely divorced from their usual way of working that it's unlikely to give a true picture of how they will code in the real job.
But we're not asking for an insight into how they will code in the real job. We're asking if they can write something like, say, FizzBuzz. More, we're not even asking them to be syntactically flawless and optimally efficient. We're asking if there's any chance that they'll produce working code at all.
I agree that there is to some extent just a disagreement about where to draw the line, and how detailed/accurate the solution has to be, but my experience is broadly in line with the worst claims - 95% of people who approach me for a job in programming can't write code at all.
For the remainder I'll ask how they usually write code, and then collorate with them on something more complex to see how they work.
"But we're not asking for an insight into how they will code in the real job. We're asking if they can write something like, say, FizzBuzz. More, we're not even asking them to be syntactically flawless and optimally efficient. We're asking if there's any chance that they'll produce working code at all."
In practice, few people are asking questions that are as easy as FizzBuzz. I've been through a lot of programming interviews, and I have yet to encounter a problem of that simplicity as an interview exercise. Instead, most companies seem far more interested in laying down the gauntlet, so that they can convince everyone (including themselves) that they only hire "the best".
It sounds like you're trying to keep your programming interviews sane, and that's great. But the problems happen when the other 98% of programmers convince themselves that whiteboarding exercises can be used to successfully discriminate the best of the best from the merely good. One needs only look at the other comments on this page to see that there's a huge and disjoint set of skills that define "competency" to different people.
Few programmers actually write correct code on a whiteboard...
But your code doesn't have to be correct. Not as you first write it down, anyway.
The advantage of writing the stupid and broken version of FizzBuzz first is that it gives you ample scope to demonstrate your ability to critique your own bad ideas out loud, write tests and conduct them out loud, eschew premature optimization, analyze the performance of something that's way too slow, and invent alternative approaches only after you've produced the stupidest thing that can possibly work. Not to mention your ability to stand in front of a whiteboard and talk while staying on task and not wasting time, which is a core skill in engineering, even if what you're doing on that whiteboard in real life is rarely actually writing code.
That would be a good solution for me, too. For example: It's years ago since I used Visual C on Windows, but I use C++Builder daily. I can whip up a good looking, surprise-free, standards-adhering, well behaved GUI in minutes and flesh it out to a fully functional application without hesitation in the hours after. If you'd put me behind Visual C I'd probably be looking for minutes to find the Dialog Editor (errr.... is that Windows Forms nowadays? Hmmm, I should give .Net a look someday).
Yes. But it doesn't have to be 100% correct. It just has to be plausible.
You don't ask an author to to write a perfect essay off the cuff. But certainly any author worth having could generate a brief outline for a story (or whatever type of writing you want them to do)
it's not that different. good programmers sketch the algorithm or data model or architecture on paper first----or however they figure out what they're doing before sitting in front of a computer.
writing a program by hand is not cruel and unusual, it's quite realistic. it's like a musician site reading, not an author writing an essay by spelling letters. (site reading is a skill that comes with being a musician. i can site read music that i would have had to learn to play in the past. i always prefer to read challenging music, but i can play comparatively easy music with little problem.)
Given that most people would be answering this in some form of imperative language, I'd say that any solution which includes "for", some "if"s or a "switch", and a few judiciously placed modulus operations would be sufficient. Actually, if I were giving the test, I'd probably ask for pseudocode anyway.
I'm sympathetic to the problem. What with increasing reliance on IDE assistance (and I'm guilty of this) I can easily imaging feeling overwhelmed with just a sheet of white in front of me.
I've tried to do thing mostly verbally, testing their underlying understanding of CS concepts, and hoping this is a decent proxy for development ability.
If you design the questions carefully, the way the applicant answers tells as much as the fact that it's right or wrong.
For example, in a C tech interview I'd ask what you need to if you want to pass variables into a function, and have the function change those values for you. There are a range of correct answers from "put an ampersand in front of the variable" to "call by reference rather than call by value", and these shows different levels of understanding.
I also liked to ask about what the difference between single quotes and double quotes were. A simple correct answer is "one's a character, the other is a string", but somebody who really gets C would tell me that one will lead to the generation of an integer value, while the other results in a pointer.
So it's not just right/wrong, it's shades in between. With some careful design, you can ask questions that tell a lot about how well somebody understand programming.
But occasionally I'd bump into somebody who just couldn't get any of my questions at all, and I'd feel really bad as I tried to think of something simple enough that they wouldn't feel completely awful.
I agree that whiteboard programming can be alien, but certainly a programmer needs to be able to communicate an algorithm in some way other than pure code.
I would think an interviewer should accept someone saying "I'm having trouble on the board, but what I want to do is loop through the numbers, and for each number, mod it with 15, 3 and 5 and then print FizzBizz, Fizz or Bizz." Or, have them draw a diagram, whatever.
Of course, this would need to be supplemented with a coding sample or anything that shows that the person can code when they are in a comfortable environment.
Social stress is different from deadlines. How you conduct the interview is important in this respect, but still FizzBuzz is the correct approach. It would help though if the interviewer made it clear that it's supposed to be a warming-up test, and there is no catch to be looked for.
Wow, so it's been so long since I've had a 'programming' test I had to go look up the FizzBuzz test. A first year compsci student should be able to whip up a solution in minutes. I can't believe that people try for programming jobs and fail FizzBuzz.
You know... I once had an experience like this when asked to do a trivial task (on paper) as part of a screening process. The interviewer didn't want pseudo-code but said any language would be fine. They wanted to know how I'd store a list of numbers and return the max.
I asked about the data structure several times as I thought they were looking for a generalizable solution that could be used with arbitrary objects and wanted to know if I had to define a separate class to implement a linked list or if I could just use a vector. This made the interviewer increasingly flustered because from his perspective the data storage architecture didn't matter and the answer was simple. I later found out he simply wanted to track the largest item on insert with a separate int and return that on max() - this was their concept of a good programmer, despite the fact it would be unnecessary and inefficient in any ordered list).
After a few minutes he concluded I couldn't program and moved to end the interview. I decided that if communication problems were this bad (probably my fault as well), this wouldn't be a good place to work. And I was also sort of shocked he hadn't bothered to vet any of the public coding work I'd done. Who brings people into interviews totally blind?
So while I might have been sympathetic to this article before, I wouldn't be so quick to blame the programmers. I was irritated this company had wasted my time, and probably vice versa. I don't know what the solution is for employers, but the solution for job-seekers is to require people to look at code you've produced before you meet them in person, and bring a laptop so you don't need to stare dumbly at paper while you try to remember the exact way to initialize a vector of pointers in C++. Either that or do a phone interview first to get this stuff out of the way.
Did you use a headhunter of placement place? They often cut all identifying info out of a resume before passing it onto the interviewer. That way they can't be cut out of the conversation when it comes time to talk about money.
This is a valid point. In my experience, the best interviewers are the ones that don't make me feel like I'm in an interview. It feels like I'm just chatting with a fellow hacker. This makes me much less anxious and stressed.
They're the ones who get the most realistic view of me as a programmer (for better or worse).
It's a common mis-conception, but we're not talking about the stage where I'm trying to get an accurate picture of what you'll be like in the job. It's still the very first stage of the interview process, and we're not yet at the stage where we need to decide which of several demonstrably competent programmers we want. We're talking about weeding out people who simply can't write programs at all.
The realistic view of your ability comes later in a second, or even third interview. At that stage we will also talk about system/program/algorithm design skills, ability to work with others, willingness to travel, preferred working hours, and a whole bunch of other stuff to get an idea of how you add value to the company, and how well you will fit with the existing staff and ethos.
But if you can't write something as simple as FizzBuzz in under 15 minutes in a slightly strange environment, you probably won't be able to program well enough here in this company to bother with the other questions. Working here requires more than just producing code in your ideal environment without working with others. I suspect we're not alone in that.
EDIT: I've corrected a few typos. To the person that down-voted me - thanks - you made me read it again and realise that it wasn't as clear as it might've been. I was on my way to a meeting and wrote it in a hurry. Although still not perfect, I think it's better now.
To others who might choose to down-vote it, I'd appreciate any insights as to why. Sometimes I hold unusual, eccentric, or unpopular opinions, and I'm willing to bear the consequences of that, but I'd really appreciate the opportunity to learn what you disagree with. Please, share your thoughts, don't just down-vote without giving people a chance to consider what you think.
Google lost a lot of good people by dragging out their interview process. It's not all or nothing, but if I disliked the first interview I am unlikely to be interested in doing a second one. In the back of my mind I can't help but wonder if I want to work for people desperate enough to jump though these hoops.
You make it sound like it's a big deal to write 10 lines of code before being interviewed. I agree that some candidates, possibly including me, would be put off by being asked to solve a problem requiring 10s of hours before interview, but we're not talking about that. We're talking about getting someone to demonstrate 10 lines of working code.
And if a potential candidate can't be bothered to do that, how bothered will they be to make small, uninteresting changes to working code? How interested will they be to fix a corner case they don't care about? How careful will they be about details when releasing code?
Not being willing to "jump through hoops" is an indication of fit, as much as many other things. My job is to make the hoops as big as possible, and as easy to jump through as I can make them. If people can't be bothered, then I guess I won't employ them.
Yes, it biases the selection process, but it's a huge save not interviewing people who can't, or won't, produce FizzBuzz (or similar).
The issue is showing up for a second, third, or fifth day of interviews. If you want to know something about my technical competence fine, figure that out over the phone or use a webbapp.
You obviously haven't interviewed at google. I had 6 technical interviews, 7 if you count the phone screening. From the day their recruiter called me to the day they called and said they weren't interested spanned about 3-4 months.
I can't remember the link, but I read about one job where they gave candidates an ssh login and asked them to write a simple, running, perl script. Something like 10% actually finished the assignment, and they hired one of them, but avoided screwing around with LOTS of screening phone calls, enthusiastic people who "were familiar" with Perl, Windows programmers who didn't really know what ssh was, etc.
I remember interviewing for the first time, where I'd asked lots of questions, and then called back potential candidates for a proper coding test if they passed the talking test. I know better now, but anyway, it was scary! The amount of people who can talk about the inner workings of compilers, internet protocols etc, and then can't do the fizzbuzz test is shocking!
The next time, I started with fizzbuzz and made life a lot better ;)
I think this article does a good job covering the second best way of selecting programmers - the technical job interview.
The best "interviews" aren't really interviews, though, they're recruiting sessions. A really strong, experienced software development manager who plans to employ programmers should have 1) a strong background in programming, and 2) a strong network of programmers who would jump at the chance to work with him again.
You already know your potential hires are good programmers, because you've worked with them before - in school, on side projects, at work, in various collaborations. You don't need to ask them to print all the nodes in a binary tree, and if you did, well, you'd probably lose the candidate.
I think Joel (on software) put it this way (I'm paraphrasing): if you're a small independent film maker, and Uma Thurman expresses an interest in your movie, you don't ask her to audition, you ask what you can do to get her on board. I think 37Signals is another example of a company that mainly recruits people they already know through programming projects (esp contributions to rails).
I won't say the technical interview is evidence of a "broken" recruiting system the way some people do, because let's face it, we don't all have the recruiting mystique to wave the magic wand and get top people to join us, and our networks just aren't big enough. So we have to interview relative unknowns.
But I will readily admit that this shows that I'm not doing it the best way. This goes both ways. If I'm in an interview and people start asking me how to implement a singleton, I start to wonder if I'm going about my job search the wrong way. If I have to ask the candidate this, then maybe I'm going about the recruitment in the wrong way as well.
But if that's your reality, then yeah, make sure you screen well.
Wow, I didn't know that. Brad Pitt actually has to audition for roles? Jeez. Well, that's the problem with analogies.
But I'm not really a fundamentalist about the code test. I've gone through many, many, many of them. I just had a little epiphany the last time I was in one. I was thinking, "why don't these guys already know that I'm a good programmer? Why don't they already know good programmers?"
It made me realize as a programmer and as a manager that I wasn't really building my career the right way. I decided that I wanted to build a network and a reputation that went both ways - essentially, I wanted to have a large number of people out there, in different organizations, who are both confident in my tech skills and my value as a manager. I'll step outside that network constantly, and when I do, I'll certainly validate programming ability (and I'd expect people to do the same for me).
But every time I do this, it's a reminder that I failed on Plan A, which is the best way to do things.
"I decided that I wanted to build a network and a reputation that went both ways - essentially, I wanted to have a large number of people out there, in different organizations, who are both confident in my tech skills and my value as a manager."
So far, it hasn't been much more than doing a decent job in the different positions I've held, and working in a few different industry sectors. I've worked for a large silicon valley company, a couple of public sector organizations, and a couple of small startups, and I've taken on management and dev roles. I keep up my contacts.
When the startup I was working for folded, I got a bunch of interviews (lots of tech questions), but my best offer came from a former coworker who didn't bother with a tech test because he'd already worked with me.
While I wish I could say I'd done more (presented at conferences, made big contributions to open source, etc...), but in some ways this is a better story, because people start to think that if they need to be a total rock star. You really don't have to, just keep up your contacts and make sure you have a reputation for good work.
Top actors rarely audition, even for top roles. In a case like that, the process more closely resembles an interview. It's really more meeting to discuss the concepts of the role.
Part of the problem here is that not all interviewers can recognize a candidates learning aptitude.
Typically the interviewer is so enamored with his own coding style that if the candidate can't mimic his style the candidate is not acceptable.
If the candidate does not communicate in the interviewers style the candidate is not acceptable and so on.
As well, most programming projects are not about such simple problem solving tasks as printing out the contents of a text file backwards. Of course most people could figure this out and as my experience has been this is when most people think about such issues, when they are required to apply it to a larger problem at hand.
First, as someone who has done a fair number of technical interviews, there are a lot of posers. I have had people claim to be experienced that had no clue how to do even very basic programming, just as described in this article. Even more common, I have had people that grossly exaggerate their skills and experience. One that stuck with me was someone who was a "senior DBA" but when I probed further it turned out he mostly ran stored procedures written by consultants and didn't really have the knowledge right then to do much more than that.
I would also like to point out that if you are up front about your lack of skills and applying for an entry level post, it can work out. I had one interviewee who had no programming skills, but was ready and willing to learn. She was up front about it, convinced the whole team of her sincerity, and willing to start for less than most people with a CS degree. By the end of her 3 month probationary period, she was competent. Still junior and with a lot to learn of course, but quite competent and a good hire overall.
Just to state the obvious but all fizzbuzz really tests for is whether someone is familiar with the modulo operator or not. Arguably any good programmer should know about it, but I think it's not a very comprehensive check on the knowledge of any one particular individual. Suppose for instance, that the only bit of code someone has ever written was something which required them to get very familiar with the modulo operator, but virtually nothing else. They could pass fizzbuzz, but not much else.
Note: The author of the 'non-programming programmer post' saw took my comment down when I had posted it there (which is why I posted it here). Guess there's no knocking of the fizzbuzz test allowed... :p
The first interview's purpose - whether in person, over the phone, or via e-mail - should be to weed out the obvious idiots.
If you don't know what the modulo operator is, door is right this way, operate it by placing your hand around the knob, turning clockwise, and pushing forward.
I've been programming for 3-4 years now, have memorized tons of functions and operators, across a variety of languages, and have never once needed to use the modulo operator for anything, so I personally don't place it at the top of list of priorities for what someone should know. As an employer and interviewer I'm more concerned the person has a grasp on general programming issues and the ability to problem solve than testing their knowledge of narrow subjects.
The post mentions a tool for live coding over the web (http://i.seemikecode.com) that looks simple but good. Etherpad (http://www.etherpad.com) is also real good for those and has some other features like line numbers.
Bespin (http://bespin.mozilllabs.com) is even better for this. It's slightly less realtime than EtherPad, however it is made for collaborative programming. It has syntax highlighting and a bunch of other features for coding.
Some of this may be due to selection bias. The good programmers find jobs quickly, whereas the non-programmers go to interview after interview, until they finally find a company dumb enough to hire them.
It makes sense to me. Programmers who can program - and don't want to stop working as employees - give up programming to pursue higher paying careers, such as project management. Actually, many "programmers" nowadays can get by simply copying and pasting code from the Internet. If you ask them how things work, they don't know. That's my experience.
Unfortunately, it doesn't necessarily end upon hiring either. Usually, the "implementationists" won't get the "abstractionist" point of view and will reject it immediately. And since they're more common, this can lead to bad ideas winning simply because everyone understands them.
I would think "bring a sample of your coding" might be better.
Non-programming might include:
Query language "programming".
Maintaining an existing "out of box application"
Maintenance programming.
End user support.
Little real coding is necessary. These are programming jobs but not really programming as I see it but.
Then there is the ability to learn new languages, processes,
dealing with the end user, is the code maintainable, code by specs?, juggling multiple projects.
I might start an interview by a sample code but that would not be my only criteria and if the person failed the "programming" test I would see it as a need for further discussion, not really a big red failure flag.
Example: If the reason the programmer doesn't know sql is because they spent their time putting out fires and supporting the end users, I would hire that person over the others any day.
I guess this is a good way to weed out people as well? Depending on the type of company you are, you probably don't want someone who has never written any stuff they can show you at all.
Most professional programmers probably view programming like any other job, rather than a passion like I expect most HN readers do. As a result, they leave programming behind when they leave the workplace.
Along with the CV I always used to ask for code samples. I wanted to see the code before I could develop any biases by talking to the person. I found it worked best to ask for "the last project/projects you had fun working on, or developed on your own time". If there are none...
In early stages, aspiring programmers need mentors. It would be interesting to know how many people who consider themselves good programmers, spend a reasonable amount of time mentoring other jr. programmers. Looking back in 7 years, there were only two good programmers I learnt a lot from. I took every chance to read their code and ask them to review my code and pair program with me once a while.
I became convinced recently with a new approach to this problem. If you want an artist, pick someone with decent interest to learn art and put a brush in his hand and he becomes a painter. It takes time but this is overall good for the programing community.
Here's hash maps A and B. Identify all values which are present in A but not in B.
In an interview at another company there might be a complication like "By the way, A and B are Google-sized" or "By the way, A and B are being written to." or "By the way, you have to do it in constant space.", but none of those things apply to our business.
Out of curiosity, why do you think questions like reverse a list without using reverse features of the language, or print out a file backwards a good judge of development skills?
I've been programming over 10 years and my opinion is that some of these questions aren't relevant for many development jobs.
Firstly, this is nothing to do about finding a good developer, it's to find out whether someone can program at all. I thought that would be clear from reading this thread, but perhaps it's now long enough that you haven't read it all, even though you have replied to a comment fairly deep in the hierarchy.
Secondly, this question didn't ask not to use the "reverse" features in the language. Indeed, the suggested solutions do use the reverse features.
Thirdly, perhaps this particular task is a good one, perhaps it isn't. Even so, it seems good enough to cull those who can't program at all. The point with these exercises isn't to get it right first time, it's to see if the candidate has a clue at all. If they get it right first time then you have a prospect. Treasure them and move to discuss design, style, ideas, and interesting stuff.
Finally, do you know any actively employed, productive developer who cannot write FizzBuzz in less than 10 minutes?
I don't even know if this is the answer. The reason being I have never needed to work with fractions or with the modulus operator (I'm 12 years out of school, which is the last time I need it). The only reason I know about it is because I was asked about a question involving the modulus character in an interview.
But I can tell you I have had many clients that have been happy with my work.
<code>
for (e in 1..100) {
if ((e % 3 == 0) && (e % 5 == 0)) { println "FizzBuzz" }
else if (e % 3 == 0) { println "Fizz"}
else if (e % 5 == 0) { println "Buzz"}
else println e
}
</code>
That's good enough to avoid having your resume deep-sixed. The next stage would depend on the position you're applying for, and whether it's algorithms, web, systems, maintenance, whatever. There are now several questions I could pursue, but that's enough to get you through the door, offered a beverage of your choice, and effectively courted.
By the way, not knowing what the modulus operator is no bar - all you have to do is write:
if is_divisible_by(n,d) ...
or something similar, and then say - "I'd need to look up the idiomatic way to do that in your language of choice."
But you are already way ahead of the game. FWIW, when we have been hiring (which we aren't now, by the way) the person who wrote something like that got the job. And yes, out of several tens of applications, only one could do it.
My C++ and Java are very rusty, and I don't think I could do it on the spot if I tried; let alone go into languages like python and ruby that are new to me. But if you gave me a week or two, I know I could get up to speed.
Tough questions without also building rapport is also bad.
I've had some interviews where they asked tough questions and I got a job offer but I declined because I felt there was no rapport or interest in my career.
It seems completely unfathomable to pose for a programming job. It's one of the rare professions where your skills and understanding can be tested very quickly, harshly and objectively.
It would probably be easier to lie your way (with mere knowledge and possibly some experience) into the position of a lawyer, an airline pilot, or a medical doctor than a programmer.
(At least I hope they don't test doctor applicants as like "Please operate this unfortunate cardiac surgery patient as an example so that we know you'll know how to do it.")
Good article but didn't he write the same article in 2007 (as he mentions but ..)? This re-posting is a subtle shill for stackoverflow careers. Not that there is anything wrong with that.
IMO if you're giving people 'gotcha' programming tests in an interview, then you don't know how to select candidates or interview in the first place.
You should be looking for competence by looking at their programming accomplishments, both in and out of work, as well as their ability and desire to teach themselves (probably the most important trait IMO).
I disagree. IMO asking someone whether they can program a loop that demonstrates the use of a mod operator is a 'gotcha' test because it proves nothing as far as their suitability as programmer and as an employee and is designed soley to weed people out based on one thing. Sure, maybe they are completely ignorant... or maybe they were just nervous that day or unfamiliar with a particular language's mod operator. Lots of programmers don't do well in high-pressure situations... which incidentally programming is not most of the time.
How about asking someone how they got interested in programming, or what sort of things they've created with their knowledge in and out of work, or what languages/technologies they've taught themselves in their free time, how they go about fixing puzzling software problems, or what they consider to be good vs. bad programming practices, or even what their hobbies are? - those kinds of things that will tell you IMO whether they are smart, passionate, and will make a good employee.
> unfamiliar with a particular language's mod operator
Just anecdotally, over some hundred interviews, I've never ever ran into a candidate who failed one of these FizzBuzz style loop writing tests because the didn't know the mod operator, or were nervous enough not to remember it.
I see a lot of candidates that can't remember what params some standard system call is supposed to take and I always tell them I'm happy as long as they define what they think it looks like. Heck, most of the time I have no clue myself.
I'm not claiming the candidate you describe (competent but can't remember a for loop) doesn't exist, just that I've never personally encountered such a beast. Quite the contrary, with a technically open minded interviewer it seems hard to get even the most introvert candidate to stop talking shop .. :-)
So I took the test and got 20 correct (as I imagine many people here would do as well). I thought it was very easy. The headhunter later told me that in 9 months, she had sent 52 people to the test, only 2 of us got 19, and I was the only one who got 20.
I'm not really sure what this means. That there are a lot of posers out there? That she wasn't very good at screening talent? That the companies seeking the best talent gladly pay $100 50 times to save their time?
I guess my biggest feeling is one of disappointment. It's just not that hard to become a good enough programmer to get 20 right every time. All it takes is study, passion, a lot of dedication, and a lot of hard work. I wish more people would do that. There aren't enough of us.