In response to last week’s article, in which I wrote about interviewing programmers for a position at my startup and giving them the FizzBuzz test, a number of people wrote in to tell me that knowing how to determine whether a given number is a multiple of another number — a task made easy with the modulus operator — was esoteric knowledge.
One such complaint came from none other than Rob Conery, who among other things is the guy behind the tutorial site Tekpub and co-creator of the excellent podcast This Developer’s Life, and doer of other interesting, geeky things. He wrote:
I would fail that test. I would fail it for a number of reasons – the first of which is what *you* perceive to be a “good programmer”. I don’t want to sound negative, but I would urge you to reconsider what you’re doing. You probably sent some very good talent out the door.
Here’s my point: these are *people*, not machines! I find the attitude of “let’s see how good you are at writing this arbitrary bit of gotcha code under the gun” soooooo very Enterprisey, where arbitrary metrics can somehow convey what the worker will do for you.
I can’t tell you the number of programmers I’ve hired “on a hunch” – that didn’t know the basics of what I needed them to do. But I could see the light in their eyes, I could *feel* their need to WIN – and you know what? They were the best damn employees I ever had.
You know who were the worst? The ultra-geek engineers that could kick out FizBuzz in minutes.
I wish this mentality would shift. I just hired a person here at Tekpub who I now couldn’t imagine doing business without. You know what he was doing when I hired him? REAL ESTATE APPRAISAL. Why did I hire him? Because he was smart and motivated. He picked up *everything* I needed him to know with 2 weeks – and he did it on his own because he *wanted it*.
Anyway – sorry Joey if I’m coming off negative. I have a soft spot for these kinds of posts that translate to “OMFG Look how DUMB these kids are!”.
/rant
Hey Rob, don’t worry about coming off negative! I have no trouble with people taking issue with my ideas in the comments, especially if it’s in the service of our craft. It’s also an honour to get a comment from one of my favourite podcasters.
Modulo Math
While I think that the modulo operator falls under “basic programming knowledge”, I don’t think it’s the crucial part of FizzBuzz. I think the decision-making part — knowing when to output “Fizz”, “Buzz”, “FizzBuzz” or the number — is far more crucial. Programming as we commonly do at present boils to down these:
- Giving instructions in a specific sequence
- Repeating actions in a loop
- Choosing among two or more possible actions based on one or more given conditions
I picked FizzBuzz as a test because it’s about as domain-unspecific as it gets. Maybe the “determine if a number is divisible by 3 or 5” part is too mathematical, but there’s no reason I can’t eliminate it.
Would the FizzBuzz test be more fair if I said “Assume that there’s a function called multiple_of_3 that accepts a number and returns true if and only if the number is a multiple of 3, and a function called multiple_of_5 that accepts a number and returns true if and only if the number is a multiple of 5?”
Pen and Paper
Some people wrote that doing FizzBuzz with pen and paper was unfair. Would you consider the FizzBuzz test to be more fair if I let candidates use a computer?
The Observer Effect
Other commenters wrote in and said that I may have passed up some good people because watching them like a hawk with a stopwatch as they wrote down their solution might have thrown them off their game.
Allow me to clarify: I did no such thing. I gave them a spot at my desk, sat down at another desk, pulled up my iPad and read. There was no stopwatch; just a quick look at the time every now and again, mindful of the fact that we had only an hour for the whole interview.
I tried to make it as non-intimidating as I could, and I was aware that even so, it must been at least a little intimidating. Call me a hard-ass if you must, but this is a startup, and we’re int he business of writing mobile software for enterprises. The field is one giant moving target, enterprises can be rather arbitrary and fickle customers, and we’ll be competing with established players with far more resources than we have. If you can’t take a reasonable amount of pressure (and hey, sometimes there’ll be an unreasonable amount), the programming jobs I have to offer aren’t for you.
Looking at Candidates’ Past Work
I asked candidates if they could show me samples of past work, especially work from their previous or current job. Some could, but others weren’t able to for various reasons. Some projects were company-internal websites that couldn’t accessed from outside. Some couldn’t be installed on their own computer for technological or legal reasons.
I also asked candidates to show me any side projects. If you have the time to have one, they’re a great way to demonstrate what you can do, and they’re completely under your control. However, not everyone has the time or inclination to have a side project.
One of the reasons I used FizzBuzz was to get around the problem of assessing their programming skill in the absence of being able to view a candidate’s past work.
Going with Your Gut
In his comment, Rob Conery pointed out something that I believe in: depending on your situation, the “ultra-geek engineers” might not be the best people for the job. He talked about a Tekpub hire whose last job wasn’t programming, but real estate appraisal. He went with his gut and simply hired someone who was a good fit, smart and hungry.
To that, my instinctive reply is: You know who also went with their guts? Every U.S. election pundit who predicted a Romney victory in spite of Nate Silver’s silly “math”.
But seriously: the guys I hired would’ve likely have been overlooked by Shopify, Unspace and a bunch of other hot-shot startups. Like Rob’s hires, they’re smart, hungry, and a good fit — and they also passed the FizzBuzz test.
Is it Fair to Even Have a Test?
The last two companies for which I worked — Microsoft, then Shopify — are quite different in many ways, but one thing they have in common is that they issue tests to people applying for development jobs. I thought I was being fair by issuing what I consider to be an incredibly basic test, but some people have written in to tell me that it’s wrong to have any test as part of the interview. What do you think?
10 replies on “A Fairer FizzBuzz? and Other Thoughts”
The test provided is simple logic. I’d be wary if someone couldn’t provide even pseudo code for such a simple test while applying for a developer position.
I remember fresh out of university, looking for a job, any job and applying to SAP to work in their IT group. They gave me a pretty simple test that anyone with experience should have been able to do. Best thing that could have happened – made me realize that it wasn’t the job for me if that was the type of stuff I would have been doing.
I’m not a huge fan of testing being someone who gets anxious about taking tests, but I also think that a well-done test is good for weeding out candidates that wouldn’t work out.
At the place where I work, we’ve definitely had a few people get hired on who quickly showed an immense inability to write code. Some had years of “experience”.
So I’m with you on administering some kind of test.
I get the feeling that a lot of people decrying the use of such questions think you’re expecting code on paper that could run the second you finish typing it in. I also get the feeling that you aren’t. A question like this would be looking at how the candidate solves problems, how they ask questions if they don’t understand something, how they make sure that they’re on the same page as to what’s being asked. The asking questions and verifying understanding is especially important in business-related programming.
To those who think this is unfair, or who think they would have failed: would you have talked to the interviewer, asked questions? Worked out what you understood and what you didn’t with them? If not, why not? This isn’t a University final, after all.
And to those who didn’t understand how to determine if a number was divisible by another: why not just use a stub function, say you’ll figure out that part later?
And finally, to everyone who’s written a solution so far: why has nobody put the FizzBuzz logic in a function, separate from the loop? How are you going to write tests if you don’t do that? :)
Meh, I think those people that say “tests are unfair” don’t know what their talking about. It depends on what you consider a test. Most interviews consist of verbal tests anyway, and this is unavoidable. If you consider “written” tests to be unfair, a verbal version of the fizz buzz test would probably be acceptable:
“How would you do fizzbuzz”
“I would make a loop to iterate over the set of numbers. Then, I would check that a number is divisible by 3 and if so, print fizz… ” and so on.
The problem with having a problem with written tests in programming is that programs are notoriously difficult to verbally “write”.
Also, other “artsy” domains would require either prior experience you can show or a “written” test. You wouldn’t hire a spell checker without making sure they can do the work with written words. You wouldn’t hire a graphical designer without seeing previous work or having them “write” a graphic for some design you come up with as a test.
In other words: Keep testing those programmers!
Comment on previous post didn’t submit so..
Yes, test…
If they don’t know about modulus, they should be able to figure out that (x/n) = int(x/n) for the correct number (or round, or whatever function it is for a given language). If you have any experience + problem solving, you should at least be able to reason this out.
My test is this (for Microsoft stack but others could be used):
You have 90 minutes to produce a calculator using your choice of Winforms, HTML/JS, ASP.NET, Silverlight or WPF.
It must have buttons for 0-9, +, -, *, /, =, C, CE.
It must operator like a typical calculator interms of the display reactions to button presses.
It must honour operator precedence.
Optional: Support multiple levels of parentheses.
This requirement can be met in under 90 minutes. The point isn’t whether you can do it all though, but how well you do and what you have to say at the end about what you have done and what you have done next.
If you can’t talk about algorithms and stacking (for example), you really shouldn’t be a developer.
The modulus operator is the key to implementing a hash table, one of the fundamental data structures. It’s like someone saying they are a competent French cook but don’t know how to make a roux. We teach kids about quotients and remainders in 3rd or 4th grade, this isn’t advanced math for crying out loud.
All of the startups around here have open plan offices jammed with computers side-by-side. People who can’t write FB when someone is sitting next to them in an interview probably aren’t going to be able to write much code on the job either. What about pair programming / debugging? Startups with pairing culture will pair with you while you code in the interview.
[…] A Fairer FizzBuzz? and Other Thoughts (Global Nerdy) […]
FizzBuzz is programming 101. If you are unable to solve something so trivial then you are simply not a good candidate. Specially ignore folks who say that the modulus operator is esoteric knowledge. It is not.
I think its pretty reasonable to have some sort of a test, particularly in a small/dynamic company. I’d justify it two ways: firstly, in a small company, the cost of a bad-hire is huge, particularly when a solid revenue stream hasn’t been established. Secondly, small companies require developers who can think on their feet & work well under pressure – the ability to hack together quick & reliable (maybe not perfect) solutions can be the difference between profitability & failure.
About 15 years ago, when I was still a mainframe COBOL programmer, my interview for one contract job contained a coding test. The job used IMS/DB (a heirarchical database), and I was well-versed in this at the time. The interviewer was my soon-to-be group leader, and he asked me to put some database access code on the whiteboard. He told me the parameters for the call, and I gave the code to him quickly and in two flavors, took me three minutes, while he watched. As soon as I finished he asked me how soon I could start work.
The thing is, he needed someone who could code to the DBMS they were using. I proved I was The One. A real estate appraiser (with no other qualifications) would not have worked out in this instance.
There’s nothing wrong with requiring a coder to code.
Wanting to “go with your gut” rather than use metrics is actually a very common phenomenon among people with expert knowledge in a field, especially when that expert knowledge is being substituted for a different kind of knowledge that is adjacent (in this case, the expert knowledge is programming, and the adjacent knowledge is hiring for programming). It’s even been consistently show that experts will commonly “go with their gut” rather than following an algorithm that has statistically better performance than expert opinion.
My point being that that commenter shouldn’t be too quick to trust his instincts and reject the metric. He might be able to look back (possibly with confirmation bias) and remember plenty of instances where his method was successful, but he shouldn’t use that as a metric to judge whether FizzBuzz is awful.