Just as it is with coding standards, everyone thinks they have the answers when it comes to hiring new people. I’m under no such delusions. I don’t pretend that this method will work for everyone. It won’t. This is simply the mechanism that I used to hire the team I currently work with. The team in question has turned out rather spectacularly, thanks in no small part to the effectiveness of this method. If you or your business is looking to hire more developers, I’d encourage you to consider this method, or at the very least, consider simplifying your hiring process to just the bare essentials that tell you what you’re really looking for.

When I first moved to Melbourne, I interviewed at maybe seven different companies. I’ve been on the receiving end of a lot of different styles of hiring processes. Let’s talk about some of the bullshit ones first. If you’re just here for tips and are not interested in anecdotes, feel free to skip to Improving Your Interview System.

Interview Only

This is the simplest one of all. This one is mostly the realm of small agencies and startups. The short of it is, there’s no real process. Candidate comes in, the business owner or whoever makes hiring decisions will talk to the person for maybe an hour, and will simply decide whether they like them or not.

The problems with this are obvious, but I want to say firstly that out of all the other methods people seem to try, this one is probably the least bullshit. It’s pretty honest, the hiring person will get a good gauge of the candidate’s personality and it’s not particularly stressful for the candidate. If you don’t have the expertise to come up with something more rigorous, or if you can’t afford it, then this is probably your best option.

Test All The Things!

Now for the complete opposite spectrum. I won’t name names, but there’s an online company in Melbourne who has an absolutely gruelling process they put candidates through. It’s not quite Google-levels of insane, but it is – in my opinion – extremely over the top and unnecessary.

First, you have to pass your recruiter’s technical competence test. This is usually some bullshit test on a testing platform like IKM. The one I did was riddled with spelling and grammar errors, and questions that legitimately have more than one correct answer (and most likely, the answer it was expecting is actually bad practice). For the love of fuck, don’t use these sites! If you’re absolutely in love with IKM, at least have someone who knows their shit go through and make sure their expected answers are technically sound! If you don’t even have that, go through the exam yourself and if you see a spelling or grammar error in the test, it’s extraordinarily likely that whoever wrote that test knows literally fuck all and is too busy sticking crayons up their nose to fix it.

Second, assuming you did well on the competence test, you now get to come on-site and meet some people! At this point, they’ll stuff you into a room with no less than four unfamiliar faces, all silently judging you. No pressure! In the interview itself, they’ll start by asking you the usual questions. Your experience, maybe ask you about your apps, your previous employment and the circumstances around your departure; standard stuff. Then comes the really foul smelling bovine fecal excretions.

You’ll be given a scenario that is not in any way, shape or form representative of your roles or responsibilities in this job. For example, I’m interviewing as an iOS developer, and my scenario was that I was in a client meeting – despite the fact that this online company doesn’t have clients – trying to work out exactly what they want us to deliver. Basically, the role of a Business Analyst. But I’m interviewing for a role as a developer. Remember, there are no less than four people all watching you stuff about doing something you’ll never have to actually do in your job. For maximum candidate embarrassment, why not ask them to demonstrate a skill that they absolutely do not need for their role, like say, drawing! Yes, drawing! This particular scenario had the four people in the room yelling abstract thoughts at you asking you to draw on the whiteboard what you think they want you to deliver.

Okay, so that’s already pretty bullshit, but it keeps going! That’s not even the last test! Once you’ve been sufficiently embarrassed by your stick figures, they move on to a written examination. It’s divided into no less than three god damn sections: multiple choice, short answer and critical thinking.

The multiple choice stuff actually ranges from pretty elementary (which of the following is not an Objective-C keyword) to downright evil (what is the result of running the following C code; note that it looks deliberately obfuscated). This is yet another example of being forced to do a task that is in no way, shape or form representative to your roles or responsibilities. If in my job, I ever had to verify what a piece of deliberately obfuscated iOS Objective-C code does without being allowed access to a compiler or developer tools, fuck, I’d resign so hard physicists would have to invent a new kind of dark energy to explain how I got the fuck out of there so fast.

The short answer section asks questions that are not always code-related. Here there were questions about resolving conflicts in git, to explain thoughts on unit testing, and to write short snippets of code to solve really simple problems. This part wasn’t so bad, but having to write code on pen-and-paper with a time limit is no fun, and again, not representative.

The final section, critical thinking, was truly surprising. I’ll talk about this later on, but it was the only part of this interview process that wasn’t an absolute tragedy. By this point most of the people in the room will have left you alone and you’ll have just one person there making sure you don’t “cheat” – I put that in quotes because how is using reference material and your actual tools cheating? It’s like interviewing a carpenter by asking them to build a house with his bare hands. How do people not see how monumentally absurd this is?

After all that, they’ll take you on a tour of the office so you can marvel at their wonderous facilities. Except, it’s just an office. I wasn’t sure what I was supposed to be seeing, but what I saw were cubicles, meeting rooms, stair cases, water fountains and lots of furniture. If they were doing something truly innovative with their workspaces, maybe this part wouldn’t be a waste of time, but they weren’t, so it was.

If you’re curious, I apparently aced this particular interview and was offered a position. I’ll get back to this later.

Improving Your Interview System

So, what exactly went wrong with all of the above? You might’ve noticed some recurring themes there, but here I’ll spell out some of the common mistakes in interview processes.

Always Give Representative Scenarios

This is by far the most important point. You might think it’s absolutely critical to ensure your developers have fantastic requirements elicitation skills, but that candidate probably knows for a fact their day-to-day won’t involve any work like that, and you probably know it too. Requirements elicitation is just an example, but the principle applies to anything. Don’t give them a task that they wouldn’t actually get in their job, and you should aim to simulate the conditions by which they’ll be given that task as realistically as possible.

This means they should absolutely be allowed to use reference material. They should have access to a compiler, they should be allowed to use the internet, their own notes and their own personal code library. Your scenario shouldn’t be Google-able.

Keep Your Interview Process Short and Straightforward

Google is probably the most notorious company when it comes to jerking around their candidates for long periods of time and putting them through extreme pressure. You should know that this reflects very badly on your company. You may say you have great working conditions, but the candidate’s first impression of your company is that you’re the kind of place that puts potential hires through stress. Personally, I wouldn’t interview at Google. I know plenty of really incredible, hard working, talented developers who shy away from interviewing there because they’ve heard horror stories about the interview process. I hate to think how much talent Google is passing up because some dickhead executive gets a hard-on over stressing their candidates to breaking point.

Ideally, your process should consist of two steps: determining technical competence, and determining cultural fit. You can come up with your own method for measuring this in candidates, but I’ll give an example of how we’ve done this later.

The company I interviewed for in the previous section, when all was said and done I had spent maybe 3 hours going through their exams and interviews. By the end of it, I was so wrecked I didn’t even care how well I did. I ended up getting competing offers from both this place and the place I ended up working for. I chose the latter in no small part because the former left a horrible impression with me due to the stresses imposed by the hiring process.

Expose Talent

The big problem with multiple-choice style questions where there’s a single correct answer is that it doesn’t give you much insight into the candidate’s thinking. There’s also maintainability problems; technology is constantly shifting, so over time you’ll have to review your questions and answers to make sure they’re still correct and relevant. Ideally, over time, the answers should change, but the question should not.

Multiple choice questions also don’t allow you to gauge how well a candidate did on a given question. A truly exceptional candidate should give a truly exceptional answer, but if there’s only one right answer, there’s no exceptional.

Creative open-ended questions also allow the candidate to really show you their chops. If they truly enjoy their job, they’ll probably be excited to get such a question, and will relish the thought of coming up with something you might not have seen before!

Comfort Counts

You should seriously consider allowing your candidate to perform the technical task in the comfort of their own home, or in whatever environment they work best. This requires them to have all the necessary tools at home to do so, but if they don’t, that should already be setting off alarm bells. If you’re uncomfortable giving your candidate unlimited time, you can always timebox the task by carefully co-ordinating emails.

If you allow your teams to work remotely, I’d consider this point absolutely mandatory. Even if you don’t, think back to when you were interviewing for your job. It’s obvious to everyone that job interviews are stressful and nerve-wracking, but people seem all too eager to perpetrate this when the shoe’s on the other foot. Stress and nerves can have serious detrimental effects on your reasoning and mental abilities. You should aim to minimise this as much as possible when examining their technical and creative ability. If you’re worried about them not being able to work well in your workplace, well, after a short while they’ll get comfortable there too. They almost certainly won’t be comfortable there during an interview.

Incompatible Candidates Should Be Let Off Quickly

The vast majority of your candidates will probably fail. Your process should expose these failures quickly, allowing you to reject them very early on. This saves your company time and money, and it means the job-seeker can quickly move on to other interviews, without worrying about being jerked around.

Our Screening Test

So there’s a lot of criteria there, but I think I’ve come up with an elegant implementation. The recruiter or talent scout should do their basic checks first, getting resumés and all that good stuff, but once they’re handed off to our team, we will give them a scenario like this:

Your manager has given you the following code from a colleague for code review. Please review the code for any issues and report back. Be creative! Outline any assumptions you’ve made as part of your response. Be sure to back up your criticism with constructive suggestions on how to improve the code. There’s an impending deadline, so you must complete your code review within 30 minutes, as evidenced by email timestamp. The code is pretty old, so it is sadly still manually reference counted.

UITeamView.h

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#import <UIKit/UIKit.h>
#import <Model/Developer.h>
#import <Model/Manager.h>

@protocol TeamViewDelegate
- (void) didSelectView: (UIView*) aView;
@end

@interface UITeamView : UIView {
	
}
@property (retain) id<TeamViewDelegate> _delegate;
@property (retain) Developer* developer;
@property (retain) Manager* manager;
@property (retain) UIView *m_devView;
@property (retain) UIView *mView;

@end

UITeamView.m

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
#import <TeamView.h>

@implementation UITeamView
@synthesize _delegate, developer, manager, m_devView, mView;

// NOTE: What you see is not necessarily the entire implementation
// of this class, but some specifically chosen methods for you to review.

- (void) setDeveloper (Developer*) aDev
{
	developer = [aDev retain];
	self.m_devView = [[[UIView alloc] initWithFrame: CGRectMake(0,100,160,50)] autorelease];
	[self addSubview: self.m_devView];

	UILabel *codeCount = [[[UILabel alloc] initWithFrame: CGRectMake(0,10,140,16)] autorelease];
	codeCount.font = [UIFont fontWithName: @"Helvetica" size: 14.0f];
	codeCount.text = self.developer.name + @": " + self.developer.linesOfCode;

	UITapGestureRecognizer *tap = [[UITapGestureRecognizer alloc] initWithTarget: self action: @selector(tapAction:)];
	[self.m_devView addGestureRecognizer: tap];
}

- (void) setManager: (Manager*) manager
{
	manager = [manager retain];

	self.mView = [[[UIView alloc] initWithFrame: CGRectMake(0,200,160,50)] autorelease];
	[self addSubview: self.mView];

	UILabel *lbl = [[[UILabel alloc] initWithFrame: CGRectMake(0,10,160,16)] autorelease];
	lbl.font = [UIFont fontWithName: @"Helvetica" size: 14.0f];
	lbl.text = self.manager.name;

	UITapGestureRecognizer *tap = [[UITapGestureRecognizer alloc] initWithTarget: self action: @selector(tapAction:)];
	[self.mView addGestureRecognizer: tap];
}

- (void) tapAction: (UIGestureRecognizer*) sender
{
	if (sender.view == self.m_devView) {
		[self.delegate didSelectView: self.m_devView];
	} else {
		[self.delegate didSelectView: self.mView];
	}
}

- (void) dealloc
{
	[_delegate release];
	[manager release];
	[developer release];
	[super release];
}

@end

As well as this, I have a fairly long “model” answer, which can be used as a benchmark, but I won’t share it here! This “model” answer doesn’t have all the answers – that’s kinda the whole point! Interesting, creative answers that aren’t in the “model” answer should actually be very exciting!

From the interviewer’s perspective, this format has several advantages:

• It properly tests the candidate’s problem solving skills.
• We can ascertain how compatible we are with the candidate’s opinion of best practice.
• It is maintainable; it’s resistant to changes in industry best practice. For the most part, the answers will change, not the question.
• It gives outstanding candidates the opportunity to really show off.
• It’s a pretty realistic scenario. There’s nothing in here that the candidate wouldn’t be expected to do as part of your job. This gives me just the information that is relevant and makes it quicker and easier to judge the competence of the candidate.
• It doesn’t take long to send this standard email template to a candidate. If they don’t pass, it’s easy to reject them, and it’ll happen early, saving us time.
• We can read into the way the candidate thinks by the kinds of answers they choose to give, in their limited time:
  • Juniors tend to focus on the superficial; the syntax errors, spelling mistakes, bad naming.
  • Seniors should be picking up on many of the superficial but also some of the logic errors and quirks that would happen when run.
  • People with architect or solutions design potential will pick up on issues with the class design, and poor adherence to standard code structure paradigms like MVC.

All this from a 30 minute test that can be done remotely!

From the candidate’s perspective, it has these advantages:

• It gives me the opportunity to be creative and really demonstrate my abilities.
• I can do this with my tools, my setup and my method, without the pressures of an interview environment or having strangers breathing down my neck.
• It shouldn’t feel like it’s outside my comfort zone.

No Code?

The most common question I get about this format is why we do not ask candidates to write actual code. Simply put, critiquing someone else’s code and suggesting better implementations is much more complicated and intricate than simply asking someone to implement FizzBuzz. As part of their answer, they’re absolutely free to completely rewrite this class if they want – the fact that they do so would even give you insight into the thinking and strategy of that particular person; that is, given a bad code review they’re likely to simply refactor/reimplement rather than help the author of the bad code improve.

People tend to be overly paranoid about accidentally hiring a developer who can’t actually code. This is relatively rare. The number of people who could give a solid critical code review, but not actually be able to write code themselves are such a statistical insignificance that devoting any amount of thought to this situation is completely wasteful. Besides, it’s not likely they’d proceed and still pass the interview. Which leads me to…

Our Interview

This is the part I was less involved in, so I’ll gloss over it a bit. If you were concerned about the candidate not being taken out of their comfort zone in the previous test, this is the part where you get to do just that. There’s not much to say really, each organisation and team will conduct their personal interviews the way that suits them best. I would suggest minimising the amount of people in the room, one-on-one is optimal, but two interviewers maximum.

Your goal here is to ascertain whether they’ll be a cultural fit. You should cycle in team members in the interview so they can all meet and have a go at asking the candidate questions. You should definitely ask questions about the screening test, so you can weed out anyone who got someone else to do their screening test for them.

It’s really up to your discretion how you run this part. Just keep in mind my suggestions above, and keep it relatively short but most importantly, representative of your actual corporate culture.

Final Thoughts

The people doing the hiring tend to forget that with the truly great candidates, when you’re interviewing, you are not the only one that is passing judgement. A good candidate will almost certainly be sizing you up for whether your company is the kind of place they want to work. As a company, you should be trying to impress your candidates, at least as much as they’re trying to impress you! Putting them through stress and ordeal will make them less likely to pick your company if they’re given multiple offers.

Hiring is not an exact science. There’s no foolproof method to get 100% great results all the time. There’s so much more involved in this than I’ve covered here, but if you find ways to optimise and cut down your interview process, it will save everyone time, especially yours and your candidate’s. Finally, getting this right might just leave everyone involved in this awful process just feeling a lot better.