Hiring programmers is a challenging process for companies. A lot of thought is being given to designing good interview questions and coding challenges to assess the skills of candidates in a hands-on way, either as a take-home exercise or a whiteboard interview. I know this from having designed and evaluated coding challenges myself, with respect to code quality, structure, and optimality of the solution.
This evaluation is useful to a certain extent. What the process really boil downs to is asking the candidate to create an algorithm from scratch, finishing it within a short amount of time (typically anywhere between 30 minutes to a few hours), and then hoping that doing well on this task will translate into being a good fit for the team.
The truth is there are many skills this practice is unable to cover. This is because working as a programmer mostly involves working with existing code, not from scratch, and within a team, not alone.
Due to this, a programmer should have solid communication skills and an effective approach to problem solving in the real, imperfect world. Assessing these skills, which are sometimes termed as ‘soft skills’, when done at all, is typically by asking the candidate questions about past or hypothetical scenarios, for example asking them to “walk us through an occasion where you had to ask for help”.
While this is useful in the sense that past behavior can predict future behavior, it also gives the candidate an opportunity to guess what the answer should be. Additionally, candidates tend to dislike these questions, as well.
Companies do not test for, say, Python skills, by asking the candidate to tell a story about a time they wrote a Python program. They ask them to write a Python program. What if there was a way to assess skills that are beyond algorithms in a similar, hands-on way? Not through a separate HR interview, not through behavioral questions, but baked in the same process that assesses coding skills.
This is what we’re going to cover in this article. We’re going to look at how to modify the different interview stages, specifically the phone screening stage, the take-home exercise, and the on-site interview, to get deeper knowledge about the candidate’s ability to communicate, how they work with an existing codebase and others, and the resourcefulness they bring to problem-solving.
Looking for top talent fast? See how Arc can help you:
⚡️ Find developers, designers, marketers, and more
⚡️ Freelance or full-time remote + fully vetted
⚡️ Save up to 58% with global hires
Hire top talent with Arc risk-free →
Phone Screening
Phone screenings are typically used to get more information about the candidate’s past experience. However, it’s possible to assess the candidate’s awareness of what they do and do not know, and how precisely they can communicate.
Self-awareness
Being aware of what one does and does not know, and being forthcoming with this information is a real gift when working in a team. If an employee is aware and honest about not possessing the knowledge needed to finish a task, it frees the team to look for alternatives, such as allocating time to acquire the knowledge, getting training, and hiring externals. The team is able to weigh the alternatives against the benefits of completing the task.
One way to assess this during the phone screening is to have an extensive list of questions across a large variety of topics to ask the candidate, for example asking a full-stack engineer technical interview questions on everything from CSS to database internals. It is good to tell the candidate upfront: “We’re not looking for guesses or for answers to all the questions, we’re just looking to discover the areas you are strong in.”
We can proceed to ask them questions for a fixed amount of time, usually 5 to 10 minutes. During the process, we can observe and record not only what the candidate knows, but also how they act when they don’t know something.
In addition to correct answers, some positive signs to look for include: quick admissions that the candidate has not worked with a certain technology, and statements that deal effectively and honestly with uncertainty, such as “I am not certain whether this particular database system implements bitmap indexes. If it does, we could use them, otherwise we’d have to go for B-trees.”
Read More: How to Find Developers: 18+ Expert Tips & Essential Strategies
You can also try Arc, your shortcut to the world’s best remote talent:
⚡️ Access 350,000 top developers, designers, and marketers
⚡️ Vetted and ready to interview
⚡️ Freelance or full-time
Try Arc and hire top talent now →
Precision
Precise communicators make things easier on many fronts. They are able to give pertinent information when describing something (such as a bug that needs to be fixed, the pros and cons of an architectural decision, how a system works, etc), and elaborate in the correct direction when prompted.
During the phone screening, we want to see if the candidate can give a precise account of their past experience. One way to facilitate this exchange is to ask clean language questions. The basic philosophy behind this is to minimize our assumptions as interviewers and explore the candidate’s model of their own work. Clean language questions in the context of a tech interview are questions like these:
- What kind of [technology/database/etc] did you use?
- What does [complex system] look like for [some kind of user]?
- And when [the data is processed], what happens afterward?
- What happens before [the logs are saved on S3]?
- Is there anything else [you have lambdas for]?
We can start this part of the interview by asking the candidate to “describe a complex project they worked on,” and continue with questions similar to the previous examples. We should use a gentle manner as we want to make the candidate feel heard, not as if they were being interrogated.
Clean language questions serve a dual purpose. First, they give deeper information about the candidate’s past work, which is valuable in itself, and also allows interviewers to see how well the candidate can elaborate when asked questions about very specific aspects of their work.
Another positive sign to look for is when the candidate asks questions to clarify what is being requested of them, such as “What do you mean by ‘complex’ project?”
Read More: How to Implement a Welcoming Software Developer Onboarding Process
Take-home exercise: Extending an existing code base
The take-home exercise is generally used to evaluate a candidate’s algorithmic skills, by looking at how they solve a problem. We can learn even more about the candidate with some alterations to the process.
Most of the work programmers do day-to-day is not a greenfield project. A large portion is reading and building on top of existing code, which requires additional problem-solving skills. Hence, instead of giving the candidate an open prompt to create a small program from scratch, we can give them an existing code base that they can fork.
This codebase would, ideally, be Dockerized so that setting up the dependencies is easy, and produced by the company. It should still be shareable, for example, a small open-sourced project. We can also set up a private Slack channel where the candidate can ask engineers questions during the development process.
We can ask the candidate to add extra functionality to the code base to solve a new problem by creating commits in their own fork and then doing a pull request to the master repository. Their ability to perform these tasks shows not only their algorithmic skills, but also their ability to read, understand, and figure out where to make changes on code not written by them.
We can make sure to include a test suite, paying attention to whether the candidate extends the test suite in addition to the code. The take-home exercise can be reused if the candidate comes onsite, which we will look at in the next section.
Through the take-home task, the candidate will get a glimpse of what the company’s typical codebase looks like, and an idea of what working with the team feels like.
Read More: 10+ Software Engineer Interview Questions to Find Top Dev Candidates
On-site interview
The on-site interview provides the opportunity to test-drive how the company and the candidate can work together. It makes sense to take advantage of both sides’ time to delve deeper into that. If we are to go beyond whiteboard interviews, here are a few ideas for tasks to assess specific skills:
Dealing with change and asking for support
A task that can give both the candidate and the company a deeper understanding of each other is asking the candidate to pair-program with a member (or members) of the hiring team. We can tell the candidate that the requirements of their take-home exercise have changed and they need to work with another programmer, and optionally, a business analyst, to reflect this change in the code.
We can then ask the candidate to sit down with the teammates and their laptops in a meeting room to collaborate on making the changes. All people involved can use the task management system the company uses, for example, JIRA, Trello, or a physical Kanban board.
We observe whether the candidate uses the resources provided (i.e. the other people and the information captured in the task management system) to figure out what exactly needs to be done, what would be the most consistent way to do it, and how they address possible ways of splitting of the work. We can see where they lie on the spectrum of approaching changing requirements: from resisting them to the other extreme of starting over completely.
This task will provide symmetrical information to the candidate: how tasks get distributed and assigned in the company, how well project managers can communicate requirements, and how potential future coworkers can share knowledge and collaborate.
Read More: Andela vs Toptal vs Turing vs Arc: Which is the Best Andela Alternative?
Constructive feedback
Another skill to test for is how the candidate provides feedback. This can be assessed directly by isolating a relatively self-contained bit of the real code base, printing it out, and having the candidate read and take notes with the intention of doing a code review by posing the question “what do you think of the code and how would you improve it?”
It’s important to pay attention to whether the candidate sprinkles positive comments together with negative ones, whether they are able to outline concise steps for iteratively improving whatever stands out, and whether they ask questions to find out if those improvements would be worth the effort or would affect other parts of the system.
People that only see mistakes in the work of others without offering solutions can be hugely demoralizing for the entire team, no matter how sharp their observations are. People who are well-meaning but want to roll out too many changes at once probably do not have a lot of production experience.
Speaking from personal experience, people who include honest and positive feedback in their reviews actually get thanked when reviewing pull requests. Those who understand while there’s room for improvement, it’s sometimes best to opt out of improving what’s already good enough, are the ones that can drive impactful change.
This task has the added effect of giving the candidate a look into real company code. Moreover, if the candidate communicates to the interviewer an alternative way of doing things that they weren’t aware of, how positively this suggestion is received can be very telling of the company’s ability to support iterative improvement. This quality may be something that the candidate is looking for in a workplace.
Read More: 13 Things Freelance Software Developers HATE Hearing From Clients
Conclusion
By going beyond the whiteboard interview, we’re able to assess the candidate’s ability beyond coding skills. This can be as simple as going through the existing interview process and tweaking it with the intention of learning more about the candidate.
Naturally, time will need to be invested to find and package specific parts of the codebase that can be shared with the candidate, and to train interviewers to ask discerning questions. This adjustment will have the symmetrical effect of also giving the candidate a clearer picture of what it’s like to work with the team. If you want support overhauling your interview process, don’t hesitate to get in touch.
As a parting thought: remember to always keep a positive disposition. As interviewers, we’re here to discover people’s true strengths, and it’s up to us to design a process that will bring them out.
You can also try Arc, your shortcut to the world’s best remote talent:
⚡️ Access 350,000 top developers, designers, and marketers
⚡️ Vetted and ready to interview
⚡️ Freelance or full-time
