I think a big challenge during the Software Development hiring process is how to determine who is good or not.
Sometimes companies give large projects as a do-at-home test, but this puts people off when they currently have a job and have other commitments that take up their free time.
You could be given a smaller task, but then it may still not give a good representation of how someone codes, and you could easily get someone else to write it for you.
Other companies leave it until the interview and make developers write code in a manner that is completely out of their comfort zone; e.g. using pen and paper, or a whiteboard. Apparently, Google makes you write code in a shared Google Doc.
I’ve sometimes been given a timed test to complete at home. So you are emailed a list of varied questions and have to quickly code them then send them back via email within an hour.
The challenges they make you do can be varied. Sometimes they can be really basic, using fundamental concepts you will use on day 1 like simple loops. Sometimes they opt for the computer science route, and ask you to implement classic algorithms, maybe a sorting algorithm like bubble sort. Sometimes I have been asked to create a structure like a linked list, or a Directed Acyclic Graph.
You could specifically study these typical questions and then get past the code part of the interview process – even though in a real job you would struggle when posed with various real-world problems. There are plenty of websites that sell training courses that target these algorithms because they are so widely used in the hiring process.
So in the worst case, you will be writing code with a pen, with no resources to help you, under pressure from people watching and judging you.
Yet, when you are employed, you are using an IDE like Visual Studio on a PC, have more time to come up with a good implementation, and can use many internet resources to help you. You will need a good knowledge of the existing codebase, and be utilising 3rd party libraries.
You need to be able to communicate effectively with team members, use Google/Stack Overflow to find solutions/answers. I think it’s important to be able to do code reviews, and show debugging skills, but this is never tested as far as I’ve seen.
In a high-pressure interview, you don’t get a representation of how the person is day to day. I tend to panic in an interview, but am completely relaxed in my job. Even when there is a tight deadline, it still doesn’t faze me as much as an interview situation. So I’d say you can’t even reliably test “ability to handle pressure”.
There was one time we have used this typical interview approach for internal candidates. What’s the point giving a programming test to a software engineer that’s currently employed at your company? It makes sense to give this test to someone moving from another department. I got denied a Senior promotion using this method, yet most of my team members would have vouched that I was among the best in the team. Yet, underperforming Developers got promoted using this process. I’ve written many stories about them here (Colin, and Derek).
I’m not sure what the best way of judging developers, but I would probably go for a programming test using a laptop while allowing use of the internet (or just the interviewer’s knowledge). The test would involve writing some new code, but building upon some existing code which has a bug or two that the candidate is required to debug and fix. The final part of my test would be to review a selection of code changes to see how they handle Code Reviews. This could take a few hours, so I’d only include a few standard questions.
Currently, it’s a failed system if someone can be an excellent programmer, demonstrated by a comprehensive Github profile, yet fail an interview because they couldn’t come up with an optimal solution for some problem while being surrounded by a panel of intimidating interviewers.