I’m not sure how many people we offered jobs to during the recent recruitment drive, but 3 new Graduate Software Developers have started. We have never had a good induction process – we usually just expect people to start working on Day 1, but people won’t be familiar with our software, and the process.
Additionally, we always supply people with a stock laptop that could have been given to anyone in the business; so it doesn’t come with any development tools at all. So you need to install Visual Studio, SQL Server, and a few other things. However, these days, our IT has ramped up the security so now you cannot install anything without them remoting on to type in the administrator password, thus exacerbating the poor induction experience.
I wasn’t involved in creating the Induction process, the group of Senior Developers and Testers in India somehow went ahead without us in the UK.
They decided to use a Kanban board to track the tasks for the new recruits. They created tasks for each aspect like “Laptop configuration”, “Agile process overview”, “C# basics” and many others that made sense to me. The kanban idea was a good way of tracking their progress but it already seemed messy with 3 new starters but there’s more coming. I suggested that they use “Swimlanes” which can split the board into sections which can be used per person, but for some reason they rejected my idea in favour of their disorganised mess. So if there’s 10 tasks for each person, there’s 30 tasks that were initially on the backlog, and as they move to In Progress/Done, you cannot really tell how far each person is through the scheme without using additional filters, or configuring colours. Then when more new starters join, it becomes a complete mess with 10 more tasks being added.
I felt some of the tasks weren’t mandatory for the induction. One of them was Automation. Some developers in some teams are involved with automation, but it is usually given to the Software Testers. I think there’s different types of automation depending on the team you are in. The Seniors listed several types of automation, and provided links to LinkedIn Learning or Youtube – courses that were sometimes a few hours, but some were up to 8 hours. Then some were C# based, some were Java based, and some were Python based (it included JMeter, Selenium, Behave, PACT, Python, General automation). So you had to learn the base languages as a prerequisite.
“the videos are short, many around 7 hours, some are 1 hour”
Indian Senior Tester
When I saw that they had created a task that could potentially last 50 hours and wasn’t required for the job, I voiced my opposition to this idea and Becky, a Senior Tester agreed with me.
I said they aren’t going to learn if they are just watching videos, and won’t remember which is which. The constant switching of programming languages will probably hinder their retention. It’s very boring with the amount we are asking to do. This “Automation” learning task would already take a month to do if you are actually going to follow along and experiment with it. Then you have all the other induction tasks which involve reading, watching videos, and installing more software.
Becky pointed out we don’t even use this type of Automation in our particular teams so we shouldn’t ask them to do it. But then they said we “want to train them just in case it is used in the future“. We said we could just train them when it was needed, but they insisted it should be part of the Induction. Becky could sense they weren’t gonna back down so she asked that we inform them about which teams use automation and how often it is used so they can make their own judgement to learn it. They replied
“why should we inform them? we don’t want to set the expectation that they won’t ever use them”.
Indian Senior Tester
Towards the end of the meeting, he then said the new employees have already given feedback on the process. He said that all three agreed that:
“it’s hard to concentrate with all the videos we want them to watch, and all the reading we want them to do”.
Indian Senior Tester
Let me get this right. I said there was far too much for them to watch, and he argued with me about its importance, and then he said they have already complained about that exact problem, and they had only been here a week. Brilliant.
Fredrik Christenson (I didn’t make a note of the exact video it was from)once stated good developers can be judged by:
How they reason about a problem
Their way of communicating with people
How they voice their concerns/make decisions. Asking the right questions, pushing back in the right circumstances in order to get the right requirements; have “Guiding conversations”.
I think the general set-up of our interview can lead to this. The first part is for them to present the take-home test we gave them. They have to write a simple application with a UI and present data it retrieved through an API. The second part is to do some live coding, and define a few technical terms. Then the final section was some behavioural questions led by Colin.
Colin’s Update
Colin had interviewed several candidates for the Graduate Developer role in India. I asked him if the interviews had been going well, and he said the Graduates “couldn’t write code“. He said there was someone with a Masters degree, who then said they
“just learnt the course material a few days before the university exams then instantly forgot it”.
applicant
I guess at that point in a programming interview, when the interviewers are asking why you can’t write any code, then there is probably no way to get yourself out of that situation.
Saying you are forgetful is just instant rejection, although points for honesty, I suppose.
My Preparation
When it was my turn to do some interviews for the UK-based applicants, I was worried. I hadn’t been given any training or guidance, and the questions we had come up with were poor and I didn’t feel comfortable asking them. I was also scrutinising their applications and was trying to come up with questions based on their descriptions of their university course, or small amount of work experience. However, when it actually came to interviewing, there was actually someone else leading the interview, and Colin turned up too. So Colin had misled me that I would have a big part in the interviews. Also, Colin said he never asks questions about their CV, and sticks to coding or behavioural questions – so I’d wasted my time.
To be honest, I’d wasted my time anyway – because the first 2 interviews I was lined up to do were cancelled with half a day’s notice. Not sure if the HR team were slack or if the candidates backed out late. Colin speculated that people were dropping out because they couldn’t do the take-home test we gave them (which they would present in the interview). Maybe that means it is a good test. Colin said that he didn’t give the take-home test to the applicants in India, which could explain why he ended up interviewing people that couldn’t write code at all.
He didn’t give them the test due to our Indian Senior Developers saying the test was too hard for Indian Graduates to do (see Recruiting Graduates #1). To me, this seems like an admission that Indians are generally inferior, especially if they are happy for us to give the challenge to the UK applicants. This seems a very strange admission indeed.
The Cheats
Colin also said many of the applicants were caught cheating. On the questions where they were asked to define technical terms, they were reading the definitions from a website. He said some of them hid it well in terms of eye/arm movements (since they had to have their webcams on). However, the phrasing was a bit suspect, so Colin Googled the response and found the website they were reading it from.
He said another candidate seemed to pause whilst they thought about a question, then all of a sudden pasted in a block of code plagiarised from a website.
“HERE’S SOMETHING I PREPARED EARLIER”
fictional quote, inspired by the classic Blue Peter line.
Scoring
Colin did actually hire a few of the Indian applicants though, but when I looked through their Interview Feedback, the interviewers were scoring them 2/5 on most sections, but then a few sections were 0. So the scores were clearly a fail but we were hiring them anyway.
I also noticed that candidates who provided the programming answers in Python were scoring lower, even though, when I looked at the code – it looked correct. The problem is that it is a C# job and the interviewers only know C#. However, many of the Universities we were targeting used Python as their main language and we knew this up front.
So for the same job, we have a stricter process for UK candidates, and are also biassed towards C# candidates even though we only ask for knowledge of programming and don’t require prior C# knowledge.
Applicants
If people with English degrees bragged about their abilities like (some) software engineers:
“I’ve been speaking for 36 years and writing for 33 years. I’ve professionally used words from 10 different languages. Most days I speak or write 1000s of words to 10s of people.”
I did read through the CV’s of the people we hired. We hired Time Magazine’s “Person of the Year 2006”.I did wonder if Colin or the other interviewers had read that because it’s an instant red flag to me. It doesn’t sound like a legitimate claim, but then if you research who the award went to, it went to YOU. To put that on your CV is a joke though, unless you really did contribute significantly to online content. So if the candidate couldn’t justify it, I would just reject them.
Game Developer
For one of the guys that dropped out, I had some concerns with his application. Maybe he was legitimate but I think his claims weren’t backed up by the evidence he provided.
The first thing I noticed is that he stated “deep knowledge and proficiency in Java, Python, HTML, CSS, and JavaScript.” but I didn’t see any examples on his GitHub page, and there’s no commercial experience listed in his Work History. I wanted to ask him:
“Can you tell me what kinds of applications you have developed using these?”
My proposed question
He did say that his GitHub profile “included group projects.” but which ones were Group Projects? Working on group projects is fine, but I think it needs to be clearer so you can judge what he has done on his own. If they had used source control properly, you would see the commits by each user – which is where I saw the major red flag. All the commits were actually done by someone else. The only thing the applicant had done on the entire repository in his name was to initialise the repo, and fix a build issue at the end.
Your profile has another “contributor”, can you tell me who that is? – They have committed all the games to this repo.
My proposed question
Many of the games on the repository were Unity games, and I downloaded them, looked through the source code and play-tested them. For someone who had done a Games Degree, I was disappointed in the quality. Most of them I felt I could have made, and there were some really basic game design mistakes, and he said one of his University modules was “Advanced Game Design”. I wanted to ask this:
“If you worked on these games again: to fix bugs or improve the game design, what would you change about them?”
My proposed question
Example answers would be: In the Platforming game, you can stick to walls and the sides of platforms if you hold down the arrow key – which seems like an obvious bug. In his Snake game, it wasn’t clear what was happening, and you can move over enemy spawn positions to instantly kill them as they spawn in.
For his experience working in Home Bargains, he claims he
“Engaged with customers and built relationships to advise customers on products they’d like to know about”.
Candidate, clearly blagging
Home Bargains is basically just a supermarket, and he is hyping it up as if it is a sales job and personally gets customers returning. That doesn’t sound like a valid description of this role. I shop at Home Bargains and I don’t ever recall seeing anyone talk to the staff there unless there is some kind of dispute.
Python, Machine Learning Expert
The first guy I actually interviewed was about to finish his Masters degree, and has a Machine Learning degree. He seemed decent enough at Python to be able to do the take-home assignment, which he said he rushed through in a few hours because that was the only time he could dedicate to it. However, he seemed to struggle on the live coding part. I tried to prompt the candidate to talk us through his code, but he seemed to prefer coding in silence. I even helped him out by pointing out a bug in his code, but he ignored me, then wasted another 5 minutes working it out. I did wonder if he understood what I was saying. His CV claimed “Good communication and Presentation skills” but his presentation was full of “erms”, and during the live coding, he barely said anything, even when I reminded him to explain his thought process in order to gain more marks. By our scoring system, he scored very average but he has good credentials and I think there is potential there. You would think with his credentials, he would aim for a fancier job which could utilise machine learning though.
Possible Cheat
The second guy I interviewed got off to a good start. The program he demonstrated looked very complex. He said he was new to C#, and had good knowledge of C++, but he used our task as a good opportunity to learn C#. He actually used ASP.Net with Razor. It was mainly C# but it also contained some Javascript. He explained the program reasonably well, but then suddenly seemed to doubt himself and say that his program wasn’t very good. I assured him it was actually very impressive. However, when we moved onto the technical definitions, he couldn’t answer them very well at all. He really struggled with the live coding questions, then reminded us he is a C++ developer. However, after letting him re-attempt using C++, he still struggled to get something even close to what we asked. I find it hard to read C++ but he came up with some strange approaches and couldn’t explain his thought process at all. It made me think that he got someone else to write the take-home test.
3rd Guy
The third guy I interviewed seemed like a much well-rounded person. He explained his code well, asked questions when he was unsure of what we were asking, and showed he could debug (well, to a basic level anyway) when his code didn’t work. I made it clear to Colin that I thought he was the best candidate, but sadly he got offered a job elsewhere for way more money than what we could offer.
In this blog, I’m going to discuss some of the proposed questions as mentioned in #1, and discuss some questions that possibly or were asked in the actual interview process. The job is a Graduate Developer. The applicants will primarily be Indian but there are a couple of job positions available in the UK.
Many of the proposed questions were misleading/ambiguous/wrong.
Here are some highlights:
Multiple Choice Coding Questions
How many values a method can return in C#?
A) Any number of values
B) Only 1 value
C) Depends on the argument passed
D) Depends on class
Answer B
A method can have 1 type defined in the return, but this can be an object which contains many values. You could also say the List type is “many values”, and you can also define out parameters. A) seems a better answer in my opinion.
Multiple Choice Testing Questions
The expected results of the software is __________.
a. Only important in system testing
b. Only used in component testing
c. Most useful when specified in advance
d. Derived from the code.
The answer is A apparently. I think it should be C. Although it’s not “most useful” – more like essential. Why wouldn’t you think it is important at any other time? Don’t you care about expected behaviour during Regression Testing? Don’t you care for Unit Testing?
A test technique that involves testing with various ranges of valid and invalid inputs of a particular module or component functionality extensively is ___________.
a. Gorilla Testing
b. Monkey Testing
c. Agile Testing
d. Baseline Testing
Answer is Gorilla Testing, but I’ve never heard of it. I was a Tester for around 4 years, and have worked with Testers as a Developer for several years. I sometimes use the phrase “Gibbon Testing“, but that’s an unofficial testing practice that involves furiously clicking with the mouse, or bashing frantically on a touch-screen. What use is a question like that if it isn’t known by the average tester?
When an expected result is not specified in test case template then ___________.
a. We cannot run the test.
b. It may be difficult to repeat the test.
c. It may be difficult to determine if the test has passed or failed.
d. We cannot automate the user inputs.
Answer: C
I assume “Test Case Template” just means “Test Case”. I think the answer should be A, B, and C, but if you can only pick one, I would go with A because that implies B and C.
Which testing cannot be performed on first build of the software?
a. Regression testing
b. Retesting.
c. Sanity testing
d. Only A and B.
e. All of these
Answer: E
E doesn’t quite make sense because D states ONLY A and B. Of course, the answer is E. I suppose you have to actually assume “All” just means all the testing methods stated, rather than “all of the above” like it usually means on tests.
Face to Face Interview Questions
Behavioural
I found a document which I assume to be the face-to-face interview that Colin was doing. On the few interviews I have been on so far, I was only invited to the coding part of the interview, and even Colin’s notes from each candidates interview weren’t shared with me.
Here are his suggested questions. I think you are supposed to answer in the STAR format (Situation, Task, Action, Result), but it doesn’t quite apply to these. I really hope he didn’t present them to the candidates using this poor grammar, or vague statements.
Will you be interested in picking up the testing activity?
You get some clarification on your query in a work item from product, will you communicate the updated to other stakeholders or not, If yes why?
Will you handle the non-reproducible bugs in the dev environment? – Fixed bug working fine in dev environment but not in test environment
Will you be interested in picking up the testing activity?
Evaluation:Should be ready to pick itShould be ready to share the team’s work
The sentiment of the question is fine because sometimes when the Testing backlog is piling up, it makes sense for Developers to help out Test (as long as the Developer is not just testing their own fixes). In the past I have heard many developers complain that they are asked to do this. The thing is, even if that is your attitude, why would you answer that question honestly?
His evaluation criteria seems to imply, you are supposed to answer “yes” then mention something about “teamwork” then you get the points.
You get some clarification on your query in a work item from product, will you communicate the updated to other stakeholders or not, If yes why?
Evaluation:
Shall reduce the repeated work from the whole team
All will be in same page
Will help more for dev for coding and unit testing
Will help more for tester to complete the test design and review
Good, all stakeholders in the same page and sync
Will decrease PO effort in communicating the same thing to all team members
Translation: I think this is saying that you have to implement a new feature, but you question one of the requirements. After you ask the Product Owner to clarify, do you share this information you have learned?
It really sounds like you are supposed to say “yes” to it, then mention something about communication. In reality, I think the Product Owner should change the wording of the requirements on the “User Story” or add extra notes to the discussion section. If the information is recorded there, then any Developer/Tester/Manager who views it will see a record of what was discussed and no communication issues should occur. So I’d say Colin’s suggested points are fine, but the overall answer is really: “no, the Product owner should be responsible for updating the User Story”. However, Graduates won’t have experience working in this way, so I think they would just make assumptions and blag an answer.
Will you handle the non-reproducible bugs in the dev environment? – Fixed bug working fine in dev environment but not in test environment
Evaluation:
Whether raised bug is within the scoped limits
Patching done without any errors
Reproduction steps
Verifying the environment client version
Test evidence in the bug
Conducting a triage call with the production team for more info
Verifying the environment details
Translation: This means you have provided a fix for a bug, but then when on the Test System (or could have gone to the Live Environment), it isn’t working as intended. So what is your strategy for deciphering what has happened?
This is a good question, but maybe hard for Graduates if they have no experience. I think the answer is more along the lines of
Check the test environment definitely has your fix on it (patched to the same version number)
See if there is any difference in Configuration (maybe some Deployment options, Organisation options, User options etc)
See if you can see any difference in the selected data. So if you select a Customer then view their contact details, is there any difference in contact details on their system – compared to when you tested it on your computer?
Debug the code against their environment to work it out.
So Colin’s notes covers some of the points, but then his other points are vague and only he knows what it actually means.
Database Question
On the technical part, the candidate was sent a question pasted in chat. The grammar was whack and the formatting of the data was abysmal. How are you meant to work with this?
Select Project names which is having employee working in multiple project
Logic Questions
Question: Measure 4 Litres of water, only using a 5 Litre cup and a 3Litre cup (assume that you have an unlimited supply of water).
Answer: Fill 3L cup with water. Fill the 5L cup with the 3L cup. Fill 3L cup again. Fill 5L cup again with the 3L until it gets filled with water. This will result in you having 1L left in the 3L cup (and 5L in the 5L). Empty the 5L cup. Pour the 1L from the 3L cup to the 5L cup. Finally, fill 3L of water in 3L cup and transfer into the 5L cup.
I think when I first saw these type of questions I was baffled, but once you see an answer, you can easily answer variants of them. I recently played Tomb Raider: The Last Revelation and the final set of puzzles were variants of these and I did them with ease.
Question: In this scenario, you are standing in a room with three light switches. Each corresponds to a different light bulb in another room that you cannot see. If all the light switches are off, how do you find out which one turns on which bulb?
Answer: I will turn on one switch and leave it for a few minutes. When I turn it off, I will quickly turn on another switch. I will go to the room which shows what light turned on from the second switch, then feel the other bulbs for which is warmer (which would be the light I left on for a while). The third switch belongs to the bulb that is off and cold.
This question is nonsense. It never says you can leave the room, and it says “you cannot see” which implies you can never see them from a distance, or get close to them to touch. If you can leave the room, you could turn one on, go and look, then go back and turn on another. I think the question needs to stipulate you can only leave the room once only.
Question: You are driving a bus. There were 10 passengers already on the bus. You pick up 8 people from Stop A, drop 12 at Stop B, pick up 5 from Stop C and drop 9 at Stop D. Then, what is the age of the driver of the bus?
Answer: Your own age.
Question: You have to enter one of the three rooms. One room is on fire, another is occupied by a man-eating lion who is hungry for three months, and the last one is occupied by a terrorist with a fully loaded AK47.
Answer: The 2nd room because the lion is dead because of hunger. (Hungry for 3 months)
I’m not a fan of a question like this because wording it as “is hungry for three months” implies it is still alive and hungry. It needs to say “hasn’t consumedany food for three months”. I think it is intentionally been written like that to reduce the likelyhood of coming up with the answer. One of the aims of the test was to examine people’s understanding of English, and throwing in questions that are ambiguous, misleading or containing poor grammar isn’t the way forward.
Our job adverts have been terrible recently with all kinds of inconsistencies between the adverts, and even some factual inaccuracies.
For our Graduate recruitment drive in India, we put out an advert with the job title of:
“Fresher Hiring”
Isn’t a “fresher” someone who is in the first year of university? It’s not someone who is about to graduate or has graduated. It should say the job title to be consistent with the other adverts, so: “Graduate Development Software Engineer“.
Also, it’s not listed under Graduate or Student categories either so you won’t find it using the filters; additionally it is under “IT & Telecoms” rather than “Engineering”.
So what are we asking for?
Requirements
Essential
n Academic Qualification: B.E/B.Tech, MCA with consistent academic performance from X, XII standards onwards.
n Experience: Freshers
n Good Analytical and Problem-solving skills
n Knowledge of Dotnet
n Good written and oral communication skills.
n Keen to learn and pick up new skills, passionate about software development, enthusiastic and innovative.
Do you love the letter “n”? well join us today because we would love to have you. Seriously, they are supposed to be bullet points, but somehow HR have used “n”s. I assume this is some lazy copy-and-paste and it hasn’t recognised the bullet points. If HR can’t be bothered putting a good advert out, then why would good candidates apply? It’s ridiculous.
Also back to the “Experience: Freshers” bullet point…
interview question:
HR: "what experience with freshers do you have?"
Candidate: "well, we all got drunk and had many nights of debauchery"
“Good Analytical and Problem-solving skills”, “Good written and oral communication skills”, and “Keen to learn” are fairly generic qualities to put on a job advert but I actually think they are very appropriate for the job.
“Knowledge of Dotnet” is debatable. I mean, the job is in C# so you do need “.Net”, but as explained in the previous blog, we are going for Graduates that are mainly using Java or Python. So we can’t say that it is essential – it’s actually “desirable”. Also, I think it is rare to see it written as “Dotnet” rather than “.Net” so we should be using the correct terminology and spelling.
So even if they find the job advert, they might be put-off by the quality of it. The thing is, we have loads of developer and testing jobs available, in completely different team, products and locations. However, we don’t list the product or give information on the team (well 2 adverts had the name in the title; but it is meaningless to external candidates without a description) you are joining, so it’s impossible to separate seemingly duplicate listings. Furthermore, there is no consistency between the job titles or how we refer to the programming languages we are using. I think the term Software Development Engineer is a bit weird, and, until last year, I would have had no idea what SDE is an abbreviation for. If you were a developer, which job would you apply for out of these active listings?:
Fresh Hiring
Fresher Hiring
Fresher Hiring – SDE
SDE – .net
SDE – API Development
SDE – C# Dev
SDE Dotnet – Fresh Hiring
SDE (C#, SQL)
SDE – Team Crick
SDE – Team Crick C#/net and WPF
SDE – Frontend Developer
SDE – Dotnet Developer
SDE – dotNet & SQL
SDE – Dotnet
SDE – Dot Net Dev
SDE
Software Development Engineer
DotNet core Developer
Looking at a “Junior Software Development Engineer In Test” (yes, that really is the pretentious job title that Software Testers get these days), there were quite a few aspects of it that made me laugh. Let’s look at this:
Jr SDET
Essential
n Minimum + years of experience in a technical testing role.
n Strong knowledge of software testing approaches including agile testing techniques.
n Superb communication skills both written and verbal.
n Experience in driving UI testing down the stack and increasing robustness of the test infrastructure.
n Exposure to some aspects/phases of automation including GUI, integration testing, and performance/load testing.
n Moderate experience working with Product Owners and SMEs to define user story examples/acceptance criteria and exposure to Gherkin using tools like SpecFlow and Fitnesse.
n Some experience in understanding and testing complex enterprise systems architecture.
n Excellent MSSQL skills and understanding of complex XML structures.
n Good programming skills required in C#/Java.
n Good experience defining lightweight, high value test plans and tests.
n Experience defining and tracking test metrics throughout product development.
n Good experience mentoring and coaching Software Engineers in Test.
n Ability to drive the technical testing approach and guide less experienced department members
n Experience working closely with a team of software engineers and product owners.
n Good knowledge of scripting technologies such as PowerShell or Python.
So the “n” bullet points are here again. The opening line says they want “Minimum + years of experience“. Minimum plus years. Brilliant.
“Experience in driving UI testing down the stack“. What the hell does that mean!?
Notice how we want experience in quite a few aspects, and good programming skills (in multiple languages!), but it is for a JUNIOR role, not a standard or Senior role. Then they want “Good experience mentoring and coaching” which is surely a Senior responsibility.
How do we expect to get many applicants with poor adverts which are hard to find on the website, then have formatting errors or other questionable content?
Stranger Coding Tings 