Tag: Colin

Recruiting Graduates #4: The Applicants

Introduction

Other blogs in this series Recruiting Graduates #1, Recruiting Graduates #2, Recruiting Graduates #3.

Fredrik Christenson (I didn’t make a note of the exact video it was from) once stated good developers can be judged by:

  1. How they reason about a problem
  2. Their way of communicating with people
  3. How they voice their concerns/make decisions. Asking the right questions, pushing back in the right circumstances in order to get the right requirements; have “Guiding conversations”.

I think the general set-up of our interview can lead to this. The first part is for them to present the take-home test we gave them. They have to write a simple application with a UI and present data it retrieved through an API. The second part is to do some live coding, and define a few technical terms. Then the final section was some behavioural questions led by Colin.

Colin’s Update

Colin had interviewed several candidates for the Graduate Developer role in India. I asked him if the interviews had been going well, and he said the Graduates “couldn’t write code“. He said there was someone with a Masters degree, who then said they

“just learnt the course material a few days before the university exams then instantly forgot it”.

applicant

I guess at that point in a programming interview, when the interviewers are asking why you can’t write any code, then there is probably no way to get yourself out of that situation.

Saying you are forgetful is just instant rejection, although points for honesty, I suppose.

My Preparation

When it was my turn to do some interviews for the UK-based applicants, I was worried. I hadn’t been given any training or guidance, and the questions we had come up with were poor and I didn’t feel comfortable asking them. I was also scrutinising their applications and was trying to come up with questions based on their descriptions of their university course, or small amount of work experience. However, when it actually came to interviewing, there was actually someone else leading the interview, and Colin turned up too. So Colin had misled me that I would have a big part in the interviews. Also, Colin said he never asks questions about their CV, and sticks to coding or behavioural questions – so I’d wasted my time.

To be honest, I’d wasted my time anyway – because the first 2 interviews I was lined up to do were cancelled with half a day’s notice. Not sure if the HR team were slack or if the candidates backed out late. Colin speculated that people were dropping out because they couldn’t do the take-home test we gave them (which they would present in the interview). Maybe that means it is a good test. Colin said that he didn’t give the take-home test to the applicants in India, which could explain why he ended up interviewing people that couldn’t write code at all. 

He didn’t give them the test due to our Indian Senior Developers saying the test was too hard for Indian Graduates to do (see Recruiting Graduates #1). To me, this seems like an admission that Indians are generally inferior, especially if they are happy for us to give the challenge to the UK applicants. This seems a very strange admission indeed. 

The Cheats

Colin also said many of the applicants were caught cheating. On the questions where they were asked to define technical terms, they were reading the definitions from a website. He said some of them hid it well in terms of eye/arm movements (since they had to have their webcams on). However, the phrasing was a bit suspect, so Colin Googled the response and found the website they were reading it from.

He said another candidate seemed to pause whilst they thought about a question, then all of a sudden pasted in a block of code plagiarised from a website. 

“HERE’S SOMETHING I PREPARED EARLIER”

fictional quote, inspired by the classic Blue Peter line.

Scoring

Colin did actually hire a few of the Indian applicants though, but when I looked through their Interview Feedback, the interviewers were scoring them 2/5 on most sections, but then a few sections were 0. So the scores were clearly a fail but we were hiring them anyway.

I also noticed that candidates who provided the programming answers in Python were scoring lower, even though, when I looked at the code – it looked correct. The problem is that it is a C# job and the interviewers only know C#. However, many of the Universities we were targeting used Python as their main language and we knew this up front.

So for the same job, we have a stricter process for UK candidates, and are also biassed towards C# candidates even though we only ask for knowledge of programming and don’t require prior C# knowledge.

Applicants

Time Magazine’s “Person of the Year 2006”

I did read through the CV’s of the people we hired. We hired Time Magazine’s “Person of the Year 2006”. I did wonder if Colin or the other interviewers had read that because it’s an instant red flag to me. It doesn’t sound like a legitimate claim, but then if you research who the award went to, it went to YOU. To put that on your CV is a joke though, unless you really did contribute significantly to online content. So if the candidate couldn’t justify it, I would just reject them.

Game Developer

For one of the guys that dropped out, I had some concerns with his application. Maybe he was legitimate but I think his claims weren’t backed up by the evidence he provided.

The first thing I noticed is that he stated “deep knowledge and proficiency in Java, Python, HTML, CSS, and JavaScript.” but I didn’t see any examples on his GitHub page, and there’s no commercial experience listed in his Work History. I wanted to ask him:

 “Can you tell me what kinds of applications you have developed using these?”

My proposed question

He did say that his GitHub profile “included group projects.” but which ones were Group Projects? Working on group projects is fine, but I think it needs to be clearer so you can judge what he has done on his own. If they had used source control properly, you would see the commits by each user – which is where I saw the major red flag. All the commits were actually done by someone else. The only thing the applicant had done on the entire repository in his name was to initialise the repo, and fix a build issue at the end.

Your profile has another “contributor”, can you tell me who that is?  – They have committed all the games to this repo.

My proposed question

Many of the games on the repository were Unity games, and I downloaded them, looked through the source code and play-tested them. For someone who had done a Games Degree, I was disappointed in the quality. Most of them I felt I could have made, and there were some really basic game design mistakes, and he said one of his University modules was “Advanced Game Design”. I wanted to ask this:

“If you worked on these games again: to fix bugs or improve the game design, what would you change about them?”

My proposed question

Example answers would be: In the Platforming game, you can stick to walls and the sides of platforms if you hold down the arrow key – which seems like an obvious bug. In his Snake game, it wasn’t clear what was happening, and you can move over enemy spawn positions to instantly kill them as they spawn in. 

For his experience working in Home Bargains, he claims he

“Engaged with customers and built relationships to advise customers on products they’d like to know about”.

Candidate, clearly blagging

Home Bargains is basically just a supermarket, and he is hyping it up as if it is a sales job and personally gets customers returning. That doesn’t sound like a valid description of this role. I shop at Home Bargains and I don’t ever recall seeing anyone talk to the staff there unless there is some kind of dispute. 

Python, Machine Learning Expert

The first guy I actually interviewed was about to finish his Masters degree, and has a Machine Learning degree. He seemed decent enough at Python to be able to do the take-home assignment, which he said he rushed through in a few hours because that was the only time he could dedicate to it. However, he seemed to struggle on the live coding part. I tried to prompt the candidate to talk us through his code, but he seemed to prefer coding in silence. I even helped him out by pointing out a bug in his code, but he ignored me, then wasted another 5 minutes working it out. I did wonder if he understood what I was saying. His CV claimed “Good communication and Presentation skills” but his presentation was full of “erms”, and during the live coding, he barely said anything, even when I reminded him to explain his thought process in order to gain more marks. By our scoring system, he scored very average but he has good credentials and I think there is potential there. You would think with his credentials, he would aim for a fancier job which could utilise machine learning though.

Possible Cheat

The second guy I interviewed got off to a good start. The program he demonstrated looked very complex. He said he was new to C#, and had good knowledge of C++, but he used our task as a good opportunity to learn C#. He actually used ASP.Net with Razor. It was mainly C# but it also contained some Javascript. He explained the program reasonably well, but then suddenly seemed to doubt himself and say that his program wasn’t very good. I assured him it was actually very impressive. However, when we moved onto the technical definitions, he couldn’t answer them very well at all. He really struggled with the live coding questions, then reminded us he is a C++ developer. However, after letting him re-attempt using C++, he still struggled to get something even close to what we asked. I find it hard to read C++ but he came up with some strange approaches and couldn’t explain his thought process at all. It made me think that he got someone else to write the take-home test.

3rd Guy

The third guy I interviewed seemed like a much well-rounded person. He explained his code well, asked questions when he was unsure of what we were asking, and showed he could debug (well, to a basic level anyway) when his code didn’t work. I made it clear to Colin that I thought he was the best candidate, but sadly he got offered a job elsewhere for way more money than what we could offer.

WE HAVE FLEXIBLE WORKING HOURS. INTERVIEWER NO. WE DECIDE YOUR WORKING HOURS. J) WorkChronicIes.com Fo//ow me on GREAT! SO 1 DECIDE MY WORKING HOURS ? WE STRETCH IT AS MUCH AS WE CAN. IT'S VERY FLEXIBLE. b

Recruiting Graduates 3#: The Questions

Introduction

Other blogs in this series Recruiting Graduates #1 and Recruiting Graduates #2

In this blog, I’m going to discuss some of the proposed questions as mentioned in #1, and discuss some questions that possibly or were asked in the actual interview process. The job is a Graduate Developer. The applicants will primarily be Indian but there are a couple of job positions available in the UK.

Many of the proposed questions were misleading/ambiguous/wrong.

Here are some highlights:

Multiple Choice Coding Questions

How many values a method can return in C#? 

A) Any number of values 
B) Only 1 value 
C) Depends on the argument passed 
D) Depends on class 
Answer B

A method can have 1 type defined in the return, but this can be an object which contains many values. You could also say the List type is “many values”, and you can also define out parameters. A) seems a better answer in my opinion.

Multiple Choice Testing Questions

The expected results of the software is __________.
a. Only important in system testing
b. Only used in component testing
c. Most useful when specified in advance
d. Derived from the code.

The answer is A apparently.
I think it should be C. Although it’s not “most useful” – more like essential.
Why wouldn’t you think it is important at any other time? Don’t you care about expected behaviour during Regression Testing? Don’t you care for Unit Testing?

A test technique that involves testing with various ranges of valid and invalid inputs of a particular module or component functionality extensively is ___________.
a. Gorilla Testing
b. Monkey Testing
c. Agile Testing
d. Baseline Testing

Answer is Gorilla Testing, but I’ve never heard of it. I was a Tester for around 4 years, and have worked with Testers as a Developer for several years. I sometimes use the phrase “Gibbon Testing“, but that’s an unofficial testing practice that involves furiously clicking with the mouse, or bashing frantically on a touch-screen. What use is a question like that if it isn’t known by the average tester?

When an expected result is not specified in test case template then ___________.
a. We cannot run the test.
b. It may be difficult to repeat the test.
c. It may be difficult to determine if the test has passed or failed.
d. We cannot automate the user inputs.
Answer: C

I assume “Test Case Template” just means “Test Case”. I think the answer should be A, B, and C, but if you can only pick one, I would go with A because that implies B and C.

Which testing cannot be performed on first build of the software?
a. Regression testing
b. Retesting.
c. Sanity testing
d. Only A and B.
e. All of these
Answer: E

E doesn’t quite make sense because D states ONLY A and B. Of course, the answer is E. I suppose you have to actually assume “All” just means all the testing methods stated, rather than “all of the above” like it usually means on tests.

Face to Face Interview Questions

Behavioural

I found a document which I assume to be the face-to-face interview that Colin was doing. On the few interviews I have been on so far, I was only invited to the coding part of the interview, and even Colin’s notes from each candidates interview weren’t shared with me.

Here are his suggested questions. I think you are supposed to answer in the STAR format (Situation, Task, Action, Result), but it doesn’t quite apply to these. I really hope he didn’t present them to the candidates using this poor grammar, or vague statements.

  1. Will you be interested in picking up the testing activity?
  2. You get some clarification on your query in a work item from product, will you communicate the updated to other stakeholders or not, If yes why?
  3. Will you handle the non-reproducible bugs in the dev environment? – Fixed bug working fine in dev environment but not in test environment
Will you be interested in picking up the testing activity?
Evaluation:
Should be ready to pick it
Should be ready to share the team’s work

The sentiment of the question is fine because sometimes when the Testing backlog is piling up, it makes sense for Developers to help out Test (as long as the Developer is not just testing their own fixes). In the past I have heard many developers complain that they are asked to do this. The thing is, even if that is your attitude, why would you answer that question honestly?

His evaluation criteria seems to imply, you are supposed to answer “yes” then mention something about “teamwork” then you get the points.

You get some clarification on your query in a work item from product, will you communicate the updated to other stakeholders or not, If yes why?
Evaluation:
Shall reduce the repeated work from the whole team
All will be in same page
Will help more for dev for coding and unit testing
Will help more for tester to complete the test design and review
Good, all stakeholders in the same page and sync
Will decrease PO effort in communicating the same thing to all team members

Translation: I think this is saying that you have to implement a new feature, but you question one of the requirements. After you ask the Product Owner to clarify, do you share this information you have learned?

It really sounds like you are supposed to say “yes” to it, then mention something about communication. In reality, I think the Product Owner should change the wording of the requirements on the “User Story” or add extra notes to the discussion section. If the information is recorded there, then any Developer/Tester/Manager who views it will see a record of what was discussed and no communication issues should occur. So I’d say Colin’s suggested points are fine, but the overall answer is really: “no, the Product owner should be responsible for updating the User Story”. However, Graduates won’t have experience working in this way, so I think they would just make assumptions and blag an answer.

Will you handle the non-reproducible bugs in the dev environment? – Fixed bug working fine in dev environment but not in test environment
Evaluation:
Whether raised bug is within the scoped limits
Patching done without any errors
Reproduction steps
Verifying the environment client version
Test evidence in the bug
Conducting a triage call with the production team for more info
Verifying the environment details

Translation: This means you have provided a fix for a bug, but then when on the Test System (or could have gone to the Live Environment), it isn’t working as intended. So what is your strategy for deciphering what has happened?

This is a good question, but maybe hard for Graduates if they have no experience. I think the answer is more along the lines of

  • Check the test environment definitely has your fix on it (patched to the same version number)
  • See if there is any difference in Configuration (maybe some Deployment options, Organisation options, User options etc)
  • See if you can see any difference in the selected data. So if you select a Customer then view their contact details, is there any difference in contact details on their system – compared to when you tested it on your computer?
  • Debug the code against their environment to work it out.

So Colin’s notes covers some of the points, but then his other points are vague and only he knows what it actually means.

Database Question

On the technical part, the candidate was sent a question pasted in chat. The grammar was whack and the formatting of the data was abysmal. How are you meant to work with this?

Select Project names which is having employee working in multiple project

Logic Questions

Question: Measure 4 Litres of water, only using a 5 Litre cup and a 3Litre cup (assume that you have an unlimited supply of water).

Answer: Fill 3L cup with water. Fill the 5L cup with the 3L cup. Fill 3L cup again. Fill 5L cup again with the 3L until it gets filled with water. This will result in you having 1L left in the 3L cup (and 5L in the 5L). Empty the 5L cup. Pour the 1L from the 3L cup to the 5L cup. Finally, fill 3L of water in 3L cup and transfer into the 5L cup.

I think when I first saw these type of questions I was baffled, but once you see an answer, you can easily answer variants of them. I recently played Tomb Raider: The Last Revelation and the final set of puzzles were variants of these and I did them with ease.

Question: In this scenario, you are standing in a room with three light switches. Each corresponds to a different light bulb in another room that you cannot see. If all the light switches are off, how do you find out which one turns on which bulb?

Answer: I will turn on one switch and leave it for a few minutes. When I turn it off, I will quickly turn on another switch. I will go to the room which shows what light turned on from the second switch, then feel the other bulbs for which is warmer (which would be the light I left on for a while). The third switch belongs to the bulb that is off and cold.

This question is nonsense. It never says you can leave the room, and it says “you cannot see” which implies you can never see them from a distance, or get close to them to touch. If you can leave the room, you could turn one on, go and look, then go back and turn on another. I think the question needs to stipulate you can only leave the room once only.

Question: You are driving a bus. There were 10 passengers already on the bus. You pick up 8 people from Stop A, drop 12 at Stop B, pick up 5 from Stop C and drop 9 at Stop D. Then, what is the age of the driver of the bus?

Answer: Your own age.

Question: You have to enter one of the three rooms. One room is on fire, another is occupied by a man-eating lion who is hungry for three months, and the last one is occupied by a terrorist with a fully loaded AK47.

Answer: The 2nd room because the lion is dead because of hunger. (Hungry for 3 months)

I’m not a fan of a question like this because wording it as “is hungry for three months” implies it is still alive and hungry. It needs to say “hasn’t consumed any food for three months”. I think it is intentionally been written like that to reduce the likelyhood of coming up with the answer. One of the aims of the test was to examine people’s understanding of English, and throwing in questions that are ambiguous, misleading or containing poor grammar isn’t the way forward.

Recruiting Graduates #1: Discussing how to make a good test

Colin arranged a meeting with a few Senior Developers (including me) and a Senior Tester. He stated that we are going to do a big recruitment drive, specifically targeting Graduates in India.

He said he is disappointed with the current Indian staff in general, since many seem low skilled and poor communicators, so he wants us to come up with:

  • Multiple choice logic questions for a timed test
  • Multiple choice programming questions for a timed test
  • Verbal logic questions for an interview
  • Verbal programming questions for an interview

So we are testing them on both written, and verbal comprehension as well as programming. Fine plan so far.

In the next meeting, a few of the Seniors had come up with some questions which they said they got from some websites. I had spent a few hours scouring the internet for ideas but I had found many websites with poorly worded questions, ambiguous questions, or material that I just thought was irrelevant to the job (either not C#/objected oriented/not graduate level). So I only submitted a few questions but I needed to confirm what our aim was…

I noticed many of the suggested questions were specifically programming questions on C# which is what the job is for, but do these graduates we are targeting actually know C#? I ask the question and the Indian seniors say that these days, university courses are mainly Java and Python. So I tell them the test they have come up with isn’t suitable. For multiple choice, the questions have to be just logical questions using pseudocode. For the interview, it needs to be phrased like “using a language of your choice” and we would have to allow them to use Java, Python or anything else.

I also found that they had many questions that were wrong, ambiguous, or irrelevant. I don’t think they used the same bad websites I found, but there’s some terrible questions out there for sure. I think it says something about how bad our staff are when they are meant to be Senior and they are taking questions from online and not really thinking about if they can answer them or not. We are going for Graduates as well. Really the hierarchy would go Graduate/Junior->Developer->Senior. So surely the questions have to be easy for Graduates to do.

“Why are people who don’t know what they’re doing, setting these questions?”

Colleague

“so we can recruit more people that don’t know what they are doing”

Me

Another problem was that we scraped about 50 questions of questionable quality, then Colin says: “I want 60 questions for them to do in 60 minutes…also we need a second set of 60 questions just in case someone leaks the answers to other applicants”.

Everyone seemed fine with this, but I pointed out that 60 questions were a struggle to come up with, and I didn’t think it was possible to answer them in 60 minutes – I said 45 minutes is better. 60 minutes sounds like pure tedium to me. The 3 other Seniors and Colin were all adamant that “60 questions in 60 minutes” was standard in the industry, and a couple of Seniors said they did tests like this in recent years and it was fine.

When we eventually came up with a test, Colin says

“I tried the test over the weekend, I do not think we can do the test in 60 minutes”.

Colin

Wow, I was correct, and was shot down 4 to 1 votes.

He wouldn’t specify what score he got but was insistent that he passed despite not answering all the questions.

I am never a fan of doing programming questions in an interview situation (unless it involves using a computer). Sometimes companies give you pen and paper, (or a whiteboard), and you have to write code. It’s unnatural and very awkward. Also, they come out with cliché problems like sorting algorithms, or data structures. Sometimes you can blag your way through these scenarios if you practice them beforehand. I prefer a take-home-test to build a small application in a few days. I’m conscious that this can be a lot of work especially if you are applying to other jobs at the same time. An alternative could be to show a program you have previously written and do a presentation on it to demonstrate your understanding and talk through the design decisions.

We already had a take-home-test from previous recruitment, so I suggested using that. The Indian Seniors said this would be far too hard especially if it involved calling a Web API. I said the candidates don’t even have to complete it (fully working app with many features), you just want to see how far they get and see how they structure their code and how they break-down the problem. If you just tell them to write one method like a sorting algorithm, then you don’t really get to see how they structure code because it’s minimal, isolated code.

The Indians even suggested a compromise of giving a choice of programs to write, but to give extra marks if you choose the first example. It didn’t seem that fair to me and there didn’t seem any noticeable increase in difficulty. One suggestion was even to get HR to call them and ask them on the phone to decide which test they want. How would that call even go?

HR: "Are you familiar with Web APIs?"
Candidate: "no"
HR: "do you want to do a banking app with a database?"
Candidate: "no"
HR: "you have failed to get the job"

or

HR: "Which test do you want to do? one with a Web API, or a Banking Application with a database? You get more marks for the Web API"
Candidate: "erm…Web API then."

Eventually, they relented and agreed just to ask for a specific program and we would see how far they got. Next up was to decide how much time they actually got. I suggested giving them up to 10 days. One Indian Senior said 10 days was nowhere near enough time given the difficulty of the program. Another Indian Senior (who had previously stated this test was far too hard), then claimed he had a similar test that he successfully completed in 3 hours. I was like “wut“. How can he go from saying it was far too hard, to then saying it is easy and he can do it in 3 hours! He then suggested we should make them create and interact with a database which I think makes it way harder! I challenged him on this and he said he has never seen a university syllabus that has interacted with an API (which my suggested test required)… Despite him showing me a university that used ASP.net which is all about web apps and web services.

I did feel bad criticising their decisions when they had put some effort in, while I did think about the situation but didn’t have any good questions to ask. However, I didn’t think they had put much thought into the problem they were trying to solve.

Some lessons learned:

  • You need to decide what level of experience you want to aim for so you can set relevant questions e.g. Graduate, Junior, Senior, Expert.
  • You need to know what kind of programming languages your applicants will be using. If they are from a Java background but your job is C#, you may have to set the test in Java, but make it clear the language of the job is C#
  • Make sure the questions make sense, aren’t ambiguous, and the answers are actually correct (this will be discussed further in an upcoming blog)

Mentoring #7

I am mentoring an Apprentice who has never done C# before, and this is his first programming job. So this is a diary of some-sort of his progression and my ability to mentor.

Managers love rearranging teams and sometimes I facepalm at how ridiculous it gets. When our teams were first formed, I was in Team A and my Apprentice was in Team B. I flagged this to my manager as a strange choice because it would make more sense if I was working alongside him and knew the work that he was assigned to do.

After several months, I got reassigned to Team B. A few months later, he has been reassigned to Team A but will work as a Software Tester. So now we are in opposite teams once more.

All this switching is just stupid, but I don’t object to him switching roles. Recently, he did say he understands C# a lot more now and can read more code that he could before, but he struggles to understand what our software currently does.

The annoying thing is, I did suggest to him early on that; temporarily, (or intermittently) switching role to a Software Tester may be beneficial to him. Testing the software will mean he learns the functionality of what it is supposed to do, he sees more of our process, and also understands how the Test Environments are deployed/configured etc. Our software is really complex so I felt his insistence to get stuck into the code was actually hindering him.

He actually sounded quite excited to try it now, and I felt that if he does enjoy it, it might suit his abilities more. If he doesn’t do well, then we may end up letting him go.

This is the thing, he has been here a few years and hasn’t really learnt anything, and when he does ask basic questions, I find myself telling him I had already gone through all this stuff before. I don’t expect him to retain everything I tell him, but I did emphasise there was a lot to learn, and this team is incredibly difficult so needs to put the effort in early on – which he didn’t do. Now Colin (who I have written about many times on the blog) has moved roles from Developer to Manager, Colin told me he is aware that there are several underperforming people, and that he is going to be looking to move them on; my apprentice is one of them.

Colin seemed to suggest that if he can’t complete work himself (either from a Development or Testing point of view), then he could be let go. A week later, my apprentice was asking questions from a Developer perspective and I asked what happened to the Testing work. He said something about “not wanting to be a Tester” and “didn’t want to get stuck doing that work”.

Not a good idea.

Showing he can do Testing would be much easier than trying to fix some difficult Bugs. Going against a manager’s idea when you are close to being let go is also playing with fire. Also, what happened to the enthusiasm he showed a week prior?

I’d also previously explained to him how there’s always times where Developers have to chip in and help Testers. Those that kick off a fuss or don’t contribute enough are always looked down upon since they are not “team players” and don’t show a “quality” mindset which everyone should show.

We will have to see what happens. I’ve only ever known people to be sacked for having major arguments with managers, or doing something really offensive. It’s a rare occurrence so I’d imagine we would just keep him around, even if he doesn’t contribute much.

Global Employee of the Year

Colin was instrumental

Director

Colin was a developer I originally wrote loads of stories about after he got unfairly promoted to Senior Developer. There was more injustice a few years later, when he found himself promoted to Principal Developer. Weirdly, his work best-friend always got promoted at the same time, or close to it, so he eventually found himself as a Technical Architect even though he is the shyest person that I know at work, and he has low productivity.

I’m not sure if Colin has some dirt on the Development Managers or the Directors but he seems to have so much influence now. My close colleagues also think it is farcical but laugh it off when I say he must have some dirt on the senior management. But how else can you explain these nonsensical decisions?

Colin had recently proactively tried to address issues that occur for users, and the CTO was impressed with him so gave him an interview at the monthly update presentation. Recently, he changed role from Principal Developer to Technical Manager, which I think is more a sideways move than a promotion but it’s interesting how he seems to get what he wants.

Near the end of last year, the Directors announced they would be hosting an Employee Awards event to celebrate people, or team’s achievements.

I’ve never been happy with these in the past because awards usually go to people whose managers are enthusiastic about such awards, or are charismatic enough to persuade the awards panel that their line-report/team deserves something. Or, it goes to a team that was assigned a project that was worth loads of money, regardless if it was actually a great success.

Anyway, our team (well, really it is a group of teams) got one of the awards even though I don’t think we have done anything particularly great, and many of the items Colin has been involved in have led to a few Major Incidents.

Furthermore, Colin got Global Employee of the Year.

Employee of the year!

Global!

HUGE impact on our customers through his amazing work following the issues we had earlier in the year, Colin was instrumental in most of the changes that have since been applied, and made such a difference. He led the team to ensure that everyone was able to contribute and fully felt supported and Colin’s high standards in “collaboration” demonstrate the best in our corporate values.

Director

So in multiple offices in the UK, India, and all those smaller companies we seem to buy out and then we don’t hear much from them – Colin is deemed to have contributed the best work last year. So he gets a promotion, a team award, and the highest-rated individual award.

Absolutely ridiculous. I think I have written this many times in the blog, but if I was someone reading it, and was familiar with previous blog entries about Colin – I certainly would not believe this was a real thing that happened.

I had a call with Colin and he said he has been really stressed this week. There were 3 Major Incidents, 4 people across the teams he is managing have resigned, and he has had to sack his work best-friend Technical Architect (no idea why this happened – it is rare that we sack people but it will be a blog entry if I find out).

You would think morale would be high if a team (or group of teams) were performing well enough to win an award. Yet we are 5 people down across these teams, another is on long-term illness, and I don’t think we were producing much when we were fully staffed. Colin sure has a job to do here.

Software Performance Tales

Looking through my draft blogs, I have quite a few to do with performance, so I have grouped them together to make a complete blog entry.

“the whirling circle thing is just whirling”

User annoyed at the slowness of our system

Do we care about performance?

I’ve listened to some Tech podcasts that have discussed Big O notation which is a way of classifying the performance of an algorithm such as O(n), O(n^2), but outside of studying Computer Science at university, I personally have never heard anyone else reference this. I think with some programming jobs, the efficiency of algorithms is crucial. For the most part, I don’t think people care unless you are doing something that turns out incredibly slow. In the age of Cloud Computing where you can be charged by the millisecond, then it will become increasingly important for server-side algorithms.

Where I work, we are very reactionary rather than proactive. Recently, we have had loads of complaints about our performance, so then we have identified the slowest areas and addressed them.

Thinking about performance upfront would mean the user gets the best experience and doesn’t complain. However, there’s plenty of examples of code which runs fine on your machine but runs poorly in live. This can be because you have a faster computer, the network speeds are faster, lower latency between the servers (when developing, the client and server is on the same computer so there is no latency), or you don’t have a database which represents live – you have 100s of rows rather than 100,000s. I think this last reason is often a cause for us, in addition to concurrency (thousands of live users simultaneously using the system, compared to a few testers using a test server).

An example of how we don’t often consider performance is as follows:

Example 1

I showed some initiative recently and essentially rewrote a project that was inefficient. 

  1. Some server calls had a check to see if a feature was enabled before attempting to retrieve data from the database, but some didn’t. Therefore these code paths ran queries against the database before returning no data. 
  2. When the feature was enabled, there were other parts of the code that were making the same server call twice in the same sequence of actions,
  3. and other places that made a server call even though the client already had the data from a previous call. 

It was a mess basically.

The functionality behaves correctly and the performance didn’t seem out of the ordinary, so the Software Testers never flagged it. We were probably talking an extra 50ms to 200ms in total when testing internally, although this would be worse when deployed live. It should have been obvious to the Development team that there were inefficiencies if they were thinking about possible improvements to their codebase, or if they were paying attention to the server calls and noticed their new server calls when the feature was switched off. 

Volume and Performance Testing

We have loads of test environments – virtual machines with databases and servers where the Software Testers can test our code changes. I would have thought these would be fine to do performance testing, and spam these test servers with calls, but a Test Environment Engineer once sent an angry email:

Our standard test environments are Absolutely not to be used for Volume and Performance testing. This can impact the whole devtest infrastructure, as well as some live services that share the same infrastructure. Doing this could cause a Major Incident.

Test Environment Engineer

This seems bizarre to me. Why would you share resources with the live production environment? Surely the Test Environment should be in its own isolated sandbox, where you can create, update, and delete with the safety and freedom. 

Example 2

We had another performance issue where a large amount of data was attempted to be retrieved. When the developer analysed it, we weren’t actually trying to retrieve a large amount of data afterall.

“There were 142,858 Id’s passed into the comma separated string, I found that there were only 64 distinct ID’s once I removed the duplicates. After the duplicates were removed it went from 76minutes duration to under 1 second”.

Software Developer

What a performance improvement! 

I wonder if passing the data in a form that isn’t a comma-separated string would also help. If you are passing in “12, 156, 201, 202, 284”, then, to get the ID’s, you need to remove the commas and place the numbers in a table. If you just pass it in as a table, then you are cutting out a step on the database layer.

Example 3: SQLite

One part of our system used a local file cache (loads of files were downloaded to the users machine, then our software would just load up these xml files rather than going to the server every time). It has worked fairly well, but overtime, the number of files has grown.

I think the performance issue comes from the fact that all the files are then loaded and are kept in memory, so it is a memory hog, and sometimes leading to “Out Of Memory” errors. Maybe the solution is to work out if we can dynamically load these files in full only when they are needed, while only having the “metadata” in memory at all times.

Colin came up with an idea to move the files into a local database using SQLite. This innovative idea impressed the managers and were eager to do it. I was sceptical because I think we would still end up loading all the files, just the place where they are stored has changed.

Day 1 of release, the following Major Incident was raised:

“SQLite corruption. Out of memory”

After that was fixed, I believe the changes caused 3 additional Major Incidents. Colin still swears by it. He thinks it was an amazing idea. I ain’t convinced.

User Feedback

It’s rare that I come across comments from users about our software, but they do often make me laugh. Part of the problem is that they use software, but aren’t really technical enough to use the correct terms, so when complaining about a performance issue, one user remarked:

“the whirling circle thing is just whirling”

User annoyed at the slowness of our system

Another thing about user complaints is that I tend to only hear about the real angry complaints and not just generic comments. I think Support had told a user there was a large amount of data on that particular record, and the more data on the record – the longer it is going to take to load.

“Regarding larger records, this is a predictable problem. What mitigations are being put in place to offset that? I still have 20 years of working life yet, don’t tell me that by 2035 I will have to wait 20 minutes to load a record???!”

Angry User

It’s a valid complaint. We need to come up with a different way of loading the record so they aren’t sitting around waiting for the full record to load.

Promotion Special

One reason I thought writing a blog would be good is that it would document my software engineering story to become a Senior Developer. That was always my ambition when starting out as a Junior Developer – to become good enough to be recognised with a fancy title.

An industry standard for Senior is probably 5 years experience, but that’s just a rough guide. I mean, 5 years of consistently being bad shouldn’t get you promoted. I’ve written about Performance Reviews in the past, and stated how the process never seems to work, or not for me anyway. So I think it’s been more like 6.5 for me.

Recently, I reflected on my promotion from Junior Developer to Developer in the blog Comparing to Others, and that was to build up to this one. Obviously you can tell from the blog title and the words that I have written so far – I have been promoted. Senior Developer. So let’s reminisce and discuss the movements over the last year or so

A few years ago, I was working on our upcoming software, but I was asked to return to the previous software I worked on, with a strong hint I would be promoted. In my performance review, my manager Chris then made some excuses why I couldn’t be promoted and hinted it should happen in January 2021. But in January 2021, Colin got promoted and I didn’t. He moved from Senior to Principal, so there should have been a free role available. My team members think I am a far superior developer to Colin, so how does it make sense that he is now 2 ranks above me?

My team members started being annoyed. I had people say to me they were frustrated I wasn’t being respected. Even Colin said he had been talking to the Head Of Development about how insane it is that I hadn’t been promoted.

everyone “technical” thinks you should be a Senior

Colleague Developer

and all the Spanish think I am a Señor

Me

I changed managers and my new manager said he would look into my claim that I was underpaid and would try to find out how to come up with a solid plan to also get me promoted. However, I had a performance review with him, and in the review, he read out the feedback my previous boss gave me, although Chris’ notes were vague. So he asked me to explain the “unprofessional comments”. I explained that I have a running-joke where I state “the project is in ruins” when theres a minor change to the scope. It’s an exaggeration and purely in jest, although someone in the team has raised a concern about it. I was really grateful for his response because he laughed and said the person that complained was wrong to do so. However the reason why I panicked when he first brought it up was – I remembered that I responded to Chris’ question on Slack a few days prior with the text “lololololol”. So I then realised that I hadn’t taken Chris’ comments on board whatsoever.

Months went by, and my new manager assured me he was trying his best to promote me but was just getting pushed back by managers. Eventually, after another team restructure, I changed managers again.

In my first one-to-one, my manager asked if there were any concerns I had with the company. I stated that I was underpaid and was getting overlooked on promotions so dropped hints that I would be looking to move on. A week later, he calls me and said he was “absolutely shocked” at how low paid I was so would be taking it to HR urgently because we can’t afford to lose great developers like me. He managed to get a £5k payrise although HR still insisted to underpay me for 2 months before they actually gave me it.

Another month went by and he confirmed I would be promoted. I stated in the past, HR don’t allow big payrises so would probably say I’d already had one and reject the request. He assured me he wouldn’t stand for that. He kept his word, although yet again, there’s a compromise. I get a further £7k but have to wait 6 months until they approve it. HR are just the worst. You would think they would know how to make staff happy. That’s their job, yet they treat people like numbers and not humans.

The good news is that I finally get a good salary and it matches the respect I get from colleagues. It’s not even a change in responsibilities either, so I got £12k extra for doing exactly the same job as before. I probably should have been on this wage/role at least a year beforehand though. Don’t get me wrong, I was grateful for my previous wage. I always remember the times I was unemployed and have known many friends to struggle on low paid/part-time jobs. It’s just frustrating knowing someone could easily join the company and be on a much higher wage than you, and sometimes you even hear underperforming colleagues boast about their higher wage.

The thing is, in the past, our performance reviews have changed several times and we have tried: filling in a form; interviews; filling in multiple spreadsheets and forms; manager’s opinion only – and you could say none of them have worked out for me. It was only when I essentially hinted at leaving did I get the promotion. It shouldn’t be this way. Great people can leave, and the most confrontational get their way. Surely, that is the worst system.

I think I can easily be happy with my job title and wage now. I shouldn’t be complacent because you never know what the future will bring. Maybe one day, I could even come up with an idea about how to fairly appraise people, because we obviously have no idea, and I’m pretty sure it’s a common problem in the industry.

Department Update – Interview With Colin

We had a department update where the Head of Development, Head of Programme Management and other senior managers were doing presentations on aspects that are happening in the department.

I was well shocked when the Head of Development said he had a special guest, and the next section is Interview With Colin. He was being interviewed about the performance updates and the important bug fixes his team was doing. 

The thing is, his team has one of the best developers, David, and David had done ~90% of the work. Colin’s work was minimal and the work he had done didn’t work at all, until other developers had helped him.

Yet there he was, taking the credit for the work by doing this important-looking interview. The annoying thing is, when the Head of Development asked if he wanted to give any “shout-outs” he named several people, but didn’t name David.

There was even one improvement that was hyped up, but that was the first time I had heard about it. So I asked David when that was implemented. He said it was done by a different team, but Colin was basically claiming credit for leading it.

I don’t know how Colin has wormed his way to such status, but he seems to be held in high-regard by management.

Terrible Colin Logic

The developer job titles are essentially: Junior, Developer, Senior, Principal, then I guess you have Architect which could be seen as a rank above that. Colin is a Principal. In this blog, I document his struggles with simple Junior-level concepts. Using simple AND and OR logic is a prerequisite to being a developer in the first place.

Let’s say you have 4 possibilities which we will label with the letters A, B, C, D and we have some code that checks a variable is one of these types (or not).

He wanted to write some logic that says “Not A or B”. In the programming language C#, “||” is the logical OR, and “!” is NOT which inverts it (so “true” goes to “false”, “false” goes to “true”). So we can denote “Not A or B” as !(A||B). Here it is as a “Truth Table” of how it should work.

Item is (A||B) “A Or B” !(A||B) “Not A or B”
A truefalse
Btrue false
Cfalsetrue
Dfalsetrue
“Not A or B”: !(A||B)

but instead, Colin wrote !A || !B (“Not A Or Not B”).

Item is!A “Not A”!B “Not B”!A || !B “Not A Or Not B”
Afalsetruetrue
Btruefalsetrue
Ctrue true true
Dtrue true true
“Not A Or Not B”: !A || !B

So A returns true because it is not B. B returns true because it’s not A. C returns true because it is not A. D returns true because it is not A. So the statement is always true.

A developer points out that it’s nonsense so he has another go. His new method was

A || !B || C || !D

“A Or Not B Or C or Not D”. It’s getting really confusing now, but also returns true for everything.

Item is A!BC!D A || !B || C || !D
Atruetrue false true true
Bfalsefalse false true true
Cfalse true truetrue true
Dfalse true falsefalsetrue
A || !B || C || !D

I couldn’t believe he has come up with more nonsense and still not tested it. I didn’t want to tell him exactly because he should be able to tell that he has done it wrong if he glances at that logic, so I left a simple comment along the lines of “this won’t work”. 

Later, after I got back from lunch, I saw that I had a missed call. So I send him a message “If you are calling about my comment, just test your method”.

He calls me back and starts explaining the requirement. I said that I don’t really need to know, I just know the method wasn’t what he wanted. I ask him to look at the method again. It took him 30 seconds or so before he spotted it.

Colin: “Why did I type the exclamation mark? I must have copied and pasted it. I’ll correct this line and you can review it”.

Me: “Aren’t you going to test it?”

Colin: “No, this will work this time”

Me: “Well, you thought the last two times would work. I recommend you look at the logic a bit longer if you’re not going to test it”.

He looks at it for 10 seconds.

Colin: “Oh there’s another exclamation mark. I’ll change that too”.

He got it right with a bit of supervision, but I don’t think he had much intention of testing it.

Increasing Customer Satisfaction

There’s been a big push to increase customer satisfaction, and the Head Of Development is actively monitoring the biggest issues seen by our users.

Colin has been looking at the most frequent errors and trying to proactively fix them. One “frequently occurring” error is from a group of users that need to log into multiple organisations. 

For this to work, you have to have both client versions installed and use a launcher to log into the correct version. However, instead of using this tool, they are just using the same client but logging into an organisation assigned to the other version. The version difference between client and server causes crashes when using certain features.

Our software wasn’t initially designed for this since 95% of users just log in under 1 “organisation”, so support for this requirement has been retroactively added (the launcher), but it isn’t perfect at the moment. 

There’s a team working on better support. Until then, we should instruct users how to use our software correctly, then release an update to stop this from happening.

So the “frequently occurring” error is that these users are not using the launcher, then encountering crashes when they try to use certain features. However Colin aims to “fix it” by adding some defensive code. All that will happen is one particular server call will now “succeed”, but there will be loads of other server calls that will have the problem.

If the user now sees a particular feature is now supported, but another feature isn’t, then surely they may ring Support and ask why it isn’t supported. Support will tell them they aren’t using the software correctly, but the user will say that we should support it because we have addressed similar issues before.

I discussed my concerns with Colin and he eventually agreed to cancel his changes. 

Really, the endgame for Colin isn’t to please customers, it is to please the Head of Development who is aware of these errors. If he sees that the errors aren’t happening anymore, then he will be pleased. We need to make sure we are actually addressing the root causes of the issues, rather than hiding the problems.