Tag: Process

Merge Ready Checklist

We have a small team of people that are known as Tech Quality Assurance. They are comprised of a group of experienced ex-Test Managers, although I often think they can’t have learned much in their experience due to some of the ideas they come up with or how unaligned they are with their communication.

A few years back, there was a project I was working on which was near completion and I got asked to fill in a “Merge Ready Checklist“. I was like “errr, what’s this?“

If the QA team are going to introduce a new process, shouldn’t this be communicated when it was created?

I looked through it, and most of the evidence they wanted were aspects that you needed to be aware of up front, so you could track them, and gather evidence as the project progresses. When I saw it, I thought it wouldn’t be possible to be allowed to merge. Even some of the simple stuff like:

“Branches are expected to be kept in sync with root development branch on weekly basis. Provide evidence of the frequency you were merging from Main and the date of the most recent merge.”

process on branch-merging

I merged when I wanted, so what are they gonna do? reject it because I didn’t merge enough? The branch history is the evidence, why is it worded as “provide evidence“. Why not just ask for the link to the code branch?

They then insist on having Test Coverage of 80% which I always think is an unreasonable ask. When I joined Development, we had Visual Studio Enterprise licences which have a Test Coverage tool available. However we have since downgraded to Professional. So I ask what Test Coverage tools we can use because it needs to be something that IT have approved to download, and that we have a licence for. We were told we could use AxoCover, but I found it wasn’t compatible with Visual Studio 2019 or above, which was an inconvenience.

Ok, so let’s run it and see what happens. Firstly, you are greeted with a phallic symbol.

Test execution started.
         ___
        /    \
        |     |
        \    /
        /    \
       /      \
___/        \___
/                      \
|     _______     |
\__ _/        \___ /
AxoCover Test Runner Console

It’s supposed to look like their logo, which looks more like a fidget-spinner.

Then I can’t even make sense of the statistics? Is that saying that I have removed methods, and deleted tests? I have added a few classes with several methods each (these obviously contain lines of code and conditional statements so I expect all values to be higher. I had also added some Unit Tests but maybe would have expected 30% on new code.

I asked Tech QA to explain the figures to me, and they were like “we dunno, we aren’t developers. We just look for the 80% number“. Then I point out that they were supposed to be judging the 80% coverage on NEW CODE only. This is for the entire solution file. So this doesn’t give them the evidence they want, and it’s not accurate either and cannot be trusted.

After running it several times and adding/removing code to see how the numbers changed, I then was suddenly low on disc space. Turns out Axo Cover reports are 252MB each! Yikes.

They also wanted you to run the static analysis tool Sonar, but with the licence we paid for, we could only run it on our Main branch. So we need to merge our project in to run Sonar, but they want the Sonar results to authorise if we can merge it in. The classic chicken and egg scenario. Later on, we did get better licences to run Sonar on our project branches.

When we submitted our Merge Ready Checklist to the best of our abilities, we got the following feedback:

“TechQA have reviewed the MRC and assessed it with a critical level of risk. There are a number of items that we would deem vital to address however even resolving these items – it would leave the project at critical.”

We failed the Merge Ready Checklist

Surely that is the biggest rejection. Even resolving the issues won’t change their judgement.

We arranged a meeting with them to discuss the way forward. At one point they said:

“These are all things that are too late to address but are common themes we raise time and time again and significantly reduce confidence in the quality of a release.”

TechQA

Surely that’s a problem with their communication. So the process is just sprung on teams when they want to merge in, then the problems aren’t even fed back into the system/communicated to other teams so they can’t learn from it, and then they proceed to fail.

I then asked the release manager what we can do, and he said that the TechQA team are there just to advise him what should and shouldn’t go in the release. Due to contractual deadlines, he just lets projects go in anyway as long as the team explains what bugs they are aware of. All the other aspects like Sonar and Test Coverage don’t bother him at all because it is more “Technical Debt” and not really that reflective of the user’s experience.

So what we are concluding is that the TechQA team are completely pointless and we may as well save money by binning them off, because they are just creating processes that no one cares about, and aren’t enforcing anyway.

It looked like they were also doing these for another company we acquired. I found this on another team’s review:

It should come as no surprise that the assessment of the MRC comes out as critical.

  1. The “Definition of Done” specified in the document is not the one followed by the team as you point out that no testing was part of the user story and a Product Owner was not attached to the project towards the end and therefore could not do the PO review.
  2. Also, no unit tests exist for the code and there is no ability to confirm the changes have not degraded any quality metrics.
  3. Elements of the regression plan have not been run and once again the team have not been given sufficient time to consider quality.
  4. On top of all that the code is written in Visual Foxpro which was EOL’d by Microsoft in Jan 2010, that’s 11.5 years out of support!

An example for the other week saw the lead developer give a lot of backchat to TechQA.

"Link is to a PR not to a build - where do I go to see the state of the build"
 > All PR's into master need a passing build. If you would like a hand using Azure DevOps please let me know and I'll see if can find time to show you how it works. 
 
"there must be some sort of output that can be used as evidence surely"
 > Once you have looked at the build, you can also view the 35k passing unit tests. 
 
"Need some evidence of your test coverage levels in the form of a report or chart from the text coverage tooling you are using"
 > You can see the tests written in the PR to master. Each PR for the new work was accompanied by tests.
 > Coverage Report attached for the impacted code, although code coverage is a useless metric.
 
 "Sonar - Please provide evidence when you have it available"
 > If the work is ever completed, then we will have automatic sonar reports for projects.

No idea why he doesn’t have Sonar running, but he obviously doesn’t care enough, and definitely doesn’t care about Test Coverage. I do find it strange that we have been using Azure DevOps for years now, and TechQA still don’t know how our development process works – and you would think it would be a prerequisite for them to do their work.

They should be liaising with the Development team in order to create processes that are feasible, useful, and accurate.

Timesheets

Government Tax Credits

Many years ago, managers asked us to start filling in a Timesheet. They were adamant it wasn’t used to actually track what we were doing, and accepted we may inflate the actual hours to fill the working day e.g. we might have been chilling for 2 hours but then claim we did the full 7.5 hours on a project.

They implied it was to help them claim for some “UK Government Tax Credits” which was some kind of tax rebate for companies involved in “innovation“.

The thing is, I think 95% of our work is contractual so we have been given some requirements by our main customer, and we implement it. So it’s not driven by our own creativity in the hope that people want it. It is driven by the customer who already announced they will pay. Therefore, I think our claims were probably a tax scam. That’s my opinion though, and based on no legal understanding.

One day, I got an email from auditors KPMG. I wasn’t going to lie, so I told them it was simply making a few changes to allow the feature to work for a different country. Also, it got cancelled.

KPMG Audit

I am from the audit team at KPMG and have been given your contact details by the group finance team and have a couple of questions regarding work on a specific project in the year.

Group finance have recorded that you have spent 1/2 of your time in 2018 on the Scotland project. Could you please confirm this through answering the following questions:

1)  Please can you describe the nature of the project we have selected for testing? (i.e. is this project to do with maintaining existing software, or enhancing/building new software)

2)  What’s the status of the overall project? (i.e. how far from completion? / have any issues been encountered which you are struggling to resolve?)

3)  Does the attached percentage of your time appear an appropriate representation of the proportion of time you spent working on this project at this time?

4)  How do you record your time, and do you know how the percentages we’ve outlined to you are derived?

If you could send your response directly to me by the end of the day that would be great.

KPMG

I wonder what they did with my response telling them the project got cancelled. I also think the time spent wasn’t correct. In different years, I have known other people get contacted with wildly inaccurate times as well, so I have no idea how the times we enter in the timesheet somehow become completely different figures when reported by the Finance department.

New Timesheet Meeting

It was my understanding that the entire business (or at least anyone involved in software) filled in their timesheets. But then more recently, we were all asked to join a meeting about timesheets, and I was surprised at how many people were protesting about this “new policy“. Even people working on the same software as me said they hadn’t had to fill it in. So I had been filling it in for years after being told it was mandatory, but then it seems it wasn’t – depending on which team you were in.

There were quite a few companies we had acquired over the years which also hadn’t been asked to follow this process.

This is what we were told:

Purpose & importance of timesheets

  • Resource plans only forecast the allocation of people and time. Timesheets provide actual and factual data which provide the following main areas of value to the company.
  • The ability to improve our scheduling — we can use this data to better plan in the work we have for the department at a team and individual level.
  • The ability to improve our forecasting — by reviewing Forecast vs. Actual time we become more knowledgeable and therefore accurate in our forecasting process, this improves our understanding of what and for how long our teams need on various items of work.
  • The ability to understand what things cost — by understanding how much time is spent we can get a generalised view of how much the work we do costs. This is important for both budgeting and recognition of capitalised revenue – It also helps us know what to charge our customers where appropriate.
  • The ability to understand if we are efficient — by understanding how much actual time is spent on various activities versus what was allocated we can get a view on what disrupts our plans, by how much and who is impacted. We can then use this data to mitigate this.
  • Person specific salary information is not used to calculate cost; finance use a generalised role-based approach and the SFIA framework.

Discussion

We were told that “Contractors are not applicable for Timesheets“. Why? Surely this goes against all the reasons why they want to do timesheets, to estimate the time and money cost of projects and improve forecasting. If outsourcing isn’t logged, it gives a false impression.

Some people were asking questions like “What’s the maximum/minimum hours to be logged per day and per week?“. Is that them admitting they had no idea what their contracted hours were?

In similar fashion, you would get people who work part time asking questions such as: “I don’t work Fridays, do I need to log anything for that day?“

Also, in a similar fashion, some people wanted to admit they didn’t really do their contracted hours, but had to inflate it to overreport it. I suppose if they want to know how many days/weeks projects are taking, then recording how many hours you have been slacking does make sense:

“When I started recording timesheets in this system I was putting in less than my contracted hours, and then got a message saying the timesheets were incomplete. I now record all of my contracted hours. Let me know if this isn’t what you want.“

Then you had people with the opposite problem

Should we accurately record the number of hours worked, or should our timesheet always add up to 40 hours a week regardless of extra hours done?“

Some Tech Lead/Architect-type staff made a good point that since they work across projects and will help others when needed, they won’t have that “project task” on their timesheet because they haven’t been assigned to that project:

“I do a lot of work doing “consultancy” for projects that I’m not formally allocated to, so naturally those projects are not in my “Tasks” list. Am I expected to try and find project codes for each of these (and get allocated to those projects even for a bit of one-off help) or should I just use ‘BAU’ (Business as Usual) for that?”

Another architect said his work actually spanned multiple projects because he was working on something that is a prerequisite for both projects. So would that mean he just halves the time against 2 projects?

In a similar fashion, sometimes problems arise with some legacy system, and the experts in that field will have to investigate or even fix the issue. Does this then just go down as a generic “Out of scope” item? That sounds like something that is important to track.

Some people were moaning that they might already fill in some time against their assigned Bugs (which was their specific team’s decision), but now they have to also report the time into this central system.

I think there’s plenty of aspects that are open to interpretation or people’s preferences. I noticed we have a “Meeting” task, but when there’s a meeting about your project, then shouldn’t that just be logged as Project Work, and then Meetings should be for non-project meetings? Then sometimes we have optional meetings where someone is demoing their project, or another department is announcing some news. Is that a “Meeting” or is that “Out of Scope”, or if it is educational, should it go down as “Personal Development”?

So it seems the simple idea of creating a timesheet has loads of complexity that a lot of people haven’t considered. I suppose the answer comes down to what you actually want to measure and why?  

The manager’s response to the questions about “if the time needs to add up to your contracted hours” – was that “the time should be accurate and it shouldn’t just be ‘adding up to something!’”. But the system doesn’t really support that. There was a time where a teammate’s time was erratic and he was starting at 2pm and sometimes staying late, or sometimes doing extra time on different days. So instead of doing a consistent 7.5 hours a day, his timesheet looked more like 2, 7, 13, 7.5, 9.5 etc, and it got flagged up as being inaccurate.

This was a brilliant quote from a colleague:

“the only accurate timesheets are the ones when I am annual leave”

Someone came up with an interesting idea. Since many people will just slap 7.5 hours against their project across 5 days, and we all know the timesheets aren’t going to be accurate, can’t we just assume if someone is assigned to a project, they automatically get 7.5 hours assigned to it, unless they log tasks on the contrary e.g. out of scope work, sick leave, holiday? 

“No” was the answer because that’s not how the Time Sheet system is designed. It is a good idea though.

There was one guy called Jason who was constantly moaning. I’d seen him on other calls and he seemed quite problematic. I could imagine he is the sort of person that would get sacked at one point for his problematic and negative attitude. The manager leading the call somehow remained patient with him, and came out with this quote which was a great response considering how frustrated she must have felt:

“I can’t work miracles Jason….I am telling you how I can help you right now”

Manager

Sounds like a film-quote.

Delayed reporting of Major Software Incidents

Process Problem

A few years ago, there was a period of time where I ended up being on the calls after a Major Incident (MI) with our software was declared. I was amazed at the clear failings with the process. There were all kinds of various managers, then maybe a few people from Deployment, but then barely anyone from Development. 

Then you’d have these managers just theorising if the Incident was caused by hardware or software, and if it was software – which enhancement/bug fix could have caused the issue?

It was pure guesswork of course, and occasionally they were right, but what is the point theorising when you could just get the people involved that can actually investigate it?

Sometimes I would hear about a Major Incident and then see that it was actually logged days ago. I think a problem is all the process between Support and Development. Support log incidents in their system and talk about “customer reference numbers”, but we need it in our system which is currently Azure DevOps and we talk about Work Item numbers. 

So sometimes there might be a delay even logging it in Support, and then there is a delay until it actually gets transferred over to our system. Often we are on these calls and we ask for a Work Item number so we can read information rather than just waiting for someone to mention it on the call… and then we are told that they haven’t created a Work Item yet.

I remember a time when users complained to Support, then they logged a Bug straight away, and did really good investigation. Then it would come to us with full details. Now, users complain for weeks, it gets escalated up to the Directors, then comes down the Development managers to Colin, then we have barely information to look into it because the users haven’t provided it, or is just “Chinese Whispers” and the information gets lost.

It’s so frustrating because how can you attempt to diagnose anything without all the required information? So the call is pointless, but when you do have the information, then you don’t want to be on a call – you want to go and start looking into it because you want to diagnose it then fix it.

So let’s look at one of the last examples I was involved in…

Example

Colin messages me about a Major Incident.

“I need some investigation…”

Colin

Apparently 179 sites are complaining about this problem and threatening to leave. Despite being complaining since the Friday release, Colin only knows about it Wednesday. Classic. Not the first time this has happened.

So what can I investigate? What is even the problem? All I know is there is a problem with a certain feature, but the thing is, we integrate with several third party API’s and I haven’t been told which one it involves, and I don’t know what the problem the users are seeing is.

I ask Colin to elaborate:

“not sure which provider. Can we look into all of them? So one of them may be broken, but we need to look at them all and hope we find it”

Colin

Wut. You want me to test them all out and hope I notice a problem? But it could involve just one and require specific steps.

When I finally got some information, it was pretty sparse.

“between 2 minutes and 12 minutes for the dialog to pop up” 

the problem

I told them I would have to see it happen, and see if there is anything different the user is doing to make it go from 2 minutes to 12 minutes. It sounds like a network issue, or not our fault.

“we will test it with a happy…well not a happy user, but an ‘engaged’ user”.

Manager’s suggestion of arranging a call with a user to witness the problem

I was then on a call with all kinds of random managers, all chipping in speculative accusations. “It’s a problem with the deployment“, “it’s a problem with the amount of data in their Tasks module“, “it’s a problem with the number of user-created resources“.

Colleague Opinion

I was discussing this with one of my colleagues and he re-iterated my views:

"they red flag anything nowadays - they just wave their flags around
It’s always the case that someone knew several days ago, but now it needs fixing immediately and it’s the first time any developer has heard about it

why can it not go:
Problem logged by customer -> Support discuss with Release Management -> Release Management arrange call with the correct Team Leader -> Prioritises the work with a developer and tester

I don't see how that's difficult

What actually happens:
Problem logged by customer -> sits in a list for a while
Problem management person looks at issue -> sits in list for a while
Someone kicks off and escalates the issue -> Director gets it in the neck -> Release Management notified
Release Management hold a call -> float many theories about what the issue is about
Developer randomly hears about issue -> mistakenly joins call mid-flow -> explains that it's probably related to the work item with the same keywords as the issue in the title
Developer now late with their own work
Release management say they'll prioritise the issue, but asks if you could start working on it -> Developer says no because you are busy working on 8 different things -> everyone in a huff
Team Leader nowhere to be seen -> likely watching the TV"

Apparent Analysis

A manager recently posted an update with facts that I was sceptical of:

“Analysis conducted into 2 years worth of Major Incidents

Some interesting trends have appeared and I think it’s worth discussing:

  • Data shows that most of the MIs are caused by either a 3rd party or tech debt/ops related issues. The number attributed to code change is pretty low.
  • We are lacking ownership around domains for existing products so not obvious which Engineering Manager is accountable.
  • We don’t have enough SQL skills to manage our complex DB
  • We rely upon a small number of people to investigate MIs.
  • MIs should be owned with the right teams.
  • Further investigation required on this and changes to ownership, structure, investment needs to happen.”

Release at Pace

Another interesting claim was made by another manager:

“There’s lots of interesting developments going on at the moment; we’re delivering more than ever before.”

Manager claim

I message the Software Delivery Manager to ask if there is any truth in that, because I thought we were releasing at the slowest pace.

“Who knows? I reckon one of those blaze statements that people just say to make it sound good”

Software Delivery Manager

On a recent Development meeting, one person posed a question to the Head of Development:

“Deploying software at speed is ultimately what we want to do as a business. However, this can come at a cost. We have had more Incidents in Q1 caused by software defects than we did in the whole of last year! How do we ensure the deployed code is at the quality required if we are going to deploy at speed?” 

Staff member’s concern about software quality

If we deliver more changes, I suppose there is more risk of introducing more bugs. If you are rushing, then you are more likely to create Major Incidents. I suppose you can say it is fine if there’s more bugs in general, but all considered low severity.

The Rapid Responders Group

As a response to this, I think Colin came up with the idea of the “Rapid Responders Group”

“We are deploying our software at a pace we have never done before, so we can miss scenarios that could happen in the production environments as we have seen in the last two MIs. If we can get information straightway, we can investigate the issue straightaway. We all have access to the Live environment. With our technical ability, we might be able to see things that other people might not, so this is why I have assembled this team.”

Colin on his new idea

However, it was quickly shot down by one of our architects and never mentioned again.

“With one of the MIs, there was absolutely nothing we could’ve done between the people in this channel to anticipate that as it relates to a combination of live configuration and state that can’t be replicated.

In general the problems we have with the MI process is making changes takes too long due to processes and permissions, noise from the non-experts present (it’s hard to get a word in edgeways with some people), and multiple people bugging those performing the fixes/changes for updates (when it should be one person who fronts the technical team working on sorting it out)”

Architect

“We had first-hand experience of this on the last Major Incident call. We could have fixed it within 30 minutes but was on a call discussing it for hours.”

Colin
Architect:
It's usually the fault of those running the call or lack of confidence in Dev.
I will say when we have enough information to make a fix - that I'm departing the call to concentrate. As you say, it will otherwise go on for hours, with multiple people fronting opinions on the cause of the problem, unless someone identifies it with confidence and takes the initiative.

When an MI occurs, it's always the same group of people, which is often a pretty good mix, but first we need to identify the cause.
 
Usually it takes a long time for the details of the MI to be logged, particularly somewhere Development can all access such as a Work Item. If you miss the start of the call, it's difficult to know what the details of the original issue were: you can guarantee you missed something of importance.
 
I would suggest the following to help the diagnosis run more smoothly:
  1. When a release is made, it must identify a point of contact for each change, preferably the team email address making the change. 
  2. When an MI is started, a brief description must be provided in written form and a source-of-truth record started (such as a Work Item). 
  3. The Release Point Of Contact list is emailed with details of the MI starting.
  4. They might not need to join the call unless there is something they can add, but at least if they feel their input valuable to the diagnosis they can join or reply with details. 
  5. After identifying the team/person working on a fix, they are protected by a single manager. The manager will be responsible for communicating with the techies and the outside world, so they can concentrate. They will update the Work Item and email those who need to know progress, what the fix entails etc. 
  6. Work out a way to expedite changes to live and retrospectively log change requests
Principal Developer:
From the MI's I've been involved in, they mostly run smoothly - the only disruption is (as Mike said) people "sticking their oar in" and having a guess at what the problem might be - sometimes forcefully.

They are not always wrong, but sometimes it can distract from getting to the route of the issue, you need to have the confidence to talk over them and get the information you need, correct misunderstanding, and to keep the discussion on track. A lot of the time the initial presentation from support can be way off the mark, but that is true of most bugs we get due the lack of technical skill in support. Only the developers on the call are going to be able to diagnose a fault as a result of a code change. 

The primary focus of an MI call is to restore service ASAP, exploring workarounds is part of the developers role on the call - can we change a stored proc / setting to improve things now? This helps users and gives development more time to work on a fix. Another role is to assist the wider business (especially those in the Safety and Governance roles) to understand the issue - is it as bad as initially reported, or is it worse!?

Once we know what the problem is, and how we will resolve it, then it is OK to drop off and work on the fix.

Conclusion

I think it is clear that too much time is wasted discussing the problem with the wrong people. Managers need to find the correct group of technical people, give them all the information they need, then give them time without hounding them – in order to actually fix it.

Recruiting Graduates #6: The Disappointment

Last year, I wrote some blogs explaining how the hiring process for Software Developers is bad.

General Summary Of Why The Software Developer Hiring Process Is Bad

When I was involved in trying to come up with something better, I was frustrated because I didn’t have the answers, but I felt everyone else involved didn’t understand my concerns.

I hate the way Tech Interviews and tests currently are, but I think it’s hard to come up with something as a replacement. When researching example questions, I see lots of questions along the lines of “who invented Object Oriented Programming?“, and I think “I don’t care“.

Then there’s loads of aspects that you would just Google if you didn’t know it e.g. memory usage of each type.

Or you may get questions like “what is wrong with this code?“, but if you actually pasted that code in Visual Studio and try and build it, you would see the error. Or in the case it is valid but not advisable; Code Analysis would flag it. Some of these can be trick questions like mathematical logic where the order of execution is ambiguous, but Code Analysis would definitely flag to tell you to add brackets, so it’s silly having these as a question.

Then I hate all the generic Fizzbuzz stuff (check if a number is divisible by 3 or 5, or even both). These are just so cliché that people could just learn from memory anyway.

Also, what do these graduates we are targeting actually know? back in my day, I knew Java but I was terrible and didn’t know how to test properly, or even debug. So there’s no point testing them on that either.

Stupid Questions From The Internet

During our meetings, I asked the team how they were coming up with the interview questions, and they said they were just Googling for C# Interview questions. This is problematic because applicants may have read and learnt from these very sites, but also many sites had terrible questions, which were sometimes incorrect.  

I came across this website:
https://www.includehelp.com/mcq/is-the-use-of-return-statement-necessary-in-every-function-in-csharp.aspx

 95. Is the use of return statement necessary in every function in C#?

 Yes No

 Answer: A) Yes

 Explanation: Yes, the use of return statement is necessary in every function in C#.

I think some programming languages may differentiate between “methods” and “functions” where “functions” always return something. But in C# you just call them “methods” which can either return something, or be defined as “void”. So this is just incorrect.

 99. Which is the first line of a C# program?

 using System;

 using system;

 using Namespace;

 namespace MyApplication

  

 Answer: A) using System;

If you create a blank class, “using System;” is the first line, but A) you could remove it B) the order of using statements matter C) in modern C# you could move it to a “global namespace”. I also asked the question of “who are we targeting?” because everyone was coming up with C# specific questions. My colleagues told me the universities we were targeting were primarily coding using Python! So we were writing the wrong test, or targetting the wrong people.

The Performance Review

In my recent performance review, my manager Colin stated:

“You were given an opportunity to lead on defining our interview process, but have not shown much interest in it. Ultimately someone else led it, creating the interview pack, redefining interview questions and leading live interviews, which I think is a missed opportunity to shine.”

Colin

I argued that: although I didn’t have a huge impact on the end result, it would have been a bigger mess had I not pushed back on some of the questions. Although it seems some of the interviewers still asked questions I said weren’t suitable.

We ended up coming up with a terrible process that some candidates dropped out of, and the ones that went through with the interview struggled and we were left with the decision to gamble based on what little we saw.

The Best Hire

One of the best hires we made was someone I didn’t interview, but I looked at the interviewer’s notes, and he didn’t exactly look like a strong hire. 

  • Implemented the basic application to consume webapi and just printed the values based on some predefined values
  • No unit testcases were written
  • application is simple
  • Inheritance – simple explanation
  • polymorphism – no idea
  • interface – not good explanation
  • purpose of inheritance – No idea
  • Print the numbers and its occurrences – Not completed, but after showing the hints he completed
  • Print the number from 1 to 100 using Recursion – Completed
  • print the numbers based on divisible values – Completed
  • Logical, cut the cakes into 8 pieces with 3 cuts – Completed
  • Candidate lacks on syntactically at some areas, but has some logical solving skills and good attitude on approaching things

When I stated to Colin that it “wasn’t a great batch of hiring”, Colin said out of the people we did interview, we ended up hiring around a third of them – which he said was above the normal hiring rate. I said that it was only above the hiring rate because we hired people that should have failed. Then later, he quoted the numbers “15 interviewed. 4 hired” (more like a quarter then). But there were loads of applicants rejected before the interview stage, and others dropped out along the way.

Intermission: Bantz from Twitter

The Polymorphism question

The Polymorphism question that was asked on some of the interviews was basically “do you know the types of polymorphism?”   

I didn’t even understand the question, and I probably have 10 years of experience.

I was only thinking about the usual polymorphism where you can refer to derived types as the base type:

 IAnimal animal = new Dog();
 animal= new Cat();

To check if I wasn’t going mad, I asked some of my colleagues. Here is how they responded:

“not sure i do, no. Inheritance and Composition, are they right?”

Dean (Senior Developer around 10 years experience)

Like covariance and contravariance? But I don’t think I could put it into words. I guess you have real polymorphism and fake…virtual functions have a virtual function table that map overridden functions from base classes. But a lot of polymorphism is just the compiler being intuitive and going “you’re not crazy” it can be treated as that type. 

Adam (Software Architect)

Types? an interface can be implemented many times. A method can have the same sig but different parameter types etc

Rob (Senior Developer around 20 years experience)

According to https://www.bmc.com/blogs/polymorphism-programming/, there’s Runtime, Compile Time, Overloading, and Casting. So if very experienced developers can’t explain the concept, then why are we using the question to target Graduate developers?

Conclusion

It annoyed me that Colin stated “have not shown much interest in it”. I was trying to make sure we had well-defined objectives on the exact type of developer we were hiring. Everyone else just started copying questions for the internet without considering the knowledge and experience of the people we were targeting. I didn’t have the solution of how to make a good software development hiring process, but I really wanted to make sure we got close – but I just didn’t have any support from everyone else that was involved.

Burndown Chart: tasks

Colin said that our Sprint Burndown chart wasn’t accurately reflecting the work done. He said we were overrunning with each Work Item which meant that we would have 0 points for one Sprint, then we get the full value in the next Sprint – which shows as a sharp drop on the Burndown chart.

I told him that’s how the Burndown charts work – but he said they want more accuracy. I argued further: If the requirements haven’t been met, then it’s not complete, so you haven’t added value to the software – i.e. you have made no progress.

A few days later, our Scrum Master had been in a meeting with him and was instructing us on his new process. Colin’s idea apparently was to add a “task” for each day and link it to the Work Item. At the end of the day, you mark the task as closed. 

I’m like “eeeeeeer wut”. So now it tracks your daily work.

I told her she must have misunderstood. Adding a task per day is just counting the amount of days in the week. I suppose if you take a day off, then you won’t count that day.

I questioned it, and she agreed that it didn’t sound right. So she goes back to Colin. No, he really does want a closed task per day, but also said to create a task even if you are on holiday.

Wut.

:all-the-things:

add tasks to all the things! 

So they want effort more accurately tracked, but are now just counting days, even the ones you haven’t worked. Surely if you create a chart, it’s just gonna be a diagonal line with no fluctuations.

What are we supposed to write for the task’s title and description? “Carried on working on it 7.5 hours”.

I just refused to do it, but the Scrum Master did the admin on my behalf. The idea lasted about a month.

I find that Burndown Charts often look unclear anyway. Here is one from another team:

So what are we even seeing here? The chart to the left shows Tasks, although the chart doesn’t seem to show the correct Completed figure – it shows as 0%. However, you do see the average is 46 which I think is per day – which illustrates the ridiculous number of tasks that teams were creating anyway.

The chart to the right shows User Stories but I think it’s not the number created, but the total points assigned. So one might be worth 1 point, but another could be worth 8 – it depends on how complicated the work is. I think this is a typical Burndown where the first few days nothing is complete because the developers are working on fixes, then the tester will get it in a few days. In the second week, more items are completed. There were even 4 points removed, presumably a change of requirement or maybe it was deemed redundant.

This is another chart that a team posted to boast about their progress. This example is a bit more unclear, but I noticed the Tasks Burndown (start 12th September) is not for the same time period that the Stories Burndown is (start 29th August).

The Stories Burndown looks interesting doesn’t it? It looks like only a small amount of work was done and then when it gets to the end of the second week, they add even more work. I did theorise that maybe they didn’t officially start the project until after 10th September but what does the 72% Completed mean? That seems to imply they are ¾ through their project. ¯\_( ͡° ͜ʖ ͡°)_/¯

Initial Onboarding Guide For Software Developers

Intro

I’m stealing ideas from a former colleague again. He had a blog post about Onboarding, and I felt I had seen many of these mistakes myself so they are worth thinking about if you are a manager and are expecting a new starter. These points are mainly about an office environment, so some points are no longer relevant to us, or are different now we work from home.

Onboarding Checklist

There’s a lot of work done during the recruitment stage; reading CVs, lots of meetings with recruitment teams (in-house or agency), and lots of technical tests and/or face-to-face chats with candidates. Once the offers are accepted, your work doesn’t stop there, but you have some time to prepare until the new-hires start their new roles. Some tasks are:

  1. Find a desk
  2. Find a chair
  3. Order the laptop
  4. Order Monitors
  5. Order the peripherals
  6. Consider collaboration
  7. Grant access to digital tools and codebases
  8. Make sure you are available on their first day

If you do not deliver on the above, the new starter’s first impression of you will be of failure. If you can’t organise equipment, desk, and have a plan to get them started, then what else will be missed? (pay reviews, promotions, training budget, etc)

Onboarding Checklist Discussion

Find a desk

If you are replacing a staff member, then you should have a desk already. However, is it clean?  If the desk has been vacant for some time, it can end up being a dumping ground for miscellaneous items (books, broken equipment, old stationery). Are there any problems with the desk? Does it wobble? Is the key missing for the drawers?

Find a chair

Chairs are more notorious for being damaged or dirty. Make sure all the wheels are still there and the back can be adjusted. The chair should be clean; not stained, or dusty, and free from any other horrors.

Order the laptop

You may be restricted by company procedure, but if there is a choice of laptop, make sure the new employee gets the one they desire. If the developer has a strong preference for a certain system Windows/Mac/Linux, it’s a productivity hit if they get something else. Another thing to consider is the form-factor – If they travel a lot, a smaller size may be much more comfortable for them, especially if they want to partially work on the train. The large screen can be gained from plugging into a large monitor at home or office.

Order Monitors

Many developers like having 2 monitors, although in recent times, it seems some are switching to 1 very large monitor. It’s great to have a combination of your code editor on one screen, then documentation/the software itself on the other.

Order the peripherals

Order the keyboard, mouse/trackpad, headset etc. Even if there are loads spare, they may be partially faulty, or just unhygienic.

Consider collaboration

Teams are full of people – full of personalities. It can be potentially tricky and there’s some nuance to it. Take some time to consider the optimal place for the new starter to sit. Is it the “replacement” spot? Is the desk more isolated? Is there some weird office joke about the desk!?

Who would they pair well with? Is the new starter’s job something someone in the team wanted (missed out on a promotion)? Is there someone who is seen as a bad influence?

Grant access to digital tools and codebases

Sort access to the code, and work management tools (licence keys for all required software).

Make sure you are available on their first day

If office based, be in the office. If distributed, you need to be free for video calls etc.

THIS IS YOUR MANAGER WHO BOB, CAN YOU SHOW WILL TAKE CREDIT FOR ALL THE NEW EMPLOYEE YOUR WORK. AROUND? SURE. (YAN G e 99 workchronicles.com AND THIS IS WHERE YOU'LL THIS IS WHERE YOU'LL WORK LATE NIGHTS ON SPEND HOURS IN EVER CHANGING PRIORITIES. 99 Hello. I make comics about work. Join r/workchronicles or follow on Instagram/Twitter/FB Work Chronicles workchronic/es.com

Bonus: New Starter Packs

When I started, I think all I got was a writing pad to make notes. Maybe I got a pen.

Years later – existing staff, and all new staff were also given a water bottle, and a company-branded hot drinks flask. Even though it’s a simple thing, everyone seemed to appreciate it.

More recently, it was announced that “New Starter Packs” would be posted out to all new starters, and we were asked for feedback on the contents. I don’t get why they didn’t anticipate some backlash because they only planned on giving them to new starters. Also, they clearly had already ordered a certain level of stock because they were taking photos of them and had quite a few company-branded items.

There were the obvious choices of pen, paper, post-it notes, drinks bottle, in addition to some chocolates, sweets and some other miscellaneous items.

“The new starters will be there with their fancy laptops, new starter kit, and high wages, laughing at us”

Me

Some people criticised the choice of unhealthy treats, given some recent schemes to promote healthy lifestyles. There was also a Webcam Cover you could put on your laptop, which someone questioned if our IT department had approved it – to which the Head of IT said they weren’t authorised because they “crack screens and they also damage the bezel of other laptops when removed.”

It was also only for staff based in England, but a high percentage of staff are now based in India, so the managers admitted we need to find a supplier over there too to be fair.

So it wasn’t a well thought out idea really.

Software Development Process

When a new development team is created, managers insist on creating a “Ways of Working” which essentially comes down to describing the software development and testing process in the ideal way.

Then after a few weeks, you end up just resorting to the default way of working.

The “ideal” way assumes that the Testers can provide ideas and inspiration to the Developer, and when the Developer writes the fix, the solution won’t deviate from their original perception. Therefore, all the Test Cases are written up front, and none are needed to be written after.

Here is an example that one of my former teams came up with:

Before fixing the defect

Analyse together 

  • Make sure you pair up as a developer and tester to work on the defect.
  • Review the problem together
  • Understand how to replicate the fault
  • Understand the extent of the problem (all regions, all set-ups, all environments?)
  • Do the initial debugging to understand where the problem lies

Fixing the defect

The developer MUST 

  • Make sure there are unit tests to support the current functionality
  • If there aren’t unit tests then implement a simple refactor which will allow appropriate unit tests to be added
  • Add unit tests that prove the defect (which will fail)
  • Fix the defect
  • Get it code reviewed
  • If you are working on code you are not familiar with, get 2 reviews, one of which must be from an expert
  • Run all the unit tests and prove the fix

The tester MUST 

        • Write test cases with steps and expected results for the initial test cases. They must

                ○ Be clear and unambiguous

                ○ Cover the different scenarios/circumstances/options

                ○ Not test things that are irrelevant

        • Once written, set the state to “Review” and assign to the reviewer

        • The reviewer

                ○ Adds comments as appropriate

                ○ Set the state to “Ready” if Ok, or back to “Design” if it needs changes

                ○ Assign it back the test author

        • Only use Given, When, Then if you are using Specflow to automate your tests

        • Only use exploratory testing

                ○ If you have passed a Rapid Testing course

                ○ And you have a clear understanding of testing heuristics

                ○ And you plan and classify sessions, keep session logs, run debriefs

                ○ And all the specific test cases and all regression test cases have been executed

        • Create regression tests of the functionality

        • Create test case(s) to prove the defect

        • Get the tests reviewed

        • If you are working on functionality you are not familiar with, get 2 reviews, one must be from an expert

Assure the Quality

  • Review the actual changes together
  • Understand the logic
  • Understand the scope
    • Minor/contained (inside a method that’s inside a class that doesn’t affect anything that’s outside it)
    • Medium/impacting (a change that impacts outside it’s own class or is called from elsewhere in the system
    • Major/widespread (a complete refactoring of existing code or large changes to core features)
    • If they don’t understand the logic or can’t establish the scope, escalate to the Technical Manager
  • Execute the tests
  • Attach and link all work items and evidence together

How Software Bugs are Prioritised

A Product Manager recently wrote about how Software Bugs aka Problems are prioritised, so I thought I’d share that here.

Prioritisation Spotlight Report 

Product Managers conduct a weekly meeting with other stakeholders to discuss Problems and their effects on our customers. A key output of this meeting is to ensure that we are prioritising the defects that are causing our customers pain or have the ability to do so. 

These Problems can range from being the result of a major incident, recent software upgrades, or internal database monitoring. However, what they share is that they all have the ability to generate customer dissatisfaction. 

The Product Managers have been ensuring we are able to accurately and consistently apply logic to the Prioritisation process. This is a key requirement of the Problem review that allows us to create the prioritisation for Development to work on. 

How does this work? 

The weekly Problem Prioritisation meeting is open to anyone who has a business interest in resolving these software defects for our customers. When discussing these defects as a group, a number of areas are covered, some of the new key areas are below:

Number of cases linked to Problem – this is multiplied by 2 so a Problem with 4 cases will generate a score of 8 for example 

CSAT (Customer Satisfaction score)– This is the level of market impact the defect has, or is expected to have, and is scored in 4 areas; Critical, High, Medium, Low. 

Software upgrade blocker – Does this hold up the ability to patch customers to a newer version of the software

Safety rating – Does this have safety implications? 

The Prioritisation reason – has the problem been raised as an internal escalation,  safety, Information Governance, Security, Customer pain/Escalation, Service Level Agreement, or an Enhancement via the User Defined Roadmap. 

IG – Does this have Information Governance implications? 

Number of users impacted – taken into account based on how widespread the issue is and how many customers are affected. 

What will this allow us to do? 

This is going to help us all become aligned with the vision across the different areas of the business and will enable key stakeholders access to a single source of truth when scoping these items into the team’s backlog. A “Top 100 Problems” list will be updated after the weekly meetings.

Bug Bash

It’s been a while since we organised a “Bug Bash”. The general idea is to focus on fixing software bugs and clearing the seemingly ever-growing list. Sometimes managers have organised this as an optional thing over a weekend, or sometimes opted for a work week.

The problem I have with this idea is that with a strict deadline, you are encouraging people to rush. If you want to follow the usual process of a developer really understanding the bug (reading the bug report carefully, recreating the issue, analysing the code, coming up with a solution, testing it manually, maybe writing some unit tests for it), then going through the code review and testing process – then you are being really hopeful that it can be done in such a strict time-frame.

I find another developer will always have some opinion on your code, so you make a change, have to retest it and then get them to review it again. Then when coming to patch it on the test environment, you then might get problems with the build server or the test environment.

So the only bugs that are really appropriate for a Bug Bash are ones that are easily recreatable, quick to fix, and low impact. All the true high priority issues will have been addressed more urgently and put into the normal release process anyway.

What you have to bear in mind, is any change has some kind of risk that you might introduce another problem, and this increases when you try and rush the change out; which is what the Bug Bash inherently prioritises.

I was looking for an example of how managers sold the idea, and I found one that actually seemed a bit more thought out:

 
Next week we are to down tools on project work and spend the week stabilising our product.
 
Service have provided a list of 300+ issues they would like us to work through. In addition, Paul has been reviewing the Error Logs to identify the most frequently reported errors and is creating items for these.
 
The domain experts are currently reviewing the list and identifying the issues they feel can be fixed next week. These will be placed into the Stabilisation list.
 
The priority order for working these items is set by a temporary field ‘Bug Fix Priority’. 1 is the highest priority.
 
Each developer should select the next bug at the top of the list that is not currently being worked, change the state to Assigned, and Owner to themselves.

All Priority 1s must be completed first. If there are no defects you are comfortable fixing, please speak to me before moving on.
 
Each bug fixed must have supporting unit test(s) before being sent for code review.
 
At no point should a developer have more than 3 items they are involved with, this being 1 bug being worked on, 1 being code reviewed and 1 with the tester.  If you’re waiting for a code review - go chase it. If you're waiting for a test to complete – go help out.
 
It is important to note the emphasis on this release is QUALITY.
 
With this in mind, the attached spreadsheet has been produced that maps a developer to someone who will review their code and DB changes. Where possible we have also mapped each developer to a test resource. The developer and tester jointly own the end to end process for each bug.
 
A kick off meeting will be held at 10am on Monday in Room 1 for all to attend.
 
Code Reviewers will hold daily stand-ups with their development and test resources which will then feed into a management review at 4 pm.
 
 
Darren
Development Manager

So, they have allocated a full week, tried to analyse the bugs to see if they are suitable to fix quickly, and they have emphasised it is about quality. However, they seem fairly strict on the order the bugs are addressed, so you may end up with developers being forced to pick up items in areas they don’t specialise in.

There was a follow-up email stating that there were issues with the build server which caused them to lose time.

All,
 
The plan is to finish coding tomorrow to allow test to complete on Friday.
 
With this in mind and as a result of build issues experienced yesterday we’re considering the possibility of overtime tomorrow evening to make up lost time.
Can you please indicate using the voting buttons by 12pm tomorrow whether you’d be available for overtime tomorrow evening.

We'll be ordering Pizza and a Beer too, to keep us fuelled 😉 It would be very much appreciated if you could stick around. Every bug fixed is giving our end users a better customer experience!
 
Thanks,
 
Darren

When Bug Bashs have occurred over the weekend, I think Developers and Testers have started at the same time, but they wouldn’t have had anything to test until mid-day at least – due to development time, code review time, then build and patch process. So the Testers will just be earning money for chilling out and eating pizza.

I found an instant messaging conversation with a colleague about a different Bug Bash:

Daniel 11:37:
one of the bugs on the list to be fixed in the bug bash was logged by me 5 years ago
so very urgent then

Me 11:38:
the bug bash isn't supposed to be for urgent issues

Overtime requests will only be approved for time spent on defects that are Fixed, Tested, PO Approved and Closed(Done)
 
So you could do some complex work, not get it done, and therefore not get paid

Daniel 11:42:
yeah I read that today
who's going to go for that
it's completely the wrong way to fix bugs

So this bug bash seemed to be for clearing down all the bugs that have just been ignored for years. The problem with old bugs like that is that you could spend time trying to recreate them, and find they are now obsolete due to other code changes, or obsolete features. The managers also decided they didn’t want to pay people for incomplete items, so you cold spend ages working on something and not get paid.

  • You could close the bug as obsolete – it’s not fixed so you don’t get paid
  • You could spend ages recreating it, coming up with a fix, but there’s not enough time for the rest of the process to actually complete it.
  • You could come up with a fix, but there’s a small bug found by the tester. You could persuade them to keep quiet so you both get paid.
  • You could come up with a fix, but build issues cause it to not get tested.

I think that every-so-often, you do need to analyse the list of bugs you have, and try and clear it down; either by fixing them, proving they are obsolete, or maybe just marking them as never intending to fix. A dedicated period of time is required but I would have thought a 2 week period is required at minimum in order to not cut corners.

Software Support Call Centre

We used to have a very large support team in our own call centre. As a software developer, we were occasionally sent to go talk to them from time to time, and I was amazed at how busy they were. Usually, as soon as they had put the phone down, they had another user call up.

Sometimes it was that the user just didn’t know how to use the system, and other times it was to complain about a software bug or slow performance. The call centre staff were rapid at entering the information into the system, and were brilliant in asking the user the right questions to really understand their problem. They could often tell the user if the issue was logged or not, and also give them some relevant work-arounds. 

After speaking to some of the staff, they explained how strict the culture was there – they were monitored on how fast they picked up the phone, how long the call was, and how many breaks they had. They said how annoying it was to be warned about being late when it was due to bad traffic.

It surprised me because it seemed a completely different culture to how the development department is run. We are flexible when we start; so you can just turn up late and no one cares. We were never challenged on how long our work took to complete. I guess if our work is poor quality, it’s the call centre staff that took the complaints!

At some point, some manager decided to use a 3rd party company for Support, and most of our Support staff either left or (presumably) got redundancy.

The amount of complaints seemed to go up on various social media platforms, and I got the impression this 3rd party company didn’t know our software so were just providing users generic statements from a script “can you try turning it off and on again?”. Maybe if they got past the initial questions, they then got put through to our smaller 2nd-line Support team.

A few years later, I think a new manager came in and decided to try to reverse the decision, but it’s going to take a bit of time to get the new staff as good as the old ones.

“As part of our drive to strengthen our customer satisfaction and experience along with simplifying our ways of working for both our customers and the service desk, the decision has been made to insource the call centre. All calls will now come directly into the service desk.

We have already run a couple of trial switch offs over the last fortnight and the initial feedback has been unanimously positive with customers preferring to be directly connected to the service desk; just in this small sample there has been an increase in both the quality of cases and first-time fixes. We will continue to invest in developing a world class service desk.”

Company Announcement

It seems obvious to me if you make the support more generic, then it’s going to decrease customer satisfaction.

The only problems I had with Support is:

A) when they would link completely different issues to the same bug report. Sometimes you see that a bug that you thought should be super rare – has had 100’s of reports from the users. Then when you look into the cases, you see that 90%+ of them are unrelated. We could have probably put some advice on how to decide if issues are the same root cause or not, in order to try to help remedy this.

B) Sometimes there’s other data entry errors that end up being misleading, like this:

Support (in the free-text description): 
"however we have been able to re-create this on the test system by"...

also support (in the mandatory fields):

"Recreated in-house: No
Reason not recreated: Unable to recreate"