Category: Process

Merge Ready Checklist

We have a small team of people that are known as Tech Quality Assurance. They are comprised of a group of experienced ex-Test Managers, although I often think they can’t have learned much in their experience due to some of the ideas they come up with or how unaligned they are with their communication.

A few years back, there was a project I was working on which was near completion and I got asked to fill in a “Merge Ready Checklist“. I was like “errr, what’s this?“

If the QA team are going to introduce a new process, shouldn’t this be communicated when it was created?

I looked through it, and most of the evidence they wanted were aspects that you needed to be aware of up front, so you could track them, and gather evidence as the project progresses. When I saw it, I thought it wouldn’t be possible to be allowed to merge. Even some of the simple stuff like:

“Branches are expected to be kept in sync with root development branch on weekly basis. Provide evidence of the frequency you were merging from Main and the date of the most recent merge.”

process on branch-merging

I merged when I wanted, so what are they gonna do? reject it because I didn’t merge enough? The branch history is the evidence, why is it worded as “provide evidence“. Why not just ask for the link to the code branch?

They then insist on having Test Coverage of 80% which I always think is an unreasonable ask. When I joined Development, we had Visual Studio Enterprise licences which have a Test Coverage tool available. However we have since downgraded to Professional. So I ask what Test Coverage tools we can use because it needs to be something that IT have approved to download, and that we have a licence for. We were told we could use AxoCover, but I found it wasn’t compatible with Visual Studio 2019 or above, which was an inconvenience.

Ok, so let’s run it and see what happens. Firstly, you are greeted with a phallic symbol.

Test execution started.
         ___
        /    \
        |     |
        \    /
        /    \
       /      \
___/        \___
/                      \
|     _______     |
\__ _/        \___ /
AxoCover Test Runner Console

It’s supposed to look like their logo, which looks more like a fidget-spinner.

Then I can’t even make sense of the statistics? Is that saying that I have removed methods, and deleted tests? I have added a few classes with several methods each (these obviously contain lines of code and conditional statements so I expect all values to be higher. I had also added some Unit Tests but maybe would have expected 30% on new code.

I asked Tech QA to explain the figures to me, and they were like “we dunno, we aren’t developers. We just look for the 80% number“. Then I point out that they were supposed to be judging the 80% coverage on NEW CODE only. This is for the entire solution file. So this doesn’t give them the evidence they want, and it’s not accurate either and cannot be trusted.

After running it several times and adding/removing code to see how the numbers changed, I then was suddenly low on disc space. Turns out Axo Cover reports are 252MB each! Yikes.

They also wanted you to run the static analysis tool Sonar, but with the licence we paid for, we could only run it on our Main branch. So we need to merge our project in to run Sonar, but they want the Sonar results to authorise if we can merge it in. The classic chicken and egg scenario. Later on, we did get better licences to run Sonar on our project branches.

When we submitted our Merge Ready Checklist to the best of our abilities, we got the following feedback:

“TechQA have reviewed the MRC and assessed it with a critical level of risk. There are a number of items that we would deem vital to address however even resolving these items – it would leave the project at critical.”

We failed the Merge Ready Checklist

Surely that is the biggest rejection. Even resolving the issues won’t change their judgement.

We arranged a meeting with them to discuss the way forward. At one point they said:

“These are all things that are too late to address but are common themes we raise time and time again and significantly reduce confidence in the quality of a release.”

TechQA

Surely that’s a problem with their communication. So the process is just sprung on teams when they want to merge in, then the problems aren’t even fed back into the system/communicated to other teams so they can’t learn from it, and then they proceed to fail.

I then asked the release manager what we can do, and he said that the TechQA team are there just to advise him what should and shouldn’t go in the release. Due to contractual deadlines, he just lets projects go in anyway as long as the team explains what bugs they are aware of. All the other aspects like Sonar and Test Coverage don’t bother him at all because it is more “Technical Debt” and not really that reflective of the user’s experience.

So what we are concluding is that the TechQA team are completely pointless and we may as well save money by binning them off, because they are just creating processes that no one cares about, and aren’t enforcing anyway.

It looked like they were also doing these for another company we acquired. I found this on another team’s review:

It should come as no surprise that the assessment of the MRC comes out as critical.

  1. The “Definition of Done” specified in the document is not the one followed by the team as you point out that no testing was part of the user story and a Product Owner was not attached to the project towards the end and therefore could not do the PO review.
  2. Also, no unit tests exist for the code and there is no ability to confirm the changes have not degraded any quality metrics.
  3. Elements of the regression plan have not been run and once again the team have not been given sufficient time to consider quality.
  4. On top of all that the code is written in Visual Foxpro which was EOL’d by Microsoft in Jan 2010, that’s 11.5 years out of support!

An example for the other week saw the lead developer give a lot of backchat to TechQA.

"Link is to a PR not to a build - where do I go to see the state of the build"
 > All PR's into master need a passing build. If you would like a hand using Azure DevOps please let me know and I'll see if can find time to show you how it works. 
 
"there must be some sort of output that can be used as evidence surely"
 > Once you have looked at the build, you can also view the 35k passing unit tests. 
 
"Need some evidence of your test coverage levels in the form of a report or chart from the text coverage tooling you are using"
 > You can see the tests written in the PR to master. Each PR for the new work was accompanied by tests.
 > Coverage Report attached for the impacted code, although code coverage is a useless metric.
 
 "Sonar - Please provide evidence when you have it available"
 > If the work is ever completed, then we will have automatic sonar reports for projects.

No idea why he doesn’t have Sonar running, but he obviously doesn’t care enough, and definitely doesn’t care about Test Coverage. I do find it strange that we have been using Azure DevOps for years now, and TechQA still don’t know how our development process works – and you would think it would be a prerequisite for them to do their work.

They should be liaising with the Development team in order to create processes that are feasible, useful, and accurate.

Employee of the Month

Managers seem to love the concept of Employee of the Month/Quarter/Year. On paper, it sounds good because it recognises the efforts of an employee, but I often find that managers don’t really understand the effort that goes into something, and they also don’t have knowledge about the entire development process.

So what will happen – is that there will be a shortlist of nominations, then managers will get together and debate on who should win. Some of the nominations will have been highlighted by any staff members, but some will be the manager’s own opinion. Regardless, the emphasis is really delivered by the manager who is representing the nomination. So a more persuasive manager will hype up their candidate more than someone with less persuasive skills.

As an aside: as a non-manager, I can only judge by their description of why they gave the award, and I might have my own view on who deserved it based on my limited knowledge of the projects that I have heard about. So their judgement may be misguided, but so is mine because I also don’t have full knowledge across the department.

Another thing to bear in mind is that managers can be biassed towards the financial gains of the project. Team members don’t decide what projects they work on, so as long as they don’t completely mess up the project, sometimes it seems pre-determined who wins.

I remember talking to a colleague whose team had won the “Team of the Quarter” award. He said it was an absolute joke because the project had overrun by 3 months or so due to mistakes within the team. He said there were all kinds of arguments and team morale was low. They knew they had cost the company money, but the late fines were insignificant compared to the money they gained when the project was delivered. Then, because it was worth loads of money, the managers gave them the award and cited “tough challenges” that were overcome.

Recent Awards

Strangely, in our last awards ceremony, an award was given to a team whose analysis and discussions led to the cancellation of one of the highest-value projects we had gone for in recent years. The CEOs seemed to make a big joke out of it, but were probably crying on the inside as we had not only paid 20+ people a year or so salary, but lost out on potentially millions of revenue.

There were also some great examples of somewhat-predetermined winners. The team leading the “Employer Of Choice” scheme won an award based on “improving work conditions“. Who else was gonna win that? The team was literally formed with that objective. Should you get it for merely delivering it, or should it only be given if you exceed it?

There was a recent problem where some users were experiencing a 20 second wait time to load up a customer’s record. A developer was assigned to the bug and put out a fix. He won a “customer satisfaction” award. Whichever developer was assigned to that was gonna win – it wasn’t anything to do with his contribution at all. The thing is, if you look at his change, and look at the code review, it was a bigger team effort:

  1. 2 expert developers had left all kinds of comments on his review and made him come up with an alternate fix. So the solution he came up with by himself wasn’t adequate.
  2. One of the comments stated how his fix simply moved the problem from “first load of a customer record” to “when the user logs in”. This was stupid for 2 reasons. That very same developer had fixed a problem with excessive login times 2 months prior so he should know it wasn’t a good idea to extend the log-in time. Secondly, some users like System Admins may never look at customer records, so extending their login time in order to save time when loading records is terrible.
  3. Then the developer had also stated that part of the actual fix was created by a senior developer.

So really, barely any of the solution was actually produced by the developer that won the award. It was a team effort and they didn’t get recognised.

What annoys me, is it was for the “Year” period, but some nominees had already won when they did “Employee of the Quarter”, so they ended up winning twice. I suppose it is stupid if you don’t win the Quarter award but then win for the Year, but maybe they could have come up with different categories, so you can’t win both.

In recent times, we have been producing some video interviews with the winners along with the people that nominated them for the award. The video is full of soundbites and emotional music. It’s so unbelievably pretentious. One video was hyping up a newly-hired manager’s Employee of the Quarter award:

“That’s what I have brought to the business. Don’t make a decision unless you understand all aspects of it.”

Winning manager

Kinda sounds like he is having a dig at the current managers and their rash decisions!

“interpersonal connectedness. He is driving that agenda which is great”

The nominator

“I’ve got a long way to go but yeah, I’ll get there, and I’ll bring the business with me”

Winning manager

My Win?

Several years ago when I was a Tester, there was one particular awards ceremony that I’ll never forget. When the awards were read out, the managers created tension by reading out what the winner did before announcing the person’s name. So they were like “this person worked on Project X, and this person took on challenge Y, showed positive attitude” etc. I couldn’t believe it because it described my work in my team! I was overjoyed how my team had recognised my work over the months and had put me forward. In that team, I had successfully delivered the previous project when I was the only Tester and we had no assigned Team Lead. Then I had moved onto a new project with the same team, plus one new problematic team member. I felt like I deserved something to make up for the fact we had a really problematic team member who caused me a lot of stress. Then when they finally read out the name – they gave it to the problematic team member! I nearly swore and stormed out of the room.

Virtual Praise Cards

We did come up with some virtual praise cards that you can send to your colleagues. There’s certain people that frequently give them out, so it leads to the same names that gain them. Often you see them just handed out for doing basic things. Like I was saying before; you are just literally doing your job and you get praise. There was one that I was joking about with my colleagues because it just seemed like – either the employee was trying to get in his manager’s good books, or was just having a laugh with the system.

 “Thanks for sharing the updates time and again. Literally under good leadership.” – Praise card for a manager

I also got quite a few of these cards during the first year of them being implemented, but then it seemed like a fad and I have rarely received them since.

LinkedIn Learning

I was recently watching a course on LinkedIn Learning about “Recognizing and rewarding great performance”. There were some good lessons in there: 

  1. Employees will respond more to non-money recognition once they are already paid fairly 
  2. Monetary reward should be proportional to the work value. If they save the company £10000, don’t reward them a measly £20. 
  3. High performers are best rewarded with growth opportunities 
  4. Peer to peer recognition is powerful  
  5. Be careful with incentives that creates more losers than winners. – might not help people you are competing against 
  6. Be mindful that incentives can be manipulated 
  7. Flexibility is a reward every employee can appreciate. Extra day off.

Timesheets

Government Tax Credits

Many years ago, managers asked us to start filling in a Timesheet. They were adamant it wasn’t used to actually track what we were doing, and accepted we may inflate the actual hours to fill the working day e.g. we might have been chilling for 2 hours but then claim we did the full 7.5 hours on a project.

They implied it was to help them claim for some “UK Government Tax Credits” which was some kind of tax rebate for companies involved in “innovation“.

The thing is, I think 95% of our work is contractual so we have been given some requirements by our main customer, and we implement it. So it’s not driven by our own creativity in the hope that people want it. It is driven by the customer who already announced they will pay. Therefore, I think our claims were probably a tax scam. That’s my opinion though, and based on no legal understanding.

One day, I got an email from auditors KPMG. I wasn’t going to lie, so I told them it was simply making a few changes to allow the feature to work for a different country. Also, it got cancelled.

KPMG Audit

I am from the audit team at KPMG and have been given your contact details by the group finance team and have a couple of questions regarding work on a specific project in the year.

Group finance have recorded that you have spent 1/2 of your time in 2018 on the Scotland project. Could you please confirm this through answering the following questions:

1)  Please can you describe the nature of the project we have selected for testing? (i.e. is this project to do with maintaining existing software, or enhancing/building new software)

2)  What’s the status of the overall project? (i.e. how far from completion? / have any issues been encountered which you are struggling to resolve?)

3)  Does the attached percentage of your time appear an appropriate representation of the proportion of time you spent working on this project at this time?

4)  How do you record your time, and do you know how the percentages we’ve outlined to you are derived?

If you could send your response directly to me by the end of the day that would be great.

KPMG

I wonder what they did with my response telling them the project got cancelled. I also think the time spent wasn’t correct. In different years, I have known other people get contacted with wildly inaccurate times as well, so I have no idea how the times we enter in the timesheet somehow become completely different figures when reported by the Finance department.

New Timesheet Meeting

It was my understanding that the entire business (or at least anyone involved in software) filled in their timesheets. But then more recently, we were all asked to join a meeting about timesheets, and I was surprised at how many people were protesting about this “new policy“. Even people working on the same software as me said they hadn’t had to fill it in. So I had been filling it in for years after being told it was mandatory, but then it seems it wasn’t – depending on which team you were in.

There were quite a few companies we had acquired over the years which also hadn’t been asked to follow this process.

This is what we were told:

Purpose & importance of timesheets

  • Resource plans only forecast the allocation of people and time. Timesheets provide actual and factual data which provide the following main areas of value to the company.
  • The ability to improve our scheduling — we can use this data to better plan in the work we have for the department at a team and individual level.
  • The ability to improve our forecasting — by reviewing Forecast vs. Actual time we become more knowledgeable and therefore accurate in our forecasting process, this improves our understanding of what and for how long our teams need on various items of work.
  • The ability to understand what things cost — by understanding how much time is spent we can get a generalised view of how much the work we do costs. This is important for both budgeting and recognition of capitalised revenue – It also helps us know what to charge our customers where appropriate.
  • The ability to understand if we are efficient — by understanding how much actual time is spent on various activities versus what was allocated we can get a view on what disrupts our plans, by how much and who is impacted. We can then use this data to mitigate this.
  • Person specific salary information is not used to calculate cost; finance use a generalised role-based approach and the SFIA framework.

Discussion

We were told that “Contractors are not applicable for Timesheets“. Why? Surely this goes against all the reasons why they want to do timesheets, to estimate the time and money cost of projects and improve forecasting. If outsourcing isn’t logged, it gives a false impression.

Some people were asking questions like “What’s the maximum/minimum hours to be logged per day and per week?“. Is that them admitting they had no idea what their contracted hours were?

In similar fashion, you would get people who work part time asking questions such as: “I don’t work Fridays, do I need to log anything for that day?“

Also, in a similar fashion, some people wanted to admit they didn’t really do their contracted hours, but had to inflate it to overreport it. I suppose if they want to know how many days/weeks projects are taking, then recording how many hours you have been slacking does make sense:

“When I started recording timesheets in this system I was putting in less than my contracted hours, and then got a message saying the timesheets were incomplete. I now record all of my contracted hours. Let me know if this isn’t what you want.“

Then you had people with the opposite problem

Should we accurately record the number of hours worked, or should our timesheet always add up to 40 hours a week regardless of extra hours done?“

Some Tech Lead/Architect-type staff made a good point that since they work across projects and will help others when needed, they won’t have that “project task” on their timesheet because they haven’t been assigned to that project:

“I do a lot of work doing “consultancy” for projects that I’m not formally allocated to, so naturally those projects are not in my “Tasks” list. Am I expected to try and find project codes for each of these (and get allocated to those projects even for a bit of one-off help) or should I just use ‘BAU’ (Business as Usual) for that?”

Another architect said his work actually spanned multiple projects because he was working on something that is a prerequisite for both projects. So would that mean he just halves the time against 2 projects?

In a similar fashion, sometimes problems arise with some legacy system, and the experts in that field will have to investigate or even fix the issue. Does this then just go down as a generic “Out of scope” item? That sounds like something that is important to track.

Some people were moaning that they might already fill in some time against their assigned Bugs (which was their specific team’s decision), but now they have to also report the time into this central system.

I think there’s plenty of aspects that are open to interpretation or people’s preferences. I noticed we have a “Meeting” task, but when there’s a meeting about your project, then shouldn’t that just be logged as Project Work, and then Meetings should be for non-project meetings? Then sometimes we have optional meetings where someone is demoing their project, or another department is announcing some news. Is that a “Meeting” or is that “Out of Scope”, or if it is educational, should it go down as “Personal Development”?

So it seems the simple idea of creating a timesheet has loads of complexity that a lot of people haven’t considered. I suppose the answer comes down to what you actually want to measure and why?  

The manager’s response to the questions about “if the time needs to add up to your contracted hours” – was that “the time should be accurate and it shouldn’t just be ‘adding up to something!’”. But the system doesn’t really support that. There was a time where a teammate’s time was erratic and he was starting at 2pm and sometimes staying late, or sometimes doing extra time on different days. So instead of doing a consistent 7.5 hours a day, his timesheet looked more like 2, 7, 13, 7.5, 9.5 etc, and it got flagged up as being inaccurate.

This was a brilliant quote from a colleague:

“the only accurate timesheets are the ones when I am annual leave”

Someone came up with an interesting idea. Since many people will just slap 7.5 hours against their project across 5 days, and we all know the timesheets aren’t going to be accurate, can’t we just assume if someone is assigned to a project, they automatically get 7.5 hours assigned to it, unless they log tasks on the contrary e.g. out of scope work, sick leave, holiday? 

“No” was the answer because that’s not how the Time Sheet system is designed. It is a good idea though.

There was one guy called Jason who was constantly moaning. I’d seen him on other calls and he seemed quite problematic. I could imagine he is the sort of person that would get sacked at one point for his problematic and negative attitude. The manager leading the call somehow remained patient with him, and came out with this quote which was a great response considering how frustrated she must have felt:

“I can’t work miracles Jason….I am telling you how I can help you right now”

Manager

Sounds like a film-quote.

Delayed reporting of Major Software Incidents

Process Problem

A few years ago, there was a period of time where I ended up being on the calls after a Major Incident (MI) with our software was declared. I was amazed at the clear failings with the process. There were all kinds of various managers, then maybe a few people from Deployment, but then barely anyone from Development. 

Then you’d have these managers just theorising if the Incident was caused by hardware or software, and if it was software – which enhancement/bug fix could have caused the issue?

It was pure guesswork of course, and occasionally they were right, but what is the point theorising when you could just get the people involved that can actually investigate it?

Sometimes I would hear about a Major Incident and then see that it was actually logged days ago. I think a problem is all the process between Support and Development. Support log incidents in their system and talk about “customer reference numbers”, but we need it in our system which is currently Azure DevOps and we talk about Work Item numbers. 

So sometimes there might be a delay even logging it in Support, and then there is a delay until it actually gets transferred over to our system. Often we are on these calls and we ask for a Work Item number so we can read information rather than just waiting for someone to mention it on the call… and then we are told that they haven’t created a Work Item yet.

I remember a time when users complained to Support, then they logged a Bug straight away, and did really good investigation. Then it would come to us with full details. Now, users complain for weeks, it gets escalated up to the Directors, then comes down the Development managers to Colin, then we have barely information to look into it because the users haven’t provided it, or is just “Chinese Whispers” and the information gets lost.

It’s so frustrating because how can you attempt to diagnose anything without all the required information? So the call is pointless, but when you do have the information, then you don’t want to be on a call – you want to go and start looking into it because you want to diagnose it then fix it.

So let’s look at one of the last examples I was involved in…

Example

Colin messages me about a Major Incident.

“I need some investigation…”

Colin

Apparently 179 sites are complaining about this problem and threatening to leave. Despite being complaining since the Friday release, Colin only knows about it Wednesday. Classic. Not the first time this has happened.

So what can I investigate? What is even the problem? All I know is there is a problem with a certain feature, but the thing is, we integrate with several third party API’s and I haven’t been told which one it involves, and I don’t know what the problem the users are seeing is.

I ask Colin to elaborate:

“not sure which provider. Can we look into all of them? So one of them may be broken, but we need to look at them all and hope we find it”

Colin

Wut. You want me to test them all out and hope I notice a problem? But it could involve just one and require specific steps.

When I finally got some information, it was pretty sparse.

“between 2 minutes and 12 minutes for the dialog to pop up” 

the problem

I told them I would have to see it happen, and see if there is anything different the user is doing to make it go from 2 minutes to 12 minutes. It sounds like a network issue, or not our fault.

“we will test it with a happy…well not a happy user, but an ‘engaged’ user”.

Manager’s suggestion of arranging a call with a user to witness the problem

I was then on a call with all kinds of random managers, all chipping in speculative accusations. “It’s a problem with the deployment“, “it’s a problem with the amount of data in their Tasks module“, “it’s a problem with the number of user-created resources“.

Colleague Opinion

I was discussing this with one of my colleagues and he re-iterated my views:

"they red flag anything nowadays - they just wave their flags around
It’s always the case that someone knew several days ago, but now it needs fixing immediately and it’s the first time any developer has heard about it

why can it not go:
Problem logged by customer -> Support discuss with Release Management -> Release Management arrange call with the correct Team Leader -> Prioritises the work with a developer and tester

I don't see how that's difficult

What actually happens:
Problem logged by customer -> sits in a list for a while
Problem management person looks at issue -> sits in list for a while
Someone kicks off and escalates the issue -> Director gets it in the neck -> Release Management notified
Release Management hold a call -> float many theories about what the issue is about
Developer randomly hears about issue -> mistakenly joins call mid-flow -> explains that it's probably related to the work item with the same keywords as the issue in the title
Developer now late with their own work
Release management say they'll prioritise the issue, but asks if you could start working on it -> Developer says no because you are busy working on 8 different things -> everyone in a huff
Team Leader nowhere to be seen -> likely watching the TV"

Apparent Analysis

A manager recently posted an update with facts that I was sceptical of:

“Analysis conducted into 2 years worth of Major Incidents

Some interesting trends have appeared and I think it’s worth discussing:

  • Data shows that most of the MIs are caused by either a 3rd party or tech debt/ops related issues. The number attributed to code change is pretty low.
  • We are lacking ownership around domains for existing products so not obvious which Engineering Manager is accountable.
  • We don’t have enough SQL skills to manage our complex DB
  • We rely upon a small number of people to investigate MIs.
  • MIs should be owned with the right teams.
  • Further investigation required on this and changes to ownership, structure, investment needs to happen.”

Release at Pace

Another interesting claim was made by another manager:

“There’s lots of interesting developments going on at the moment; we’re delivering more than ever before.”

Manager claim

I message the Software Delivery Manager to ask if there is any truth in that, because I thought we were releasing at the slowest pace.

“Who knows? I reckon one of those blaze statements that people just say to make it sound good”

Software Delivery Manager

On a recent Development meeting, one person posed a question to the Head of Development:

“Deploying software at speed is ultimately what we want to do as a business. However, this can come at a cost. We have had more Incidents in Q1 caused by software defects than we did in the whole of last year! How do we ensure the deployed code is at the quality required if we are going to deploy at speed?” 

Staff member’s concern about software quality

If we deliver more changes, I suppose there is more risk of introducing more bugs. If you are rushing, then you are more likely to create Major Incidents. I suppose you can say it is fine if there’s more bugs in general, but all considered low severity.

The Rapid Responders Group

As a response to this, I think Colin came up with the idea of the “Rapid Responders Group”

“We are deploying our software at a pace we have never done before, so we can miss scenarios that could happen in the production environments as we have seen in the last two MIs. If we can get information straightway, we can investigate the issue straightaway. We all have access to the Live environment. With our technical ability, we might be able to see things that other people might not, so this is why I have assembled this team.”

Colin on his new idea

However, it was quickly shot down by one of our architects and never mentioned again.

“With one of the MIs, there was absolutely nothing we could’ve done between the people in this channel to anticipate that as it relates to a combination of live configuration and state that can’t be replicated.

In general the problems we have with the MI process is making changes takes too long due to processes and permissions, noise from the non-experts present (it’s hard to get a word in edgeways with some people), and multiple people bugging those performing the fixes/changes for updates (when it should be one person who fronts the technical team working on sorting it out)”

Architect

“We had first-hand experience of this on the last Major Incident call. We could have fixed it within 30 minutes but was on a call discussing it for hours.”

Colin
Architect:
It's usually the fault of those running the call or lack of confidence in Dev.
I will say when we have enough information to make a fix - that I'm departing the call to concentrate. As you say, it will otherwise go on for hours, with multiple people fronting opinions on the cause of the problem, unless someone identifies it with confidence and takes the initiative.

When an MI occurs, it's always the same group of people, which is often a pretty good mix, but first we need to identify the cause.
 
Usually it takes a long time for the details of the MI to be logged, particularly somewhere Development can all access such as a Work Item. If you miss the start of the call, it's difficult to know what the details of the original issue were: you can guarantee you missed something of importance.
 
I would suggest the following to help the diagnosis run more smoothly:
  1. When a release is made, it must identify a point of contact for each change, preferably the team email address making the change. 
  2. When an MI is started, a brief description must be provided in written form and a source-of-truth record started (such as a Work Item). 
  3. The Release Point Of Contact list is emailed with details of the MI starting.
  4. They might not need to join the call unless there is something they can add, but at least if they feel their input valuable to the diagnosis they can join or reply with details. 
  5. After identifying the team/person working on a fix, they are protected by a single manager. The manager will be responsible for communicating with the techies and the outside world, so they can concentrate. They will update the Work Item and email those who need to know progress, what the fix entails etc. 
  6. Work out a way to expedite changes to live and retrospectively log change requests
Principal Developer:
From the MI's I've been involved in, they mostly run smoothly - the only disruption is (as Mike said) people "sticking their oar in" and having a guess at what the problem might be - sometimes forcefully.

They are not always wrong, but sometimes it can distract from getting to the route of the issue, you need to have the confidence to talk over them and get the information you need, correct misunderstanding, and to keep the discussion on track. A lot of the time the initial presentation from support can be way off the mark, but that is true of most bugs we get due the lack of technical skill in support. Only the developers on the call are going to be able to diagnose a fault as a result of a code change. 

The primary focus of an MI call is to restore service ASAP, exploring workarounds is part of the developers role on the call - can we change a stored proc / setting to improve things now? This helps users and gives development more time to work on a fix. Another role is to assist the wider business (especially those in the Safety and Governance roles) to understand the issue - is it as bad as initially reported, or is it worse!?

Once we know what the problem is, and how we will resolve it, then it is OK to drop off and work on the fix.

Conclusion

I think it is clear that too much time is wasted discussing the problem with the wrong people. Managers need to find the correct group of technical people, give them all the information they need, then give them time without hounding them – in order to actually fix it.

Recruiting Graduates #6: The Disappointment

Last year, I wrote some blogs explaining how the hiring process for Software Developers is bad.

General Summary Of Why The Software Developer Hiring Process Is Bad

When I was involved in trying to come up with something better, I was frustrated because I didn’t have the answers, but I felt everyone else involved didn’t understand my concerns.

I hate the way Tech Interviews and tests currently are, but I think it’s hard to come up with something as a replacement. When researching example questions, I see lots of questions along the lines of “who invented Object Oriented Programming?“, and I think “I don’t care“.

Then there’s loads of aspects that you would just Google if you didn’t know it e.g. memory usage of each type.

Or you may get questions like “what is wrong with this code?“, but if you actually pasted that code in Visual Studio and try and build it, you would see the error. Or in the case it is valid but not advisable; Code Analysis would flag it. Some of these can be trick questions like mathematical logic where the order of execution is ambiguous, but Code Analysis would definitely flag to tell you to add brackets, so it’s silly having these as a question.

Then I hate all the generic Fizzbuzz stuff (check if a number is divisible by 3 or 5, or even both). These are just so cliché that people could just learn from memory anyway.

Also, what do these graduates we are targeting actually know? back in my day, I knew Java but I was terrible and didn’t know how to test properly, or even debug. So there’s no point testing them on that either.

Stupid Questions From The Internet

During our meetings, I asked the team how they were coming up with the interview questions, and they said they were just Googling for C# Interview questions. This is problematic because applicants may have read and learnt from these very sites, but also many sites had terrible questions, which were sometimes incorrect.  

I came across this website:
https://www.includehelp.com/mcq/is-the-use-of-return-statement-necessary-in-every-function-in-csharp.aspx

 95. Is the use of return statement necessary in every function in C#?

 Yes No

 Answer: A) Yes

 Explanation: Yes, the use of return statement is necessary in every function in C#.

I think some programming languages may differentiate between “methods” and “functions” where “functions” always return something. But in C# you just call them “methods” which can either return something, or be defined as “void”. So this is just incorrect.

 99. Which is the first line of a C# program?

 using System;

 using system;

 using Namespace;

 namespace MyApplication

  

 Answer: A) using System;

If you create a blank class, “using System;” is the first line, but A) you could remove it B) the order of using statements matter C) in modern C# you could move it to a “global namespace”. I also asked the question of “who are we targeting?” because everyone was coming up with C# specific questions. My colleagues told me the universities we were targeting were primarily coding using Python! So we were writing the wrong test, or targetting the wrong people.

The Performance Review

In my recent performance review, my manager Colin stated:

“You were given an opportunity to lead on defining our interview process, but have not shown much interest in it. Ultimately someone else led it, creating the interview pack, redefining interview questions and leading live interviews, which I think is a missed opportunity to shine.”

Colin

I argued that: although I didn’t have a huge impact on the end result, it would have been a bigger mess had I not pushed back on some of the questions. Although it seems some of the interviewers still asked questions I said weren’t suitable.

We ended up coming up with a terrible process that some candidates dropped out of, and the ones that went through with the interview struggled and we were left with the decision to gamble based on what little we saw.

The Best Hire

One of the best hires we made was someone I didn’t interview, but I looked at the interviewer’s notes, and he didn’t exactly look like a strong hire. 

  • Implemented the basic application to consume webapi and just printed the values based on some predefined values
  • No unit testcases were written
  • application is simple
  • Inheritance – simple explanation
  • polymorphism – no idea
  • interface – not good explanation
  • purpose of inheritance – No idea
  • Print the numbers and its occurrences – Not completed, but after showing the hints he completed
  • Print the number from 1 to 100 using Recursion – Completed
  • print the numbers based on divisible values – Completed
  • Logical, cut the cakes into 8 pieces with 3 cuts – Completed
  • Candidate lacks on syntactically at some areas, but has some logical solving skills and good attitude on approaching things

When I stated to Colin that it “wasn’t a great batch of hiring”, Colin said out of the people we did interview, we ended up hiring around a third of them – which he said was above the normal hiring rate. I said that it was only above the hiring rate because we hired people that should have failed. Then later, he quoted the numbers “15 interviewed. 4 hired” (more like a quarter then). But there were loads of applicants rejected before the interview stage, and others dropped out along the way.

Intermission: Bantz from Twitter

The Polymorphism question

The Polymorphism question that was asked on some of the interviews was basically “do you know the types of polymorphism?”   

I didn’t even understand the question, and I probably have 10 years of experience.

I was only thinking about the usual polymorphism where you can refer to derived types as the base type:

 IAnimal animal = new Dog();
 animal= new Cat();

To check if I wasn’t going mad, I asked some of my colleagues. Here is how they responded:

“not sure i do, no. Inheritance and Composition, are they right?”

Dean (Senior Developer around 10 years experience)

Like covariance and contravariance? But I don’t think I could put it into words. I guess you have real polymorphism and fake…virtual functions have a virtual function table that map overridden functions from base classes. But a lot of polymorphism is just the compiler being intuitive and going “you’re not crazy” it can be treated as that type. 

Adam (Software Architect)

Types? an interface can be implemented many times. A method can have the same sig but different parameter types etc

Rob (Senior Developer around 20 years experience)

According to https://www.bmc.com/blogs/polymorphism-programming/, there’s Runtime, Compile Time, Overloading, and Casting. So if very experienced developers can’t explain the concept, then why are we using the question to target Graduate developers?

Conclusion

It annoyed me that Colin stated “have not shown much interest in it”. I was trying to make sure we had well-defined objectives on the exact type of developer we were hiring. Everyone else just started copying questions for the internet without considering the knowledge and experience of the people we were targeting. I didn’t have the solution of how to make a good software development hiring process, but I really wanted to make sure we got close – but I just didn’t have any support from everyone else that was involved.

Burndown Chart: tasks

Colin said that our Sprint Burndown chart wasn’t accurately reflecting the work done. He said we were overrunning with each Work Item which meant that we would have 0 points for one Sprint, then we get the full value in the next Sprint – which shows as a sharp drop on the Burndown chart.

I told him that’s how the Burndown charts work – but he said they want more accuracy. I argued further: If the requirements haven’t been met, then it’s not complete, so you haven’t added value to the software – i.e. you have made no progress.

A few days later, our Scrum Master had been in a meeting with him and was instructing us on his new process. Colin’s idea apparently was to add a “task” for each day and link it to the Work Item. At the end of the day, you mark the task as closed. 

I’m like “eeeeeeer wut”. So now it tracks your daily work.

I told her she must have misunderstood. Adding a task per day is just counting the amount of days in the week. I suppose if you take a day off, then you won’t count that day.

I questioned it, and she agreed that it didn’t sound right. So she goes back to Colin. No, he really does want a closed task per day, but also said to create a task even if you are on holiday.

Wut.

:all-the-things:

add tasks to all the things! 

So they want effort more accurately tracked, but are now just counting days, even the ones you haven’t worked. Surely if you create a chart, it’s just gonna be a diagonal line with no fluctuations.

What are we supposed to write for the task’s title and description? “Carried on working on it 7.5 hours”.

I just refused to do it, but the Scrum Master did the admin on my behalf. The idea lasted about a month.

I find that Burndown Charts often look unclear anyway. Here is one from another team:

So what are we even seeing here? The chart to the left shows Tasks, although the chart doesn’t seem to show the correct Completed figure – it shows as 0%. However, you do see the average is 46 which I think is per day – which illustrates the ridiculous number of tasks that teams were creating anyway.

The chart to the right shows User Stories but I think it’s not the number created, but the total points assigned. So one might be worth 1 point, but another could be worth 8 – it depends on how complicated the work is. I think this is a typical Burndown where the first few days nothing is complete because the developers are working on fixes, then the tester will get it in a few days. In the second week, more items are completed. There were even 4 points removed, presumably a change of requirement or maybe it was deemed redundant.

This is another chart that a team posted to boast about their progress. This example is a bit more unclear, but I noticed the Tasks Burndown (start 12th September) is not for the same time period that the Stories Burndown is (start 29th August).

The Stories Burndown looks interesting doesn’t it? It looks like only a small amount of work was done and then when it gets to the end of the second week, they add even more work. I did theorise that maybe they didn’t officially start the project until after 10th September but what does the 72% Completed mean? That seems to imply they are ¾ through their project. ¯\_( ͡° ͜ʖ ͡°)_/¯

Initial Onboarding Guide For Software Developers

Intro

I’m stealing ideas from a former colleague again. He had a blog post about Onboarding, and I felt I had seen many of these mistakes myself so they are worth thinking about if you are a manager and are expecting a new starter. These points are mainly about an office environment, so some points are no longer relevant to us, or are different now we work from home.

Onboarding Checklist

There’s a lot of work done during the recruitment stage; reading CVs, lots of meetings with recruitment teams (in-house or agency), and lots of technical tests and/or face-to-face chats with candidates. Once the offers are accepted, your work doesn’t stop there, but you have some time to prepare until the new-hires start their new roles. Some tasks are:

  1. Find a desk
  2. Find a chair
  3. Order the laptop
  4. Order Monitors
  5. Order the peripherals
  6. Consider collaboration
  7. Grant access to digital tools and codebases
  8. Make sure you are available on their first day

If you do not deliver on the above, the new starter’s first impression of you will be of failure. If you can’t organise equipment, desk, and have a plan to get them started, then what else will be missed? (pay reviews, promotions, training budget, etc)

Onboarding Checklist Discussion

Find a desk

If you are replacing a staff member, then you should have a desk already. However, is it clean?  If the desk has been vacant for some time, it can end up being a dumping ground for miscellaneous items (books, broken equipment, old stationery). Are there any problems with the desk? Does it wobble? Is the key missing for the drawers?

Find a chair

Chairs are more notorious for being damaged or dirty. Make sure all the wheels are still there and the back can be adjusted. The chair should be clean; not stained, or dusty, and free from any other horrors.

Order the laptop

You may be restricted by company procedure, but if there is a choice of laptop, make sure the new employee gets the one they desire. If the developer has a strong preference for a certain system Windows/Mac/Linux, it’s a productivity hit if they get something else. Another thing to consider is the form-factor – If they travel a lot, a smaller size may be much more comfortable for them, especially if they want to partially work on the train. The large screen can be gained from plugging into a large monitor at home or office.

Order Monitors

Many developers like having 2 monitors, although in recent times, it seems some are switching to 1 very large monitor. It’s great to have a combination of your code editor on one screen, then documentation/the software itself on the other.

Order the peripherals

Order the keyboard, mouse/trackpad, headset etc. Even if there are loads spare, they may be partially faulty, or just unhygienic.

Consider collaboration

Teams are full of people – full of personalities. It can be potentially tricky and there’s some nuance to it. Take some time to consider the optimal place for the new starter to sit. Is it the “replacement” spot? Is the desk more isolated? Is there some weird office joke about the desk!?

Who would they pair well with? Is the new starter’s job something someone in the team wanted (missed out on a promotion)? Is there someone who is seen as a bad influence?

Grant access to digital tools and codebases

Sort access to the code, and work management tools (licence keys for all required software).

Make sure you are available on their first day

If office based, be in the office. If distributed, you need to be free for video calls etc.

THIS IS YOUR MANAGER WHO BOB, CAN YOU SHOW WILL TAKE CREDIT FOR ALL THE NEW EMPLOYEE YOUR WORK. AROUND? SURE. (YAN G e 99 workchronicles.com AND THIS IS WHERE YOU'LL THIS IS WHERE YOU'LL WORK LATE NIGHTS ON SPEND HOURS IN EVER CHANGING PRIORITIES. 99 Hello. I make comics about work. Join r/workchronicles or follow on Instagram/Twitter/FB Work Chronicles workchronic/es.com

Bonus: New Starter Packs

When I started, I think all I got was a writing pad to make notes. Maybe I got a pen.

Years later – existing staff, and all new staff were also given a water bottle, and a company-branded hot drinks flask. Even though it’s a simple thing, everyone seemed to appreciate it.

More recently, it was announced that “New Starter Packs” would be posted out to all new starters, and we were asked for feedback on the contents. I don’t get why they didn’t anticipate some backlash because they only planned on giving them to new starters. Also, they clearly had already ordered a certain level of stock because they were taking photos of them and had quite a few company-branded items.

There were the obvious choices of pen, paper, post-it notes, drinks bottle, in addition to some chocolates, sweets and some other miscellaneous items.

“The new starters will be there with their fancy laptops, new starter kit, and high wages, laughing at us”

Me

Some people criticised the choice of unhealthy treats, given some recent schemes to promote healthy lifestyles. There was also a Webcam Cover you could put on your laptop, which someone questioned if our IT department had approved it – to which the Head of IT said they weren’t authorised because they “crack screens and they also damage the bezel of other laptops when removed.”

It was also only for staff based in England, but a high percentage of staff are now based in India, so the managers admitted we need to find a supplier over there too to be fair.

So it wasn’t a well thought out idea really.

Software Development Process

When a new development team is created, managers insist on creating a “Ways of Working” which essentially comes down to describing the software development and testing process in the ideal way.

Then after a few weeks, you end up just resorting to the default way of working.

The “ideal” way assumes that the Testers can provide ideas and inspiration to the Developer, and when the Developer writes the fix, the solution won’t deviate from their original perception. Therefore, all the Test Cases are written up front, and none are needed to be written after.

Here is an example that one of my former teams came up with:

Before fixing the defect

Analyse together 

  • Make sure you pair up as a developer and tester to work on the defect.
  • Review the problem together
  • Understand how to replicate the fault
  • Understand the extent of the problem (all regions, all set-ups, all environments?)
  • Do the initial debugging to understand where the problem lies

Fixing the defect

The developer MUST 

  • Make sure there are unit tests to support the current functionality
  • If there aren’t unit tests then implement a simple refactor which will allow appropriate unit tests to be added
  • Add unit tests that prove the defect (which will fail)
  • Fix the defect
  • Get it code reviewed
  • If you are working on code you are not familiar with, get 2 reviews, one of which must be from an expert
  • Run all the unit tests and prove the fix

The tester MUST 

        • Write test cases with steps and expected results for the initial test cases. They must

                ○ Be clear and unambiguous

                ○ Cover the different scenarios/circumstances/options

                ○ Not test things that are irrelevant

        • Once written, set the state to “Review” and assign to the reviewer

        • The reviewer

                ○ Adds comments as appropriate

                ○ Set the state to “Ready” if Ok, or back to “Design” if it needs changes

                ○ Assign it back the test author

        • Only use Given, When, Then if you are using Specflow to automate your tests

        • Only use exploratory testing

                ○ If you have passed a Rapid Testing course

                ○ And you have a clear understanding of testing heuristics

                ○ And you plan and classify sessions, keep session logs, run debriefs

                ○ And all the specific test cases and all regression test cases have been executed

        • Create regression tests of the functionality

        • Create test case(s) to prove the defect

        • Get the tests reviewed

        • If you are working on functionality you are not familiar with, get 2 reviews, one must be from an expert

Assure the Quality

  • Review the actual changes together
  • Understand the logic
  • Understand the scope
    • Minor/contained (inside a method that’s inside a class that doesn’t affect anything that’s outside it)
    • Medium/impacting (a change that impacts outside it’s own class or is called from elsewhere in the system
    • Major/widespread (a complete refactoring of existing code or large changes to core features)
    • If they don’t understand the logic or can’t establish the scope, escalate to the Technical Manager
  • Execute the tests
  • Attach and link all work items and evidence together

How Software Bugs are Prioritised

A Product Manager recently wrote about how Software Bugs aka Problems are prioritised, so I thought I’d share that here.

Prioritisation Spotlight Report 

Product Managers conduct a weekly meeting with other stakeholders to discuss Problems and their effects on our customers. A key output of this meeting is to ensure that we are prioritising the defects that are causing our customers pain or have the ability to do so. 

These Problems can range from being the result of a major incident, recent software upgrades, or internal database monitoring. However, what they share is that they all have the ability to generate customer dissatisfaction. 

The Product Managers have been ensuring we are able to accurately and consistently apply logic to the Prioritisation process. This is a key requirement of the Problem review that allows us to create the prioritisation for Development to work on. 

How does this work? 

The weekly Problem Prioritisation meeting is open to anyone who has a business interest in resolving these software defects for our customers. When discussing these defects as a group, a number of areas are covered, some of the new key areas are below:

Number of cases linked to Problem – this is multiplied by 2 so a Problem with 4 cases will generate a score of 8 for example 

CSAT (Customer Satisfaction score)– This is the level of market impact the defect has, or is expected to have, and is scored in 4 areas; Critical, High, Medium, Low. 

Software upgrade blocker – Does this hold up the ability to patch customers to a newer version of the software

Safety rating – Does this have safety implications? 

The Prioritisation reason – has the problem been raised as an internal escalation,  safety, Information Governance, Security, Customer pain/Escalation, Service Level Agreement, or an Enhancement via the User Defined Roadmap. 

IG – Does this have Information Governance implications? 

Number of users impacted – taken into account based on how widespread the issue is and how many customers are affected. 

What will this allow us to do? 

This is going to help us all become aligned with the vision across the different areas of the business and will enable key stakeholders access to a single source of truth when scoping these items into the team’s backlog. A “Top 100 Problems” list will be updated after the weekly meetings.

Password Discussion

In a previous blog, I wrote about some routine training courses we need to do each year. I briefly discussed some security questions but there is one example I forgot to add. One course we did, my manager was raging that he got one of the questions “wrong” and he seemed more annoyed that he knew this course was one that NHS staff must also do. He Googled the question to see if anyone was complaining, and he found a post on stack exchange.

The question is as follows

Which of the following would make the most secure password? Select one:

a. 6 letters including lower and upper case.
b. 10 letters a mixture of upper and lower case.
c. 7 characters that include a mixture of numbers, letters and special characters.
d. 10 letters all uppercase.
e. 5 letters all in lowercase.

The answer we went with is B, because it’s the answer that contains the most characters, and as a bonus, has a mix of upper and lowercase.

However, this is apparently wrong, and we should have chosen C.

The Stack Exchange user, Robin Winslow, then referred to the famous XKCD comic, which illustrates what is now commonly thought of as the best type of password.

The key thing is that a password is useless if the person cannot remember their own password, and forcing complexity at the expense of this fact means the person is then most likely going to write it down. Additionally, forcing certain types of characters into the password also makes it more predictable.

If you ask me to choose a password, I may choose a memorable word, eg “programming“.

If you tell me I cannot have that because it needs to have an uppercase letter. I could add another character, or choose a random existing character to be uppercase, but that makes it hard to remember, so I am gonna choose “Programming“.

Then if you tell me it needs to have a number. Again, I could add a number somewhere in there, or substitute a letter, but that can be hard to remember, so I am going to put it on the end. There could be a significant number to me, or I can just choose the number 1 and put it at the end, because that is easy to remember, or easy for me to guess that I did that. So now it is “Programming1“.

Exactly the same thing happens if you tell me it has to be a special character. Probably end up choosing “Programming!1“. It has more characters than my initial password of “Programming” but it’s at the expense of memorability.

Another aspect to consider is password expiry. At work, we used to have to change our passwords every three months, but now it’s currently six months. However, what’s my next password going to be? Probably “Programming!2”. So if the aim is to stop people accessing the system if they somehow got my old password, they still will get in because they can just guess the new one.

I wasn’t sure what the rationale was to change the expiry from 3 to 6 months because if you think it’s not a great feature, then you would just outright remove it. When the Head of Security started working for us, the first thing I saw him do was walk away from his laptop without locking it – so maybe that illustrates how good his skills are.

Troy Hunt points out that Microsoft’s latest policy is to not use expiry at all.

Research has found that when periodic password resets are enforced, passwords become less secure. Users tend to pick a weaker password and vary it slightly for each reset. If a user creates a strong password (long, complex and without any pragmatic words present) it should remain just as strong in the future as it is today. It is Microsoft’s official security position to not expire passwords periodically without a specific reason, and recommends that cloud-only tenants set the password policy to never expire

Microsoft Azure

Troy makes some follow-up points

Geez there’s some debate about this one! Mostly support but also some misunderstanding so let’s fill some gaps: Firstly, password managers don’t solve this problem, not when you’re talking about the credentials to logon to your PC. That’s a rare case where you need to type it…

…unless you’ve gone passwordless via security keys, biometrics etc. Clearly this negates the need to use the password with such frequency thus reducing the opportunity for compromise. There may still be a password (e.g., fallback from biometrics), but exposure is much less.

The argument of “well what if the password is exposed a year later” could just as easily be “well what if it’s exposed a month later” when there is a 90 day rotation cycle. Different windows of risk, so why not make the cycle a week? Because it’s painful…

…plus, the solution is the same: MFA. The usefulness of a password alone for AD login has dramatically reduced with the mass adoption of multi factor. Mandate it on your AD tenant and the value of rotation dramatically reduces.

Troy Hunt

Also, there are always ways around policies for those more-determined users

I used to be at a company that did that. Someone quickly came up with a script that changed the password ten times in a row and set it back to the original (last ten passwords were not allowed).

Kerry W. Lothrop (https://twitter.com/troyhunt/status/1578149402304094208?s=20&t=Uk9uhXXLK2vPj2EA_pE4uw)

I’m not sure I 100% agree on the “well what if the password is exposed a year later” point. Expiry does remedy that in some cases but no expiry will always leave you vulnerable. You are relying on knowing there has been a breach.

Troy Hunt has discussed that the best security involves a mixture of methods. It generally comes down to

  1. Something you know
  2. Something you have
  3. Something you are

Aaron Toponce did recently post on Twitter about this, but his account has since been deleted. This is why dumping things into OneNote is useful. Troy Hunt has also discussed this idea many times.

Not sure what Taylor Swift has got to do with it. Maybe she pretends to be happy.

So if your account has all 3 types associated with it, even if someone knows your password, they cannot get in because they will need your token (Authenticator app on your phone). If they have managed to also steal your phone, then they can’t get in if they also need your fingerprint. If they are holding you at gun-point to make you unlock your account using your finger, then you have bigger problems.

Aaron’s musings was this:

If a password is stored in a password manager, is it still something known, or does it become something possessed? A password manager vault can be stolen.

Knowledge of where the passwords are stored and how to retrieve them then becomes “something known” and the vault itself “something possessed”. No?

Aaron Toponce

This has since become quite topical in the recent LastPass breach where keyvaults have been posted on hacking forums. If the master password is then cracked, then hackers will have access to many people’s accounts. I was always intrigued by password managers, but I did feel you really are putting your trust in one company.