Project Diary: Voice to Text AI

Introduction – August 2025

These days, our competitors are producing AI features, and since AI is the cool thing that all corporations want to promote alongside the claim they have “cutting-edge innovation”; we want to follow suit and we want to rush an AI product out.

If we make it too advanced though, it will meet the requirements for a 6 month (or more) legal approval process, so we want to keep it basic.

In recent years, even though we have struggled to get software out in a reasonable time-frame, they want this feature to be out after 3 months at most. The initial implication was that they wanted it finished by the end of year so would release straight after Christmas.

The lead developer reckoned the initial set up, which includes all the DevOps tasks, would take 3 weeks. Creating a repository is a few clicks, then there’s the pipelines to create, and all the “tooling” like installing and configuring Jest and Linting on the application side. I know Terraform is fiddly and the approval process for the AWS user permissions etc takes a while due to the process, but 3 weeks seemed a bit extreme to me.

The flow of the planned AI application was initially as follows:

We have an old product SystemNow, and its (eventual) replacement ProjectFutures. ProjectFutures is currently in use as a “Companion App” with minimal features. ProjectFutures runs as a browser based app. They don’t communicate effectively with each other in my opinion. So the old product SystemNow opens a new screen which then triggers opening a new page on the new product ProjectFutures. Then the user can hit record on ProjectFutures, upload the transcript to the 3rd-Party, who then sends back the transcribed audio in an annotated text format. Then we transform that data to then display inside SystemNow.

Reading through the architecture, it seems crazily complex, that the speech-to-text process involves sending the data to a 3rd party API, it then comes back with a processed transcript and we then send that to the cloud to send back to a different program on the same machine (this part was later changed to send a local message to the WebView component). There can be some kind of delay as well though; so the user could be waiting, but has to keep the program open.

Then what happens if there’s errors at each of those stages? How easily can the user recover if they have recorded a 10 minute clip but haven’t filed the data due to an error? I’d imagine once that happens a single time, then the user will stop using the feature.

I’m skeptical that this would save the user time, but might make their note-taking richer as long as the 3rd party actually sent detailed notes back.

To keep the company and product anonymous, I don’t want to disclose the actual product. To imagine the use, picture a customer having a consultation, in person or telephone, with a house designer. The designer will create a “consultation” which can have multiple pages, one for each room. Most consultations could just have one page, but multiple pages consultations are also used. Once they fill in text for each section in a page, they need to switch the free text entries to actual products. So if they have typed “white toaster”, they see some product suggestions, and then select an actual product in the system, which replaces the free text with a “code badge” eg [Russell Hobbs 27370 Eden 2 Slice Toaster] which makes it read-only. They can delete the “code badge” or swap it for another product, but cannot modify part of the text. This has a product ID associated with it, and can have “qualifiers” which are slightly different product variations without requiring a different code e.g. colour options for white and black.

So our AI speech to text feature aka “Scribe” will replace all (or most) of the typing. It listens to what the consultant and customer discussed, then generates the consultation pages and sections, and suggests product codes for each section. The consultant then just needs to verify each section looks correct and do the product coding.

Week 1 – 18th-22nd-August

Day 1: It seems Dave (Lead Developer) and Liam (Team Manager) have already created quite a few User Stories (aka “Work items” or “tickets”; where the requirements are supposed to be described as user-focussed) in meetings they had separately from the team. 

The idea was that the team could go into a Refinement meeting and focus on Story Pointing (estimating) but the disadvantage is that Jonny fully understood the requirements and the rest of the team didn’t.

Story Pointing is where we assign a score based on the predicted size using the fibonacci numbers 1,2,3,5,8,13; where 1 point is the smallest task. It’s a team vote so the most popular or average score is supposed to be chosen. My frustration is that we often scored too high, influenced each other’s votes, and didn’t consistently choose the average vote; so estimating always seemed like a waste of time in this team.

The initial User Stories were mainly the basic set-up, the prerequisites for development, so have nothing to do with user features. It’s setting up the repositories, pipelines, and configuring SonarCloud. I often wonder if these should be classed as User Stories.

For one of the set-up stories, I stated “it is a 1 point and we don’t need to discuss it, just move on to the next ones”. The team decides to vote anyway. Lucy chooses a 3. Then chickens out and comes down to a 1 without explaining why she thought it was far more complicated.

James then drops off planning.

Lucy chooses 2 for the next one, but Dave chooses 3, so Lucy then makes out she meant to choose 3 all along. Now we have two votes for 3 and my vote for a 2, so now I am outnumbered. I managed to get them to change to a 2 though.

For the next one, we all choose 2 apart from Lucy who chooses 3. She comes down to a 2. So that is 3 chickens in a row.

James comes back after 45 mins; what was he doing?

Week 2 – 25th-29th August

We now have the two code repositories created; one for the Client-side (UI) and one for the Server-Side (API) along with all the PR checks via GitHub Actions, and CodeQL.

There’s a debate of which 3rd-party provider we are using. Surely we shouldn’t be starting the project without API keys licensed?

We get assigned another 3 developers who are contractors. One contractor will join the business later. Two are currently employed with us but were originally assigned to another team but their project got cancelled. Is there a chance our project will get cancelled? It’s slow progress and the requirements aren’t locked down.

Everyone seemed to take one day’s leave at short notice but there’s not too much going on so not a big impact.

James declares he doesn’t want to do development work, but wants to focus on testing. Testing what though? we hadn’t got the repos fully set up, and hadn’t written any code to test. It will be a while until there’s any real testing work.

Week 3 – 1-5th September

The two contractors, Marek, and Andrei are sitting around reading documentation. They do ask for work but Liam (Team Manager) is unsure what to suggest as he didn’t expect to get them so soon.

I told Marek that our ReadMe in the API repo was a template and can be ignored. There’s literally no features so nothing to build or configure despite what the ReadMe says. Later he declares he doesn’t have the AWS access required as stated in the ReadMe. I tell him again “it’s a template”.

The next day Marek says he can’t install something as stated in the ReadMe. I tell him again “it’s a template”.

Andrei keeps starting meetings early when Outlook reminds him there’s a meeting in 5 mins. One of our meetings overruns so the manager declares a 10 minute break before we start our next meeting. Andrei instantly starts the next meeting.

We had a meeting with Sam, the UX Designer. The user flow is absolutely massive with a few questions over confidentiality and security etc. Also what happens if there’s connection problems during each phase. Does it carry on recording, and how long for? The user can’t really wait around, so can they quickly revert to manually entering the data instead?

Since we only planned work for 4 people, we don’t even have work to give the contractors. We need more refinement meetings but this means the entire team aren’t working.

In the estimation session, Dave (Lead Developer) shows how easy it is to set up Release Please which generates release notes. I say, “isn’t it quicker just to check it in now than to write the User Story for it?”. Dave agrees, but he creates the User Story and asks to story point it anyway. To my surprise, the team votes were 1 2 2 3 3 5. So we literally saw Dave do the work within seconds and yet the only person voting 1 was me! Lucy said she “went high just to be safe” (voted 3), and Marek said he “went 5 because I’ve never done it before“. Mate, Dave showed the config file from another repo. It’s like 10 lines to copy and you change nothing.

I suppose different teams vote differently, but from past experience 5 usually means 5-8 days.

What is the point even of Story Pointing when people are voting 2,3,5 for 5 minutes of work? Can you trust any estimates? I feel like it’s a complete waste of time.

By the end of the week we have added Sonar, and set-up Playwright testing framework on the UI side.

Week 4 – 8-12th September

We had meetings all day Monday. One was a demo about the prototype that was written by AI from Chris (Framework Lead) and Mark (Architect). We had another refinement session.

Marek asks “What is ProjectFutures“? It’s the start of week 3 for him but surely he should have been aware what product he is working on. I thought Liam (Team Manager) would have given them an Induction; it would have been the sensible thing to do.

Later in the week, we had a few more long refinement meetings. The contractors are still pointing high because “they’ve never done it before”. For one User Story, James gave a link to another repository which showed the exact method we had to add for authentication: 1 2 2 3 5 5 5 were the votes. Surely it is a 1!

Week 5 – 15-19th September

I start working on adding the Automated Testing framework; Refit to the API, although we don’t have an API, so what am I even adding!?

There was a major NPM security incident where malware was infecting several packages. On investigation, it didn’t affect us because we didn’t use any of those packages, but NPM was disabled by our security team “as a precaution” and we were told to just do some “personal development“.

Week 6 – 22-26 September

Another contractor, Ivan joins us. The NPM incident is still ongoing so we kept on doing “personal development”.

Week 7 – 29th Sept-3rd October

“You guys fly through work so this incident has come at a bad time”

Jean (Product Owner)

We are in Week 7 and don’t have any features complete. I suppose Dave (Lead Developer) was correct about it taking a long time to set up, but it really shouldn’t. I think it’s because we keep splitting tasks down and spreading them throughout the team, so one person does an hour-long task but then drags it out to fill the day. Yet one person focussed on it could just whizz through.

We also spend too much time planning which is also because we have broken tasks down too much. So we end up discussing what the scope is; where one task ends and where the next one begins, but the second one is instantly blocked by the first as a dependency. So it’s all just stupid. If you can’t get two developers working on different items at the same time, then what’s the point in splitting the feature down like that? 

I’m still trying to get Refit working in our repository but I still don’t really know what the objective is since we don’t have an API. I just want the boilerplate code I guess, but our internal package APIAutomation was showing an error about missing data.

Week 8 – 6-10 October

We had another bizarre planning session. One item had the following scores: 2 3 3 5 5 5 

James, who voted 3, says it’s most likely a 1. Why did he vote two scores higher? 5 is the majority, but then people admit it’s probably simple to implement a config flag, it’s just we haven’t got experience creating config flags for ProjectFutures. What is the point of voting if everyone doesn’t believe in their vote apart from me. 2 shows it’s easy but allows more scope if we have trouble. I suppose I should have voted 1, but voted 2 to attempt to be more in sync with their numbering.

I’m still struggling with the APIAutomation package error.

There seems to be an idea that we won’t actually open our Scribe in the “Companion” mode. It will now be launched as an embedded Webview2 control inside SystemNow. I’m not sure what the reason is because this surely means you can’t place it on a separate monitor and we will be really restricted in terms of space.

Week 9 – 13-17th October

I took a week off. Perfectly timed really since not much was happening.

The lead automation tester at the company admits that there was a bug in the APIAutomation package and I was the first to see it.

The rest of the team are adding basic UI components and test tools like Storybook and Playwright.

Week 10 – 20-24th October

I had to reset my laptop at the request of Group IT, which would then have all the new security policies. However, the update failed to reinstall, so I was completely screwed.

I called IT Tuesday morning, and after investigating if anyone else had the same problem, they asked me to come in tomorrow to reset it. At the office, they have a couple of attempts and don’t succeed, so I head home. They later fix it so come and deliver the laptop to me.

There’s some weird restriction about having to wait 24 hours before we can download applications from the Internal App Store. With meetings like the End of Sprint review and Friday’s Retrospective, I’ve gone the full week without doing any work again.

When I think back to the original plan for the project, I suppose we have missed the 2 month project deadline that the managers wanted. It’s an absolute joke that we don’t have any real functionality to demo. Liam (Team Manager) keeps saying it’s because of the NPM major incident that prevented us from doing work, but we weren’t making much progress before that.

Week 11 – 27th-31st Oct

Liam (Team Manager) keeps mentioning “version 0.5” which I get the impression was an outcome from a meeting during my week off.

So the first thing to start the week is yet another planning meeting to discuss the items in scope for version 0.5. Not only do we have extra meetings about descoping work, there’s extra work for temporary config flags. Due to the fact we are behind, we want to rush out a demo to a small group of users to get feedback which actually comes at the cost of delaying the main project by a couple of weeks.

One of the User Stories is to install the Fluent Validation package. We wasted more time talking about it than it would to take to install. I think items like that are dumb because a User Story should deliver functionality to the user, and this is just Step 1 of delivering some value, and it’s an implementation detail. It’s to check that part of the message content is valid, then we will send a Rejection if not.

I spent most of the week reconfiguring my laptop after the reset. All kinds of things hindered me setting up my laptop. The AppStore triggers installations off in the background so it is hard to know if and when they are running. There can be hour delays, and required restarts. Having to set up things locally and on WSL adds to the pain, and restoring the Linux image took ages.

The team collectively are working on Launch Darkly (Configuration Flags), and some Message Validators.

Week 12 – 3-7th November

Andrei kept on talking about adding a DateTimeValidator even though for his story, he just needed Date (not DateTime). When we asked him why he was doing it, he kept on saying he had “only ever used DateTime before and wanted consistency through the application”. We are like “no mate, some things need Date, and some need DateTime” so we eventually need both types of validations.

We have our End of Year Reviews to fill in at short notice so this is another distraction.

We completed a high number of Story Points this week, I believe around 45 when it was more like 20 in previous weeks.

Liam (Team Manager) asked what caused this increase. Then Andrei joked “GitHub Copilot“. Although he did increase productivity, it definitely took longer to review his code; it had loads of scope-creep and complexity. I keep thinking he gets ideas in his head like wanting DateTime then begins doing more, or the wrong thing – to what was asked.

I wasn’t sure why our velocity actually increased. It might just be that all the basic set-up is complete and we were doing work which was small and the Story Points were inflated, so we were getting 3 or 5 points for items that were really 1 or 2s.

Week 13 – 10th-14th November

The week started with an incredibly long planning session. We spent way too long debating how many Story Points to bring in. Do we go for 45, or try to use an average, so plan around 33 points?

I saw this quote from one of the managers. I think they were talking about costs. I hate all this business jargon:

“We’ll design a good enough path for both pppy and pupm taking into considerations user facing Flow and internal ops constraints that can be ready by Q2 2026”

Andrei and I have been adding Validators. Marek added one as well which is only his 3rd PR in 11 weeks.

James is still working on hosting issues.

Liam (Team Manager) announced that we were reducing the number of Microsoft Teams chats associated with the project. This was the first time I had heard that another team had started their work. I knew my mate Adam was going to be working on the SystemNow side of things at some point because he had told me. I didn’t know that he had started, and didn’t know what his team was called. When I asked him about it, he said he wasn’t aware his team had their own Microsoft Teams chat.

There was a demo arranged Friday to show the project so far. On the call, there were Software Architects, Safety, Marketing, Data Scientists, all kinds of managers at various levels in the hierarchy. During the introduction, the managers were constantly hyping up the features and overpromising the project. The Marketing staff seemed really excited since it is a new product  to sell. After all that hype, the development team were basically like “here is a button“.

These demos seem to be expected to be weekly but how much progress are we going to make in a week? and it’s another distraction that we have to plan for the demo. It’s like Thursday afternoon will be wasted just to put a demo together for Friday morning.

The 3rd Party that will provide us with the embedded application that handles the recording and transcription is called VerbatimEcho. They are a small company but claim to be a market leader in other English-speaking countries.

Week 14 – 17th-21st November

Dave (Lead Developer) reckons we should make sure the “elements” in the JSON are ordered, but the requirements in the user story were written with the implication that unordered elements were invalid and should be rejected, rather than our code correcting it and accepting it.

I asked the question on Microsoft Teams, mainly for the Architect to confirm, I also asked which User Story we can pick up because most of the items we have seem to be blocked. Both questions were ignored.

I started looking at a User Story that involved rejecting the message if there were extra elements that weren’t in the schema.

I started writing a unit test and the GitHub Copilot added some schema parsing code using Newtonsoft. I wasn’t sure if we could use Newtonsoft with the new security changes. I searched for it in Nuget Package Manager and couldn’t find it. I looked on JFrog directly and found something similar named but couldn’t tell what it was since there was no link to the actual package. I noticed that Netwonsoft now have moved schema validation to a new package and they charge for it. So I was kinda stuck not knowing what to use.

I’m finding it progressively frustrating that the only reason we have our Daily Stand Up at 10AM is because of James and he keeps turning up late. It’s 3 hours into the day for the Contractors so I think sometimes they sit around waiting to be told what to do. It would be very beneficial for us to have the meeting earlier at 9.

A big turnout on the Friday demo; 36 people. Adam said they have nothing to show but then Jack, the tester in their team, showed a JSON generator tool to use rather than actually using the VerbatimEcho app. Our team demoed a tab control, and some error messages. I’m sure all these people join this demo wanting to see some revolutionary AI, and we are showing them basic UI controls.

Week 15 – 24th-29th November

On Friday at 6pm, James decides to tell the team he has annual leave this week, and he hasn’t handed over the work that he was assigned, so now that is probably blocked.

Mark confirms the schema story I spent most of last week on was another AI hallucination of ours. Jean (Product Owner) has been using Microsoft Copilot to generate the User Stories.

With Andrei changing how error messages work, he has taken the initiative to update all the old messages and remove the old code. Some of the helper methods that were suspiciously AI have now been deleted.

There was one message that said 

needs to be in one of the following formats: <formats go here>” 

but we only ever specified a single format it can be. I told him to create a separate message for when it is single.

He changed the message so it would read like it will specify one format, but we still allow an array to be passed in so it looks like it will display multiple. I asked him why he did this, and he said “although there’s no requirement to support multiple formats, the code is there if we do“. This makes no sense because if we do have the need for multiple formats, we would have to change the message like I suggested. However, since there is only a requirement for singular format, then why don’t we just code for that, and support multiple later? (but I don’t anticipate it ever being needed anyway).

On the side, and on a similar note: Andrei has made some custom error logging framework. He allows multiple errors with an array, yet we only produce 1 error, then the validation stops processing. He also has a property for each individual error to hide it from the user, then an overall property to hide all errors in the array from the user. When asked why he had done this, he said that “we might change the validation to allow multiple errors” and “It might be useful to hide errors from the user“.

So there is no User Story for it; he has just invented some work and showed off. For both validation and error logging; he is saying he has implemented them in a flexible way but both will require more work to get them to work that way. They look like they support multiple items but they don’t.

I don’t know enough about API development, but I can’t imagine something like this wouldn’t exist in the framework natively. So he is doing what he wants and creating extra work instead of just doing the planned work. It’s getting so frustrating.

We are quickly getting through our assigned work though. I don’t have much experience with UI but with GitHub Copilot, I quickly got my item done which was worth 5 Story Points. Although I thought I’d got it done; but it did need several minor tweaks which then delayed it.

The architect tells us that the schema will change. So the work we did to validate the schema will need tweaking.

Chris (Framework Lead) is apparently more involved now, although I think it is for framework changes. Although the work is driven by our use case, it will allow any app to work in this embedded way. Chris is due to finish 16th January.

There was talk of a March deadline for the project.

Week 16 – 1-5th December

James is back from his week off. He turns up at 10:05 though in classic James fashion. I ask him a question at 10:56 and he responds at 12:13.

I wanted to ask Sam (UX) about a few minor tweaks to the UI but he was off.

We were talking about Version 0.5, and James started kicking off, which I thought he was talking himself out of a pay rise during this End of Year Review period. “I wasn’t aware of that, so there’s not much vision and direction to the team“. Liam (Team Manager) was a bit annoyed.

Liam was talking about getting more people on the project to speed things up.  It’s the classic management question, thinking you can speed progress by throwing more people at it. Liam sounded aware it wasn’t a great idea if it would be disruptive to train people up, and split more work across more people. However, maybe we can delegate certain tasks like the DevOps side?

Liam mentions that Alan (Head Architect) is working closer with Adam. Sounds like he’s not happy with his estimate. Alan basically vibe-coded a prototype in a week. Adam reckons another 4 months’ work is required on top of the 2 months Adam has done so far.

Sam (UX) is going on about mono tertiary buttons. I don’t know what that means. Turns out the design has changed a bit and Dave (Lead Developer) doesn’t like certain things so I have to rewrite my control I made last week… Again

Christmas Party removed a day from the week.

Managers have demoed a UX Prototype to some users to get them hyped. Probably end up getting them too hyped when we deliver something different 6 months later!

Week 17 – 8-12 December

We had a User Story to create a database, but no one had come up with a design. Dave (Lead Developer) lists the columns we need then begins trying to rearrange the tables so it’s easier if we then “partially migrate” a table to DynamoDB or something daft.

I don’t understand this idea, but assuming we did go ahead with it; I don’t think it will be any easier because you’d still be dropping 1 table and modifying the other table to add a URL column to point to the new location. So sacrificing the design now means we have a potential overhead until we do the migration, but then we might not ever do this migration.

Then he wanted to add audit fields in there like Update/Created By/On. So 4 new columns. I didn’t think you’d need audit fields in both tables because the initial write would add entries to both tables, then additional rows for that entry would just add to the second table; so you can infer who added the entry by only looking at the first entry in the second table.

Regardless, the design would come from Mark, our Architect and database expert, so why are we even discussing this in a Sprint Planning meeting? The User Story should just be “Create database” and the exact design gets added later. Despite my protests, we carried on talking about the design ourselves.

Nearly an hour went by in this Sprint Planning meeting and we hadn’t even planned 1 item, and we needed 7 to keep people busy for now.

Andrei said it made sense to create a 4 column composite key. Wouldn’t that mean that you have to add 4 duplicate columns into the second table that it foreign keys onto? That sounds bonkers. Andrei claims to have experience in database design.

Andrei then suggested using a GUID for the primary key rather than an incrementing Integer. I don’t see why a GUID has any advantage over an Int, but Ints will have lower storage memory cost and be easier to read.

After 1.5 hours, we had 13 points planned in (we wanted about 30).

On the next item, it was Marek’s turn to chicken out, who voted 5 “I am happy to go for 3 or a 2“.

I noticed James was late to every Daily Stand Up by at least 5 mins. Then he’d disappear for long periods of time throughout the day.

I pick up an item to do with authentication, but we need some Azure Client ID and Secrets which only managers can create. I requested one from Liam (Team Manager) but he was busy. The next day he was off. The next day he was busy again until the afternoon but I managed to get some of his time. Dave (Lead Developer) then decides we can reuse some credentials for now so Liam wasn’t even needed.

Week 18 – 15th-19th December

We’ve had many Town Hall Meetings that have caused disruption. 1 hour long presentations of business guff.

James still keeps being late. We moved a Daily Stand Up to 10:30 because of a Town Hall meeting clash but he was still late. The next Town Hall a few days later meant we moved it to 11:00 and he was late again. It illustrates that Liam (Team Manager) does a bad job if he isn’t having words with James.

I don’t recall the last time I saw Dave (Lead Developer) actually work on something on the board. I never understand how we get random issues like a single unit test will be failing, and Dave is the one to notice it. He will then fix it and only tell people when he has the PR for it. Then there was a package he replaced due to some security issues, then another package he replaced because it had a bug that sometimes affected us so took the initiative to find a better one. It’s almost like he is inventing problems.

For my authentication work, I showed Dave what I had done so far. I created a temporary method to create a bearer token and Terraform to store the secrets. After looking at the current authentication package and API we need to call; Dave says the package actually handles the authentication for us so my User Story can just be binned.

James is taking ages to create the Transcript UI. I thought he was good with UI stuff but I suppose he is really a Tester and not a Developer. He also keeps being late so maybe he isn’t working. He keeps claiming GitHub Copilot is generating bad code, and he is struggling to sort the CSS out himself. I am very doubtful of this unless he isn’t asking the correct questions.

Marek and Andrei are doing some database work but it sounds like there’s a bit of crossover or dependencies with their work. They also discuss how to do unit tests which shouldn’t have a dependency on a database. Using EntityFramework you can use the InMemoryDatabase if you want some simple Integration tests, but then Andrei claims it doesn’t work with DynamoDB or something. I thought we were using SQL anyway, so I’ve no idea what is going on.

Marek asked about if he should check in the DB schema changes but I said something needs to be checked in to create the tables on the different environments (INT/PRD) because they aren’t gonna create themselves. No one in the team responded to me; an awkward silence as if I had said something stupid.

They keep using the term “Migration” as if everyone knows what that means. It does seem like it is a EntityFramework name, but I hadn’t heard of it before. It sounds weird to talk about Database Migrations when we haven’t created the initial database – there’s nothing to “migrate”.

Week 19 – 5th-9th January

Most of the team were away over the Christmas period. The 2nd was a Friday so pretty much everyone had that off as well. So a full 2 weeks are basically skipped.

The 5th was the real start of 2026. Liam (Team Manager) said James was off so “everyone was here on the call“. I pointed out Lucy was missing. Lucy then joins. James turns up 15 mins late. Great start from Liam; he is in charge of approving annual leave and has no idea who is supposed to show up to work.

7 items are still waiting for reviews to complete.

4 are Ivan’s and he should have had them ready over the last couple of weeks but they actually still have outstanding comments to resolve on them.

The Sprint Planning session was the usual debating the UX for a control so we planned nothing in by 11pm. We agreed to focus on clearing down the current items on the board at Liam’s request who seemed angry at the delay. But it’s really all on Ivan, so we are sitting around not doing anything. Dave (Lead Developer) said there will be a proper planning meeting before 2pm. It didn’t happen.

On Tuesday’s Story Pointing session, there was one item where everyone voted a 5 except me who voted an 8. Since the team is in agreement, it makes sense to go with 5. However, Andrei then says it could be an 8. It still makes sense to go with 5 because it’s still the majority vote. Dave (Lead Developer) then comes up with the idea of creating a “Spike” to investigate the scope of the item, then we can more accurately point. Absolutely nonsensical idea.

There was another item where we discussed for a while. The majority vote was 2, with a couple of votes for a 3. Dave votes 2 too. However, as Team Lead, Dave overrules and gives it a 3. There’s no real consistency in the process.

Another story involves clicking a Text Field and if the transcript panel is open, it then scrolls to the evidence in the Transcript and highlights the appropriate text evidence. I thought it was pretty standard, like 5 points. However, we then split it up, and score the scrolling 5 points, then the manual controls (next/back buttons) was a 3. So we are splitting work up too small, pointing really high, and we cannot often share the work out because there’s a dependency on each other.

In today’s jargon news, there’s a call for the “Provenance Derivation and Semantic Match Criteria within class 1 criteria

I had my User Story mostly complete by Monday evening but then I made some changes suggested by Copilot on the PR but they didn’t work. I struggled to understand the Jest output and I couldn’t see a way of debugging the tests. GitHub Copilot stopped working with some random network issue but I could connect to everything else.

On Thursday, I had it completed, so I picked up a new item and got it done within an hour. I encountered a random test failure but couldn’t understand why my changes had any impact on it at all. A bit of a weird week for me.

Week 20 – 12-16th January

James is on annual leave and hasn’t finished his story so Dave (Lead Developer) agreed to look at it Monday.

I saw a few PRs from Dave which were fixing something to do with a build (code coverage reports) – completely unrelated to what James was working on.

I picked up a related story which was to create a navigation control. It was really simple, just a coloured border around some text with 2 chevron icons to interact with. So I got it done within a couple of hours. It had a dependency on the item that James/Dave was meant to do so I think some of the functionality is blocked.

On my other item, only Lucy reviewed it, and pointed out some changes which were a good suggestion, but we could have just fixed it on the next related User Story rather than delaying this one.

On Tuesday, I pushed up another change for Lucy but I was mainly idling. Daily Stand Up was meant to be at 10 but we had a “Timesheet meeting” so moved it to 11:15 which seemed a bit excessive especially when the timesheet meeting didn’t last as long as scheduled, so I idled a bit until the Daily Stand Up. Then people were talking about all kinds of things so it ended up lasting around 35 minutes. So by 12pm no one had done anything productive.

We seem to organise the work in the least efficient way, have too many meetings about it, at inconvenient times, then people just idle between these events.

On Wednesday, there was a long call, estimating/refining work for next week.

As a team, despite the distractions, we ended up making quite a few changes throughout the last 2 weeks and got quite a few items checked in on Thursday ready for the Friday demo. These were some updates to API Schema changes, some controls to scroll through the Transcript evidence, some database changes, and a few tweaks to the build pipelines. Enough to talk about but still not exciting for the managers.

Week 21 – 19-23 January

In contrast to last week, not much seemed to happen.

I worked on the Transcript Navigator control.

Dave (Lead Developer) made a Mock site improvement.

Lucy finished off Ivan’s work.

In Week 16, Liam (Team Manager) asked about increasing the team size. We actually went with the idea of delegating some DevOps tasks to someone. So Richard was now in charge of tweaking or configuring deployment. That’s one less distraction for Dave.

Week 22 – 26-30 January

I created a drop-down menu. For some reason, the usual “fire event” in the unit test doesn’t work and I had to “use user event”, so that was a day wasted in the confusion. Lucy added a few comments on my review, then after I addressed them, she added some more for something completely different. Valid points but frustrating she keeps moving the goalposts recently.

On Wednesday, we were told that the Senior Managers wanted the release one month early. Do they know how software works? It’s done when it’s done unless we cut out features that aren’t Minimum Viable Product (MVP), but A, we already cut the scope, and B, we will release something more underwhelming so it’s a harder sell.

Maybe there was some pushback, because Liam (Team Manager) declared the date as 27th March so the old claim of a March deadline is still accurate.

In the Story Pointing session, for one item the votes were:

5 5 8 8 13 13

So a complete tie, but 8 would be a good average. Lucy tries chickening from 8 to a 13 because Dave (Lead Developer) had voted 13. However, Dave says we should average to an 8.

Week 23 – 2-6 February

On Wednesday, James said he’s been waiting a week for VerbatimEcho to answer a question. Liam (Team Manager) said it was only sent Monday and VerbatimEcho had the day off. Sounds like the typical communication problem where we have to tell a manager like Liam, and hope that he then asks VerbatimEcho. It would be better if we could just message them directly.

I keep finding that User Stories are vague and not sure of the actual scope. I picked up one that involved sending messages to SystemNow but we don’t have the integration yet to test it so it’s all theoretical.

Week 24 – 9-13th Feb

No real progress from me. One User Story kept on running into merge conflicts as the team checked in their items, then GitHub Copilot was re-reviewing it, and coming up with new comments on code it had already seen.

Meanwhile I ran into problems with WSL, with networking failures and Extensions failing. I think part of it was that my current version of DotNet was then on the block list for my employer with their new strict policies.

On a call with Andrei, who claimed he could help with WSL problems, he insisted part of it was the “obj directory”, but he is wrong because that’s for the compiled DLLs and this was an error on VS Code load, and nothing to do with launching our app or compiling it.

He then said he writes code on Visual Studio on Windows, and then copies the folder into Ubuntu to run via VS Code. I asked him repeatedly why he does this and it was just that he is “unfamiliar with VS Code”. Yet he is still using VS Code to build and run the app isn’t he? and this is an incredibly cumbersome and tedious process he is using.

With Dave (Lead Developer)’s changes to a Mock API, we now need a different command to launch our app. We also need to specify Database Config on the API side even when a database isn’t needed (I think this was Andrei’s fault). It annoys me when colleagues change the process but then there’s no clear guide to getting everyone set up.

The first pilot customer is live in pilot with the “Partner API Scribe version”. No one in the team seems to know what that means. They definitely aren’t using our API and app and we thought there were only 2 teams on our project, and the other team doesn’t know what it is either.

Week 25 – 16th-20th February

I’m trying to remember how our Refit Automation even ran. I added some ENV variables the GitHub workflow used but it was still erroring. It was calling out a specific variable so I added it and tried again. Repeat about 5 times. Then it moans that the INT URL isn’t available but I want to run this locally, but how do I do that?

I tried to test the API with the JSON files we had on Confluence which were examples that matched the schema. One of the objects cannot be parsed. I manipulate it to match our code. I get loads of errors about invalid date formats and missing attributes

This is a great example that if you mock everything in tests, you can end up creating nonsense. Why aren’t we using the JSON from Confluence in our tests? The JSON format is a key part of our project and has been massively wrong all this time. Ridiculous.

The next day Dave (Lead Developer) writes almost a blog post on the issues and the team reacts surprised like they didn’t know about it. Dave seems to be getting credit for finding this mess but it was me that spotted them all.

In today’s “Andrei being weird” story:

Lucy creates a PR with some hardcoded test data in a control she is working on. She declares it in the description. Copilot Code Review comments on it, stating that despite her declaring that it is temporary data in the description, it is better to be declared in an inline comment in the code to state it is temporary i.e. a TODO comment. Fair enough. 

Andrei then comments saying “add these strings to the Translation file” as if at some point in the future we will translate it to German. It’s temporary code mate. He hasn’t paid any attention has he? He is just looking for text strings to write that comment on. A few weeks ago he correctly flagged up some hardcoded button text that we needed to move to the translation file, but here he looks like an idiot for saying it.

In Week 17 we had the idea of adding 4 audit columns which seemed excessive to me. This week, Andrei has added initiatedByUser column to the Database even though we have “CreatedBy”. Dave (Lead Developer) points out it’s redundant and asks where the requirement came from. Andrei says we need to “stay flexible and consistent with the database design. Relying on CreatedBy might create additional work later if it can be triggered by 3rd parties“. What is he on about? He’s gone mad.

On another PR, I noticed his database code doesn’t always update the UpdatedOn timestamp so I asked “if that was intentional because I didn’t know what scenarios the code was executed”.

He replies “you’re absolutely right!” then explains the exact scenario back to me, clearly an AI response. He then asks “can you confirm this is not expected?

It’s frustrating because I’m asking him that exact question.

He then screenshots it and sends it on Microsoft Teams which angered me further. If he gives me a link, I can get to the review to have a look, but a screenshot means I have to navigate there myself. He then says “please resolve if it’s not relevant, or we can have a call“. How do I know it’s relevant? He hasn’t given me more info. He loves Teams calls. So I tell him if the code can be called when the same user edits, then their new UPDATED ON isn’t logged. The line needs to be moved out of the IF statement. I also asked why he is pasting AI responses to me when he hasn’t understood what it has written.

He then explains it’s only really auditing if the user’s display name has changed and we update their name. So that sounds fine then, my comment was irrelevant after all. However he then says “it needs more thought because the change tracker object is excluding them“. So I say “I’m happy to mark the comment as irrelevant, but if he’s spotted a problem then I’ll wait for him to fix it before I approve the change”. He immediately says that he “can’t see any issues with it”.

However 30 mins later he sends me some code on Teams which implies there is a problem but most of it is commented out so it’s quite unclear what the fix is. I ask him where the timestamp is updated because I don’t see it in the code he pasted, and he gives me some line numbers but there are no line numbers in his snippet, so I don’t know what he is on about. He asks for a call, I think it’s a wind up.

Week 26 – 23rd-27th February

I pick up a User Story that was something like

“as a user, I would like customer details in the API call”

The user doesn’t want that. The user knows nothing about API calls. This isn’t a User Story in my opinion but most of our User Stories seem to be like this. Also, it has no detail to be clear what the scope of this work is. What customer details do we need? Which API call is it included in? I stared at the code for a while trying to make sense of what it could mean. I eventually wrote a single line of code but is it correct?

I kept on running into environment issues. Sometimes I cannot log in, sometimes it’s slow, and sometimes says ApplicationID is missing so the app doesn’t load. Then the data I wanted didn’t exist, but did the next day but I hadn’t changed anything.

Dave (Lead Developer) keeps doing what he wants. Multiple surprise PRs each day for things not on the board.

Ivan is going on about Redux but we aren’t using Redux. Andrei is going on about “caching enums” but I don’t know what that means.

VerbatimEcho are still violating our schema. You’d think they would have implemented it pretty quickly and got feedback on it, but we’ve been coding against a mock app so we’ve rarely been testing it.

VerbatimEcho changed their URL so now we need to add that to the firewall and security aspects like CORS.

Adam says his team can’t merge into Main until next month and only then can we see if our project works. 26 weeks in and we have no idea if it’s going to integrate together. Also, surely we need to Integration Test before they put their code in Main? When it is in Main, the code will then be in the next release. We really have no idea how to release software anymore.

Week 27 – 2-6 March

On the 4th we found that VerbatimEcho have said they won’t be complete until 13th March. Adam’s team will be ready by the 17th March.

Our app doesn’t work in the Staging for some reason.

The Project Manager suggested we should have had Daily Stand Ups with VerbatimEcho much earlier on. Isn’t that her job to approve each of their milestones? You’d think there would be a contract and she would be in charge of managing the project.

Marketing Manager – “We are changing the game when it comes to AI – end users watch this space

I keep running into environment issues which are a massive inconvenience to developing; app not loading, product search giving me 400 errors.

I picked up a bug to fix which needed debugging but no one in the team actually knows how to bug a Typescript app. They all just do “console.logs” like noobs. I managed to get it working after a bit of trial-and-error but annoyingly it launched a browser window without the Keeper extension so I couldn’t use the saved credentials to log in. Instead, I have to keep a different browser window open with Keeper logged in and copy each username/password/2FA across. Since the bug actually happens in another repository, I tried changing the package reference to point to the local files. However their repo refers to many other packages using this “workspace” keyword which then doesn’t build, presumably because they aren’t in the main repo’s workspace. Nothing is ever easy is it?

I later realised there were similarly named packages and I was actually updating the wrong reference. With the correct one in place as a file reference, I then got it working. Although I couldn’t get the debugger to work with it so had to resort to “console.logs” like the rest of the team.

Week 28 – 9-13 March

I realised that the issue was caused by one of our changes many months ago and seems specific to certain Product Codes and only when this property we added was set to True. Therefore it would only impact our project which is why no one found it earlier.

I then noticed that GitHub Copilot had added a comment on the original change which the developer ignored. I added the suggestion in and it worked. Although some unit tests were now failing and it looked important. With some refactoring, I made the change simpler and the unit tests passed.

The functionality in general seemed a bit janky though and the method was triggered multiple times when it didn’t need to. The main problem was a hardcoded 2 second timer which I think was to basically ignore some of these refreshes. I asked Copilot to fix it and it added even more timers! I asked a Premium AI model and it came up with a fix which was a bit copy and paste, but it works.

However, Ivan has been making large changes to this Add Product functionality. From what I understood, the current other issue is that you can select a Product Code, then go back to edit it; it hasn’t updated with your changes. Some Product Codes have more fields to fill in, and these weren’t set to your current selections.

I thought Ivan was fixing this “save state”. I tried our changes together and it wasn’t retaining the data or updating the data. Also when you clicked a new row, it also added the same selected product code there too which was obviously wrong. I told Ivan about it and he claimed it works. Then we ran into environment issues where the Customer Find wouldn’t work, or the Add Product picker wouldn’t work, or you just couldn’t log in. So that was a day written off.

We had a meeting to discuss outstanding issues and it seemed the other team members had found various issues. Some we needed to fix now and others maybe could be delayed until after the initial pilot release. Managers were raising concerns that we have never seen it integrated all together; it’s always been tested with mock data and we have interactions between SystemNow->ProjectFutures->Third Party, so there is a massive scope of things to go wrong between.

We had a demo where Alan (Head Architect) said our implementation was wrong. When the user selects a Product Code, then the text should be replaced. There was a huge message thread back and forth where part of it was debating about the behavior and others were debating about the word “replaced”. Alan was insisting that we never “replace” but I didn’t understand his point.

I can’t really picture how his requirements work in all scenarios. If you select a code then replace it, then what text does it then show? does it then revert to the original text? Then what about a standard example where the Product name isn’t as descriptive. So start with the text:

"High chair"

then they select

[Chair]

do we then display

[Chair]

then it's up to the user to type the "high" back in, or select the appropriate code.

Jean (Product Owner) asks how to log into our test system and says she has never seen any of the functionality herself. Week 28 and the Product Owner hasn’t seen anything.

Week 29 – 16th-20th March

We had Sprint Planning at 10 so that’s basically 1 hour written off. After 30 mins, we had an ad-hoc department meeting so we had to attend that. Then not everyone was free to resume planning straight away.

Rather than simply Sprint Planning, we ended up Story Pointing and/or prioritising items. We were going to drop one item from scope after discussing it for 10 mins. The discussion was that maybe it wasn’t needed for the Pilot release but was valuable later. Then Andrei said it was ready to review, he just hasn’t assigned it to himself and was working on it last Friday. Why didn’t he say that straight away?

Andrei keeps going on about the order of an enum and pasted a whole essay from AI about it. When looking at his changes, he has defined an enum to represent Gender with values F M I U but for U (unknown) should be 0 as that’s the default value. So instead of putting U first, he has then just specified values 1 2 3 0. When Marek asked why he couldn’t just put it first, he says that there’s some “migration problem with Postgres“. My thought was that if Postgres has its own enum with F M I U then now we have a problem because the database uses F as 0. Turns out we actually don’t store the number value, we only save a single character. So it seems fine, but then does the order of the enum actually matter then?

We then talk about another item Marek was currently looking at. We debated 3 or 5 points then shortly after the meeting, Marek sent it for review. So he could have just put some points on it without discussion because it was also already complete.

We were about to be ready to integrate SystemNow launching our Scribe app containing VerbatimEcho.

James: “With stakeholders wanting to access the app now via SystemNow and us yet to actually see it working ourselves, it might end up being a bit of a disaster. Is it worth mentioning to avoid any disappointment when it doesn’t work yet?”

Dave (Lead Developer): “we’re currently in the period of integration testing so it should be expected that we are finding issues.”

We had a meeting with all the teams to talk about the remaining work. Or that’s what I thought it was for. It turns out there was an issue where the other team need to refresh the browser 3 times for VerbatimEcho to load properly.

Alan (Head Architect): “why do you have to refresh the browser 3 times for it to appear?”

Jack: “Sometimes it is more!”

It completely cracked me up when he said that. He said it like it’s just an accepted thing. I mean, I suppose he has put up with it for a bit without previously flagging it as an issue but it’s a serious issue if that gets released like that.

Then we were told our App wouldn’t work at all in the Staging environment which was due to some missing config from the Users team so it wasn’t added to the list of “accepted products”. Then there was another intermittent networking issue.

Chris (Framework Lead) pointed out the communication flow was

SystemNow->Companion->B2C Auth->Companion-CA->VerbatimEcho->VerbatimEcho(Auth)->ProjectFutures-Users->VerbatimEcho

Sounds horrendously complex.

We did more of our classic story pointing. Four votes for 2, Three for 3, so you’d think the majority wins, but Dave (Lead Developer) went with 3. There was one later on which was even more where it was five votes vs two votes, but two won.

Dave is surprised that replacing a Product Code doesn’t work. I literally told the team this morning on the Daily Stand Up, and everyone apart from Lucy seemed surprised at it. Ivan kept quiet so I didn’t know if he was actually there, but I told him to fix it because he didn’t have anything to do and I needed to finish off another item by addressing the code review feedback.

Week 30 – 23-27 March

Dave (Lead Developer) is rushing quick wins out to appease Alan (Head Architect) and Liam (Team Manager).

Tuesday was a write off. I kept on running into different problems; couldn’t log in, or I could but the app wouldn’t load, or horrendously slow to load customers etc.

We are starting to integrate with the real VerbatimEcho app now. Some bugs we are finding now because they are blatantly obvious should have been spotted earlier.

If we add new Headings in our app, they error when sending to our API due to missing fields.

We aren’t populating the “Main Product ID” of the section, and not populating Product IDs for headers that aren’t “Main Product” which contain Product IDs. Or that’s what Andrei claims.

We aren’t setting the Provenance for new items. Andrei reckons we should just remove the validation on the API rather than passing correct data through the app. His reasoning that it is quicker. It’s going to be roughly the same effort in my opinion. If it was a complicated object to set, then it’s only going to be an extra day or so. If we do his suggestion, we have to come back to it in the future, undo the deletion and fix it properly anyway.

When AI fills in a row, there is a button you can click to “Show Source” which should open the Transcript and highlight the appropriate sentence(s). We have the button but it doesn’t do anything. I added the implementation to make this Show Source button work. Lucy is working on removing the button. Classic. This is the classic thing that happens when you do your own thing rather than what’s planned on the board. My item was on the board.

In the Daily Stand Up on Wednesday, Dave (Lead Developer) was attending another meeting so I spoke up. I tell Lucy where the prioritised list of items is. She wanted to work on the bug James logged about not populating values automatically. I said “we cannot do it without having a guarantee that the units are the same”. For example, you could have a size listed in cm or metres which is the same metric but a different scale (as in 1 metre is 100 cm). We could do a simple text match but it might not work in all scenarios. (e.g. match “cm” but the unit is sent as “centimetres”).

An example in another context is “M” could be “metres” or “miles”. There’s thousands of possibilities across all types of products so unless we verify all of them and have an understanding of the metric then we cannot do it. If we get the 3rd party to use a standard ID associated with the measurement then we can guarantee it matches.

The next day she insists on working on it. At first, Dave (Lead Developer) seems fine with it but then realises we had recently said it wasn’t safe. He then explains similar scenarios I had explained yesterday when Dave wasn’t on the call. Despite that:

Lucy : “Shall I carry on working on it? I’m not sure if I can continue

James : “We can populate the values at least, then we know its unsafe

What is their obsession with doing this work? We have probably wasted 30 mins at least explaining why it’s not safe to do it.

Week 31 – 30-3rd April

Adam said some of our events aren’t working correctly.

VerbatimEcho say they changed start_finalization event to camel case to match the rest of the events. Dave (Lead Developer) says we’ve been using startingFinalizationProcess which used to work. So he doesn’t know what they are talking about since that is already camelcase and has different wording.

Jean (Product Owner) is seeing it integrated for the first time. The side panel is small and when extra warning banners are shown it squashes it vertically, cutting off the buttons and therefore unusable.

VerbatimEcho recording is currently not working. It intermittently records what you are saying so either misses the entire sentence or has several missing words in each sentence. It’s extremely frustrating to have to say loads of statements to hope you can fill the minimum words to get to the app. We really need to go back to our mock app but it needs some extra config changes or something to work again.

In Story Pointing, there was one item where everyone voted 5 apart from Lucy:

Lucy: “I was thinking 5. I’m not sure why I voted 3”

I don’t understand why she has to lie or chicken out of her vote every single time.

We did tell VerbatimEcho yesterday that their audio recording suddenly barely worked. They didn’t seem too fussed about sorting it. But since that is step 1 in the process, it should have been high-priority.

After chasing it up, VerbatimEcho then said it was only present on the Developer environment which is “vulnerable to instabilities“, and implying that contractually they had no obligation to fix it as a priority. I think “vulnerable to instabilities” just means it gets new features first, and therefore the bugs first; so he is just saying “these things happen”. But surely the entire feature not working 99% of the time is somewhat urgent, at least so we don’t think a company that specialises in speech-to-text software aren’t a bunch of cowboys when the recording feature doesn’t work. You would think they would have automated tests which would catch such a significant problem so it’s impossible to release like that.

Regardless, he was lying, or misinformed. It also affected their Production environment so it’s a shambles of a process if a broken feature can be rolled out like that.

Another problem being spoken about is that there is roughly a 125 consultations per hour limit due to a Third-party API rate limit that VerbatimEcho are using. VerbatimEcho say they are making around 40 calls per consultation on average. Our Software Architects are quite concerned that this is a major flaw that we need a solution for.

Week 32 – 6-10th April

James says he needs help, then was gone 11:30 to 12:30 when he finally replies to me. I asked him a follow-up question then he was gone till 1pm. I was gonna ask Lucy but she was gone for 1 hour. Another question, another 45 minute wait. At 4:15, I asked him another question. I noticed he’d been gone since 3:30.

On Wednesday, UsersAPI made a change which was reported at 10am but might have occurred earlier. It showed BLOCKED when attempting to log in. Lucy and James only noticed at 12:15 which makes me wonder what they have been doing all morning. UsersAPI team reverted it but it didn’t build.

People are spamming the channel with bug reports. We’ve got Jean (Product Owner), Safety Officer and UX designer testing it out. If only they viewed it earlier rather than when we were supposed to be finished the project.

On Thursday, we had official confirmation that the “Autopopulate units” idea (which Lucy keeps insisting she works on) cannot be done safely.

The “End of Development” demo was scheduled today. We now have loads of bugs to fix so we delayed the project once more.

The Config team added a new value without telling anyone which resulted in no apps when logging in. So that was quickly reverted Friday morning but blocked most teams for a few hours.

The Friday Retro was cancelled for the 3rd time in a row. (We didn’t have a retro again in subsequent weeks either).

Managers are very concerned with the slow performance from VerbatimEcho. VerbatimEcho claimed their production environment was fast, but we say otherwise. I saw Mark’s claims of “7 seconds app load” and “3 seconds customer load”.

Week 33 – 13th-17th April

It was Bank Holidays for the Contractors so some days we had fewer team members.

Usually we had a process of:

  1. Developer makes a fix, 
  2. team members review,
  3. a dedicated Tester tests it works, 
  4. then demo to the Product Owner for the final sign off.

Currently we have: 

  1. Developer makes a fix, 
  2. team members review, then that is it. 

So that is why the PO, UX designer etc keep finding bugs or missing requirements.

We were discussing making a change to our process but then Liam says:

“There’s absolutely no reason that the person who fixed it cannot verify that it works”

Liam (Team Manager)

Which is just completely against the usual software development process. Any experienced person in software development knows that.

Liam goes on about how we’d have to track metrics on how long it takes external reviewers to approve. His aversion was also that it would hurt our velocity if things got delayed or reopened. This is also dumb. Software Quality is about “delivering the right thing”, not “delivering the right thing fast”. He is just chasing “velocity” which results in the wrong thing fast. I don’t think we have gone fast though, we are massively delayed.

It sounds like our managers have had strong words with VerbatimEcho about their shoddy performance. It wasn’t just on app load, but the time it took to process the recording.

They came back with a detailed PDF hyping up their technology:

“Following extensive benchmarking across six models and multiple commercial providers we can confirm that the current implementation is performing as close to state of the art as possible.”

VerbatimEcho

Thursday morning, I had the “no headers yet” control ready for review and told the team in the Daily Stand Up. On Friday I made it clear that I still need 2 people to review. By Friday 5pm, no one had reviewed it, and I had my new item completed to change the Enter/delete/backspace behaviour in the editing controls.

Week 34 – 20-24 April

Lucy has the week off. Andrei logged a bug about needing to double-click for the caret to appear in the editing controls. I didn’t notice that problem on my branch, or on Main, so I got his branch that was “Ready for review” and saw it happen every time. I wrote that on his PR.

Wednesday; Pilot release is delayed again; pushed back to the 5th May. Ivan is ill so took a day off. He did a Classic developer blag update “Rebased… begun investigation”; so you did nothing then.

Adam wanted 2 weeks for regression but has been told they have 3 days. Liam (Team Manager) says we need more people in the team, which we definitely don’t. Our current size is the problem, since it means we split the work up too small, and we end up dragging items out with the usual Parkinson’s Law.

Andrei still hasn’t fixed his bug but he’s still saying it needs a review when in the Daily Stand Up.

I noticed a loading spinner on the VerbatimEcho finish button. It was positioned slightly off the control which was a really weird choice.

On Thursday, Andrei says he “tried to sync” with me about the issue I pointed out on his PR Monday morning. I never noticed the message and he had never chased up. So he has been blocked on it for 3 days but only saying that now.

Our contractors began having problems with our VPN which meant they could no longer even connect to GitHub so a couple of days were a write-off for them.

We discussed in the Daily Stand Up that the new VerbatimEcho spinner is due to the “Real Time Fact Extraction”. Since we complained about how long it took to summarise the recording, they had changed it so it starts working as you speak. It’s just a waste of processing if you cancel, or accidentally leave the recording running. If there’s a slight delay with the final words, there is a delay before you are allowed to click finish. The UX designer absolutely despised this. I suppose it doesn’t look good and can seem frustrating. The alternative is just to let them click finish and just have the main loading screen like they were showing previously. Suppose that would probably be the most logical implementation.

Some sites are already trying out VerbatimEcho without the integration with our software. I’m not sure what version is on live because they are constantly changing the text it generates, yet claim to have loads of customers around the world. So what does their currently released software do?

We’ve had some feedback on the VerbatimEcho feature in live compared to their biggest rival. VerbatimEcho:

  • Doesn’t pick up technical terms as accurately
  • Doesn’t understand other languages
  • Often duplicates entries
  • Negation statements not understood

Liam (Team Manager) says the negation is a massive issue and declares it UNRELEASABLE

Week 35 – 27-1st May

Our contractors are still having problems with our VPN. Liam (Team Manager) asks to describe the problem in detail so he can escalate it to Group IT.

Andrei says he is following steps to add “New Company”. Dave (Lead Developer) points out his screenshot says “Previous Company”, so he is using the wrong credentials.

Andrei says he cannot add “Previous Company” and shares a screenshot showing he has typed “Previous Company”. No error message or anything. Just showing he has begun to do something dumb; using “Previous Company” rather than “New Company” credentials.

Marek then acts surprised that deleting “Previous Company” allows him access to GitHub.

There did legitimately seem to be a problem because they couldn’t sign-in with “New Company” credentials either, but removing “Previous Company” from the application allowed them to connect to GitHub at least.

I don’t know why the contractors were set up differently. They all claimed they had always been set up with Previous Company credentials in the VPN and hadn’t been told to switch at any point.

There were a few other instances where they asked how to get access to things like Dynatrace. We told them it used “New Company” credentials but they clearly tried to use “Previous Company” then posted a screenshot showing as such.

We had a surprisingly long refinement session. We are supposed to have finished the project; but then there’s all this work I haven’t heard about. It all seemed like the most boring work. Some UI changes or Background jobs to delete data when past the retention policy.

Andrei finally fixes his “focussing on first row when you switch tabs” issue. It’s full of hacks. I tell Andrei that 150 lines of hacks probably isn’t a good idea. It’s messy and going to cause more bugs.

On Monday Alan (Head Architect) starts kicking off that the “associated text” bug isn’t fixed. It has to be fixed by tomorrow! Tuesday morning it was! Were VerbatimEcho just holding onto it all this time? Or was it that easy to fix?

After the Tuesday Daily Stand Up, Dave (Lead Developer) calls a meeting about Andrei’s change. I say that it could most likely be fixed in the “Add-Product” package. Dave agrees that’s most likely the case. We agree to check in the simplest fix for Andrei’s item. A new one will be logged for the edge cases.

Ivan has picked up the bug Andrei logged about double-clicking. You know, the one I told the team was only in Andrei’s branch. They now agree that the reason why Ivan cannot recreate it is – that I am correct.

Apparently, we’ve given VerbatimEcho a deadline of tomorrow but I don’t understand how we expect them to get it all perfect.

  • Inconsistent splitting
  • Qualifier duplication
  • Hallucinations
  • Not using preferred product codes
  • Duplication of comments
  • Some comments not descriptive enough
  • Wrong product codes used for certain items

Check out this confusing series of statements from Alan (Head Architect)

Alan says it crashes when adding value codes to the Main Product Heading. You shouldn’t be able to add value codes there, but he specifically says “we must not SEND values”. The logical way of phrasing this would be to say we must not allow the user to ENTER values (therefore we wouldn’t send them). Saying SEND implies it’s fine for the user to enter them.

He wants it fixing, hopefully by tomorrow.

Dave (Lead Developer) says it should be handled natively by the Product control we used but we got told to set the property to MainProduct=false because we didn’t want all the other qualifier fields which aren’t shown when MainProduct=false. As a side-effect, it also means it allows entering values as an unintended behaviour.

Alan asks when we do send the values, does SystemNow ignore or crash? But he just saw it crash so why is he asking?

However Adam reckons he added validation code to ignore values in Main Product Heading; implying the crash is a different cause.

Alan declares everything is fine then: “no change necessary“.

We are all like “wait…what?

Dave says we shouldn’t just “stop sending” values, or ignoring values. The actual fix is a third option – to stop users entering them.

Jack confirms it does crash. Alan says that it’s not possible to add a value to a Main Product Heading in ProjectFutures (contrary to his initial claim where he said it crashes when you do that).

James shows a screenshot of the Main Product Heading having values in ProjectFutures.

Dave repeats that the cause is that Main Product Heading allows values to be entered due to us setting MainProduct=false. It needs more thought to be able to hide both qualifier fields and values fields.

Alan repeats, (probably in an aggressive tone) “you cannot add values for Main Product Headings, full stop.”

Alan then uploads a video of him adding values to Main Product Heading codes and it crashing on submit.

On Friday, Liam (Team Manager) questions why Andrei’s item has been thrown back in the New column. Dave says it will be fixed as part of the additional bug we logged. I interject because that makes no sense, we created the bug to fix what wouldn’t be fixed as part of the main change. Andrei should have checked in a simple fix, and not abandon it. So I take over and complete the work instead.

Week 36 – 4-8th May

Ages ago, Andrei was talking about “caching enums” which he had a Draft PR for, then just abandoned it. This week Andrei told Marek to pick up the work.

A new bug was found. It turns out we allow any type of Product Code in each heading but SystemNow actually has a restriction in some sections. If we send data that violates that restriction, then it is rejected. I thought we agreed early on that any Product Code can go in all sections; the exception of course is Main Product Heading like we discussed last week.

James is saying the API throws validation errors if you save a blank page or a page contains a blank section. Shall we add validation client side then ask the user to remove blank sections before trying to save, or just strip the data client or API side for the save process?

“Sometimes the user will forget to add data and won’t want the section to be removed. What do you think about that?”.

Andrei

I think you are chatting nonsense. Why would they want a blank section to show after saving? They can always edit it and add new sections if they legitimately forgot to add data.

On a Town Hall, one of the Senior Managers stated to the department:

“Scribe is far superior to anything else on the market”

Senior Manager

Ironically, a day later, VerbatimEcho went wild and became even worse.

Me: “Weight 12 stone four”

output:”Work off stone farm”

The inconsistency that you can do the same consultation on the same day and get different results is worrying. But then it has days where the quality drops significantly.

I also noticed “centimetres” was always “cmso the Transcript isn’t verbatim like we asked in the requirements. It made me wonder if this was why the end result was often so inaccurate – that it changes what you said, then AI can’t make sense of the modified transcription.

Another theory was that I just wasn’t speaking clearly so I put my phone next to the headset and recorded at the same time. I then generated the transcription with AI on my phone. My phone was at least 95% accurate and the words it did get wrong was a logical possibility when you listened to the recording (my pronunciation was ambiguous). Turns out great speech to text technology does exist after all.

VerbatimEcho reckon their solution is “state of the art” yet it’s usually 60% accurate to me on a good day, and completely unusable on others.

Week 37 – 11th-15th May

Liam (Team Manager) says that Pilot should start 18th May. However, he also said VerbatimEcho are rewriting their implementation due to the poor quality. Weird, I thought it was state of the art last week?

Marek and Andrei are still going on about caching enums. I still don’t quite understand what it is. Their PR described it as follows:

“Pre-cashing enums at startup removes one time initialization costs from the hot path keeping performance test clean and consistent “

and

“Isolated into a non parallel collection because some tests mutate or poison the process wide static caches. Eg warming up assemblies containing duplicate keys”.

What the hell does that mean? It sounds AI, but the spelling mistake (Cashing) makes me think they partially wrote it at least.

Ivan is supposed to fix the focus issue, but I test it and don’t see any difference. The main method that sets the focus is unchanged and there’s several places in that same file that can also set the focus, so he can’t have tested it, or even made much attempt to investigate the problem. 

The “New VerbatimEcho” date is 19th May. I can’t expect much changes if they think they can turn the problems around in a week. They plan to drop more enhancements in “phase 2” which is 22nd May which sounds even more bonkers. Why would it take several months for a terrible solution then they can fix it in a week just because we are complaining more?

I spent ages writing a handover document so someone could investigate the bug I was working on. I had a fix that worked but was a few hacks, plus some good changes just to reduce component re-renders.

Week 38 – 18th-22 May

I took the week off. I’ve been waiting for a slow period in the project, and was fed up with how bad VerbatimEcho is. I hoped they would live up to their promise and have it fixed when I return.

The team switched to a different JFrog server, which ended up being several small PRs in classic Dave (Lead Developer) style.

Another bug was discovered where: if you apply a Product Code suggestion, then uncode it, the original suggestions no longer appear.

Idempotency solution doesn’t work, or so Mark claimed. He pasted loads of claims from Copilot which Dave said was incorrect. Dave countered with his own Copilot. I think Dave was right in the end but there was a lot of back and forth and it was hard to follow. Just ridiculous that we debate using different outputs of Copilot.

On Thursday, a new firewall rule blocked authentication so no work could be done because people could not log in.

VerbatimEcho have pushed out an update that has drastically increased the description detail. Where you have said loads of content, the consultations contain massive amounts of text. It’s way more verbose than we are used to. We are not sure what caused them to change this and why it wasn’t possible before. It was soon rolled back.

There was a discussion about what to do with silences. Do they show a warning so users can consider microphone issues? What about if it’s not a silence but lots of audio interference which makes the audio unclear and unprocessable? The concern people had is showing a warning when the user intentionally pauses can be very annoying or alarming to the user.

What I don’t understand is this: surely these are “solved problems” by VerbatimEcho with all their other customers. Unless they don’t actually have any. This idea seems more and more likely doesn’t it?

Mark pointed out that 6% of recorded consultations get wrongly rejected for not having Product-related content. He also sees the old processing version numbers in the JSON, so it’s either wrong, or we aren’t actually using VerbatimEcho’s new updates.

Week 39 – 25th-29th

Monday was a bank holiday. Dave (Lead Developer) has the whole week off. Lucy and James had the day off Tuesday.

I checked my User Story that I had written the handover document for. No one had looked at it. I was fuming. I spent hours writing that document, and I really didn’t want to look at this bug again.

Andrei said he would pick up Dave’s work that Dave had told the team to finish whilst he was off. It really just needed testing. However, Andrei then said he didn’t want to complete one of Dave’s PRs for a bug fix because I had already got a similar fix in the branch for my bug fix (that I may or may not include in my actual fix), yet Liam (Team Manager) had said Dave’s item was urgent to get in ASAP, so Andrei should just check it in.

Also, I had some updates so rebooted my computer, then WSL wouldn’t work. I rebooted and had more updates, so I rebooted and had more updates. Restarting took ages, then it was the end of the day. I definitely need Andrei to check in that fix if it is needed.

On Wednesday, Andrei asked to talk after the Daily Stand Up. He said he thinks it is important to get the bug fix in as soon as possible, so it shouldn’t be checked in with my changes. I tell him that’s exactly what I told him yesterday. He then asks me to do it, even though the work is assigned to him. Hours later he then says he hasn’t got the “Add-Product” repo building due to an PNPM error. He was supposed to have been working on this Monday, what has he been doing?

I ended up completing it by Wednesday lunchtime, then when Andrei was ready to check in the PR on our repo, I told him we needed to update the version number of the “Add-Product” package to actually get the bug fix that I checked in. He ended up typing loads of words about it being irrelevant or something. I noticed that Ivan also had a fix to check in for “Add-Product”, so I thought it would just get updated when Ivan completes his work. I don’t get why Andrei volunteered to finish Dave’s work then has spent more time typing meaningless words to not do the work.

Marek is working with Andrei to remove those CreatedBy and UpdatedBy fields. They keep talking about “Orphaned Tables” and “Self Referential Foreign Keys”. I didn’t know what they were talking about at first, but I think there’s a UserTable which has these CreatedBy/UpdatedBy columns and it references a UserId. So I think when the User uses the feature for the first time, then they get a User created plus these audit columns then refer to itself. Then if you deleted the row, it sounds like it throws a foreign key violation. Not sure if that is true though; I’ve never heard of a row referring to itself. You could only create that by inserting a row but with NULL in the CreatedBy column, then instantly update it to then populate it with its own ID. 

Regardless, this is extra work due to Andrei and Dave (Lead Developer) insisting we have these columns because “they might be useful, even though the Architect did the DB design and designed it to be useful to his Monitoring tool, and he didn’t have a need for these columns.

VerbatimEcho still seems inconsistent. Sometimes missing words, sometimes abbreviating words, but nearly always misunderstanding at least one word in each sentence. What happened to this new version we were promised?

Week 40 – 1st – 5th June

One of our managers had complained to VerbatimEcho that the transcript was randomly missing out words, and completely mishearing words, which is what I’d been saying for a while but no one else seemed to have noticed it, unless they have just been chilling, or ignoring it. 

The manager said he could record the audio, then reuse the same audio and expect the same result. However, he gets different transcripts and gets different consultations when processed; even in the cases where the transcript was very similar.

VerbatimEcho’s response was:

“Just to clarify, were these the exact same audios? If the audio was recreated slightly differently, then this would make sense which is the only assessment we’re able to come to”

I’ve got an idea, why don’t you test it out yourself? Record on your phone then compare the transcript. It took me a few minutes to complete my recreation. I checked my phone recorded and transcribed it perfectly, and then compared to VerbatimEcho which had various inaccuracies on every line. Then I used the same recording for 2 more consultations and got different results. 

Now, you could say each time I made the recording by playing my phone’s recording – that the recording wasn’t EXACTLY the same because of various background noises like cars driving past the house. However, you would think the audio processing would strip out background noise and would be analysing the words and tone which is EXACTLY the same because it’s a recording. They can probably just send in a pre-recorded file anyway to get an EXACT test.

It winds me up so much that they instantly deny our reported bugs, claim it doesn’t happen, or can’t happen, then later have to admit it does happen when they finally check it. They clearly haven’t bothered investigating or completing an “assessment”. They just lie all the time.

We’ve been discussing rolling back some of the UI changes. The removal of the delete button on each line has annoyed Alan (Head Architect). Then we were talking about rolling back the change we made to group together the text areas into one larger one per section. I said we shouldn’t make changes without approval of the UX designer, otherwise what is the point of having him assigned to the project? He had spent hours designing this. Anyway, the UX designer has been temporarily reassigned to one of those stupid ideas we have occasionally; where it’s a project to discuss a new process and becomes top priority and everyone is completely ringfenced. Under no circumstances can we contact him.

I realised the Customer API was broken, but couldn’t see anything posted in the Environment’s Team’s channels. This would affect everyone working on ProjectFutures. I asked my team about it. 30 mins later, Andrei says the integration tests are failing on PRs. I thought it makes sense if the Customer API is down because that is a prerequisite. 30 mins later, Dave (Lead Developer) says he’s re-run the tests and hoping for the best. But James says it’s most likely that an API is down. It’s almost like my entire team didn’t see my message.

The next day Dave says he’s noticed the API is down. Then other teams begin to complain about it too. The Customer API team admits they pushed out an update. 30 mins later it “is fixed” because they rolled it back. Except that it wasn’t fixed because you couldn’t SELECT a customer (only SEARCH for them).

What I’d like to know is – how people do so much work without using the app? Or is it that they are chilling so it’s always me that finds the issue first? Even though I’ve been massively chilling the entire project. I’ll call this the “Chill Paradox”.

Liam (Team Manager) said that they tried to set-up a new environment and although the app loads, it doesn’t transcribe. The animation plays like it is recording but no words show at all. I said I had seen the issue but then it started working later on. He had thought it was a problem with their Staging environment but I said I was using the Development environment. Liam informs VerbatimEcho of the problem but never mentioned I had seen it.

We had some kind of Sign-Off meeting for the app to go to the initial Pilot sites. What happened to the 18th May deadline!?

Just before the meeting, Alan (Head Architect) posts a recording. Everything he said was transcribed wrong. Words and sentences were missed out, numbers he said were transcribed as a different number. He didn’t complain about any of that though. He was angry that he has a weight in Stone and our app is supposed to convert to Kilograms but it showed as Stone so failed to file into SystemNow.

In the meeting, I find it weird how we are all pretending it is good enough to go live. It’s been a disaster. 

“How do we know VerbatimEcho won’t update something and make the solution worse”

Liam (Team Manager)

Me (to myself): “It can’t get worse than this

I get annoyed how I find problems with VerbatimEcho and I am the first one to rant about them but then others are like “yeah I have seen this before”. WHY IS EVERYONE PRETENDING IT IS OKAY? I also pointed out on the recording screen, the Transcript shows the accurate times. When the processed Consultation loads, the Transcript shows completely different times, presumably because it has trimmed the silences and then showing the new times. But the user does not have access to replay the audio. The only way they could do that is by recording it separately but now the times will not match.

Week 41 – 8th-12th June

Dave (Lead Developer) and Liam (Team Manager) log in at 7am or something daft for the site activation. But it’s actually turned on at 12:30. No users use it that day, or even the day after.

We have another Story Pointing session, mainly about reverting some of the UI changes to the text areas. Some of the stories for reverting are 3 points. Some tweaks to behaviour are 3 points. Similar tweaks to behaviour that could probably be fixed together, all at 3 points. No one really cares about my protests that these are 1 point stories, or a 2 at a push, and we don’t need to split them up so much.

Mark is trying to process files for his Monitoring tool. I still don’t know what it does or how it works but he claims that a 4 min file is stuck on processing status for 45 mins. VerbatimEcho say they have only just activated scaling on that environment. Mark reckons he is talking nonsense because it’s a single request, nothing to do with scale.

Andrei fixes one of the deletion stories but then I realise he has removed the behaviour for deleting the Product Code Badge. I point it out and he sends me links to the User Story. He messages me telling me to resolve the comment “as irrelevant“. I told him I understand the main issue and confirmed he fixed it, but he had also wrongly removed functionality for the Product Code Badge. He said “it’s in the user story” and pastes the entire Acceptance Criteria to me, but states “further improvements can be made in other user stories“. That doesn’t make sense to me because these “improvements” should be done now, or shown to actually be requirements in other User Stories; otherwise it won’t get done.

After reading the Acceptance Criteria again, I then noticed a requirement I had overlooked. It specifically calls out that “deletion of Product Code Badge should remain unchanged”. Andrei backs down and says I’m correct. Funny that he said “it’s in the user story” to do it, but it actually specifically says not to change the behaviour.

Dave (Lead Developer) runs out of GitHub Copilot credits. Andrei says “we should either reduce usage or request a higher limit.” Cheers Captain Obvious what would we ever do without you?

Andrei fixes quite a few small items. A mix of 2 and 3 Story Points but clearly just took a few hours like I thought they would. So my protests during the Story Pointing session were proved correct.

Week 42 – 15th-19th June

The upper management are annoyed at the metrics because they seem inaccurate and there’s not much data. Liam (Team Manager) had an example in Dynatrace where he looked at the last 7 days and there were 4 entries. He manually selects the 10th-15th which is just the previous 5 days; there were 80 entries! I had problems like this when we used Datadog. When it shows data that makes no sense, then I don’t know why we spend an insane amount of money using it.

Dave (Lead Developer) and James were trying to add extra info to get more logs, and of better quality. Seemed quite trial and error, but eventually they got some improvements in.

We seem to have basically run out of work but no one seems to be bothered. Marek and I were asking for work, and the rest of the team had nearly finished with theirs.

Marek was upgrading the DotNet version to 10. There were some code analysis errors that appeared, so he suppressed them. Andrei challenged why he didn’t fix them, and he seemed to make out it was a lot of work. It wasn’t a lot of work, but just needed to make small changes in maybe 20 places which would clutter the PR. So it did make sense to split it out across two PRs, but then he started asking to create a new User Story for it, but it was unnecessary. I challenged him on it, and the team agreed with me.

We eventually had a long refinement meeting but we were mainly saying how the old bugs people logged were contradictory and that’s why we had basically blocked them. Lucy says she has some new bugs to log. Why didn’t she just log them? They were quite simple things that were similar to previous issues we had raised like interacting with controls whilst the Product Coding Panel is open (it then closes and ignores what you clicked on).

They will be straightforward fixes but we probably spent longer talking about them. The main problem was that Jean (Product Owner) didn’t understand them and kept on trying to recreate them as we talked her through it over and over again. Then when the fix was debatable, we asked her to make a decision but she either didn’t understand it or was incapable of making the decision.

We had a couple of User Stories to change how the Product Coding Panel works on small resolutions. It usually opens when you click the Add Product icon, or just click into a row. On small resolutions, we will just open it when you click the button. Another User Story was to  overlay rather than squish the text fields. We will also move the Product Coding Panel to the right so it will cover the end of sentences rather than the start. I didn’t think that would make too much difference overall but that’s what the UX designer wants.

Lucy disables adding codes on small resolutions completely. I told her it was just to disable opening the Product Coding Panel when clicking the text field; but you can still open it by specifically clicking the icon. She then says: “the intent here was not to break small screen coding – it doesn’t seem to be readable”.

Well, you have broken adding codes, and the new design is still unreadable in my opinion because we still have limited space and are now masking the rest with an overlay. So I think we should have questioned the new design, rather than actually just doing it.

Lucy says Ivan will fix what she has intentionally broken in another User Story. Andrei says he agrees with Lucy’s approach. I don’t understand because what Ivan will have to do is just restore the logic she removed, but then add a condition to block specifically when clicking the text field. But that describes the exact requirements from Lucy’s item. So in my opinion there is no advantage to temporarily break the functionality, and it just moves the required work to another team member.

I finish the week with 5 items waiting to be reviewed. I don’t know why people are reluctant to do reviews and it frustrates me when I keep saying each day that they need reviewing and it still doesn’t get done. It’s more frustrating that people were saying they had nothing to do, and still didn’t review my changes.

There was some User Feedback shared from the pilot sites: “it’s a bit rubbish“. They also mention it often “fails to generate the summary so you lose all your work“. In order to keep using it, he “Doesn’t have the confidence so will have to record using another AI product at the same time as a backup”.

Remember my initial prediction at the start of the project?

“How easily can the user recover if they have recorded a 10 minute clip but haven’t filed the data due to an error? I’d imagine once that happens a single time, then the user will stop using the feature”.

Me

Week 43 – 22-26 June

There was a call arranged to discuss the possibility of releasing to 4 more Pilot sites. The meeting was cancelled and I think it was decided to not release.

On Tuesday, we had a meeting to discuss the remaining work and a new item about displaying a warning to the user that certain warnings won’t be displayed when saving when a consultation is created via the Scribe.

Jean (Product Owner) said that pilot users have been reluctant to use our software because they have seen crashes. It’s exactly what I feared. As soon as they see a failed consultation then their trust in it will be zero and they will no longer use it. If you have spoken for 10 minutes then it fails, it is maximum frustration.

As the discussion progressed, there was a suggestion that users did not like the Product Coding Panel which had caused us loads of problems to get working as it does.

Then there was talk about scrapping our data entry completely in favour of the existing Consultations module that has also been drastically improved recently. So that will mean the UI side could be completely replaced.

Then there was talk of another team attempting to recreate our own AI product so we can bin off VerbatimEcho.

So it currently sounds like the product we have worked on is going to be completely replaced.

James claimed he had been secretly investigating making his own Scribe. No one had asked him to do this, and it wasn’t clear what he had done because it would probably involve signing up to various tools/services to make it work.

Week 44 – 29 June-3rd July

We were shown the new Scribe, it was made by Alan (Head Architect) and Mark so I suppose that was no surprise.

They showed a complex consultation and it seemed 100% accurate with transcription but also generated the consultation notes in real time, and was suggesting actual Product Codes for each statement. It seemed very fast as well. So it seemed better in every aspect.

No idea if it was using any hacks or would scale well, but they have proved the idea is possible regardless. It seems like they got this working in 3 weeks too which is incredibly impressive when you compare to VerbatimEcho who are supposed to specialise in this.

Tuesday, Liam (Team Manager) confirmed that the VerbatimEcho contract will be cancelled. At the end of the day, it was suggested that the plan really was to get rid of our UI work too.

I’d love to be a “fly on the wall” at VerbatimEcho. This contract would have been absolutely massive for them, and to lose it could be potentially crippling to them.

Conclusion

Another project cancellation, and it was supposed to be the most exciting project for the company in recent years. 44 Weeks to decide it wasn’t good enough for users, and that’s after around 6 months of delays and internally declaring “”Scribe is far superior to anything else on the market”.

I think the following would be a good summary of the main failures. Some directly contributed to the cancellation, whereas others just contributed to the joke.

  1. Microsoft Copilot used to create the User Stories; sometimes invented irrelevant requirements.
  2. Splitting up work so it’s too small. Makes the refinement meetings too long, and creates blocking dependencies.
  3. Team size slightly too big which meant we needed to spend longer in planning meetings to get enough items for everyone.
  4. Planning sessions ended up including some refining and estimating which should have been separate meetings.
  5. No conviction when estimating: People instantly backing down on their vote. Choosing minority vote as the final score made voting often feeling pointless.
  6. Daily Stand Up delayed till 10AM so contractors are sat around for up to 3 hours.
  7. Liam, Team Manager not clamping down on James’ constant lateness.
  8. Liam or Dave not holding people (mainly Ivan) to account when they’d claim to fix items but were still obviously broken.
  9. Andrei often made up requirements.
  10. Lead developer, Dave, not doing work that is on the board.
  11. Project Manager wasn’t meeting with VerbatimEcho. Not holding them to account when their delivery was sub-par.
  12. Early product demos were a waste of time. Marketing don’t need a demo of a button or a tab control.
  13. Team not informed of other teams related to the project. The sudden declaration of Adam’s team, or the “Partner API” AI Scribe.
  14. Took 30 Weeks to see the application integrated, then we had a massive influx of discovered bugs.

Whether to log a new bug

Recently, we had the classic debate whether when discovering an (unrelated) problem, a bug fix:

  • should be rejected/marked as failed, 
  • or if you pass it, but log a new bug.

Sam said he had never logged a new bug if testing failed, even if this issue wasn’t called out in the bug report; in the description or recreation steps.

So to explain using this situation: I had fixed the Entity Framework code which saved a new row to the database table. The bug was for passing correct values into it, and my change was fine; the correct values were now saved in the appropriate columns in the new row. However, if you send multiple calls at once, Sam noticed it wasn’t incrementing the number by 2 as expected, only by 1 (the first call was then essentially being overwritten – a classic concurrency problem).

I think it’s reasonable to use the original Work Item (Bug Report) to flag any issues testers find, but once we confirm the new issue isn’t caused by my change it should go into its own Work Item. You might assess that the new bug could be fixed in a later release. It makes no sense to fail bug A because you found bug B. 

Sam is basically increasing scope, then moaning that we might fail the sprint, because we cannot complete the new item within the 2 week deadline.

Complex Processes lower morale and encourage bad practice

I always find it interesting when people work in a particular job then get promoted into management. It’s a completely different set of skills and if it’s a fair promotion, the idea of getting so good at your job, that you no longer do that job anymore; is another illogical aspect of it.

One thing that always amazes me is when people make decisions that they know are a bad idea from their experience doing the job.

When I worked as a software tester, my view is that we were essentially there to find any bugs that exist. Part of finding them is to document how to recreate the bug so that developers could fix it. Extending this process so it’s more complex, more stages, or involves more people – causes people to not want to find bugs.

There were times where I witnessed people do the bare minimum and they would ignore bugs that didn’t appear severe to them.

One of the worst people I’ve worked with was an average tester who wanted to become a Test Manager, and he ended up trying to make the process more complex and often announced changes in a condescending way.

When testers found a bug and wanted to investigate it, they would often try to recreate it, sometimes under different scenarios to work out the scope and impact of the bug, then will tell a developer their findings and only then get it logged.

Therefore there was a delay between finding the bug and actually logging it. So we got an email from the Test Manager like so:

All,It is important that as soon as you discover a defect, you raise a defect for this BEFORE you speak with the developer. Any defects raised can easily be closed if they have been raised in error or discovered by the developer to not be an issue. We run the risk of releasing with defects that could potentially cause serious issues for our customers.

I understand his point that – if managers are checking the system to see what bugs are outstanding and they don’t see them all, then potentially, the software could end up being released with bugs. However, the process started getting worse from then on

Please can you include myself and Becky on any emails that are discussing a defect with a developer. This is so that we are both kept updated with any defects that could cause issues. Also for every defect you raise, I’d like an email to myself and Becky with the follow information :
-- WorkItem ID
- Title
- Area
- Any other information you feel relevant.

So now when we discover a bug, we had to log it straight away without the investigation, email two Test Managers, then copy in any further emails to them. Then as more information is known, update the bug report, and making sure we also had an appropriate workaround if the bug did get released (or is already released).

All,When you are filling out the SLA tab for a defect you need to ensure that if you’ve specified that there is a workaround available that the Workaround box is filled in with the Workaround.

If you’ve raised any defect that is a Severity 3 this MUST be fixed before the branch is signed off. This is our exit criteria, we do not sign a release off with any Sev 1, 2s or 3s. if the developer disagrees with this, escalate it to myself and Becky and we’ll deal with it.

Often when we logged a bug, he was either emailing you or comes to your desk to ask why you haven’t triaged it with a developer yet. Sometimes he did that within 10 minutes of you logging it. So he wanted you to log it before triaging, but would then demand that you triage it even if you haven’t had chance to contact an appropriate developer.

You’d also have other test cases to run which he was always on your back to give him constant status reports. It was hard to win because if you have tests to run and have found bugs, then he will want you to triage them but sometimes helping the developer could take hours which means you aren’t testing, so he will be asking why you haven’t run your tests.

That level of micromanaging and demanding updates wasn’t great for morale and also encouraged Software Testers to stop logging the bugs they found because it just added to their own workload and stress.

It seemed better just to steadily get through the tests, but I suppose if you didn’t want to log bugs, then what was the point in actually running the tests? I did suspect some people just marked them as passed and hoped there wasn’t an obvious bug they missed.

How Not To Write Unit Tests

Introduction

Unit testing is a software testing technique where individual components of a program are tested in isolation. These components or “units” can be functions, methods, or classes.

When implemented correctly, Unit testing is a crucial practice in modern software development. It helps ensure the quality, reliability, and maintainability of your code. By incorporating unit testing into your development workflow, you can catch bugs early, improve code quality, and ultimately deliver better software to your users.

When I heard about Unit tests, they did seem awesome. But then the more I used them, I found that my opinion on them has declined. I find it quite hard to explain though.

I think in general, to make things testable, you have to split the logic up into smaller methods. But then when they are smaller, A) they are easier to understand and B) they are less likely to change. So if a developer has looked at that code, what is the chance they are gonna change it and break it? If you have a unit test and it never fails in the software’s lifetime, has it provided any benefit?

Then in the case that you decide to change the behaviour, then you have the overhead of rewriting all the unit tests and it can basically double the development time.

When there’s certain scenarios which could end up taking ages to manually test it, the unit tests are very beneficial. When there’s loads of permutation/optional aspects to logic, it is a prime candidate for unit tests. Without unit tests, retesting every time you make a simple change is incredibly tedious. But with unit tests, you just click a button and wait a few seconds.

Unit tests give you confidence you can refactor without risk. However, they are not automatically the silver bullet. Well-written, fast, reliable tests accelerate development and save time. Poorly-written, slow, flakey tests hinder development and waste time. 

Fast Tests

A test that takes a second to run doesn’t sound slow, but what if you have hundreds or thousands of tests? If the tests take a long time to run, the developers won’t run them as often, or at all, then what value do they serve? 

They also should run on a build to ensure only quality releases actually go live, but you want your release process to be fast.

There was a recent change where the developer was claiming to have sped up a long-running call, however, he hadn’t carried over that performance enhancement mindset to the tests, and had actually increased the time to run them by 6 seconds.

The code “Thread.Sleep” can be used in threaded code to intentionally call a delay. I’ve seen many developers add this to a unit test. Tests are supposed to be fast, so you should never add this in a unit test.

Measuring Tests & ExcludeFromCodeCoverage

When people write unit tests, they want to try to understand how much of their code is covered by tests. We have this metric of Code Coverage but it has some severe limitations in the way that it is measured. It’s often a simple metric of “does the line get hit by at least one test”, but since methods can be executed with different combinations of variables, you can end up having 100% statement coverage but without actually testing many combinations at all.

The metric is one that impresses managers so you often see developers writing bad tests simply to game the test coverage metric. This is bad as you end up being misled that your code changes haven’t caused any bugs but yet it could have introduced something severe because the unit tests weren’t adequate.

I’ve seen quite a few code changes purely to increase the code coverage. So the title of the change would be like:

“Added more code coverage”

Then when I check the build output:

“There might be failed tests”

How can you be adding more tests then not actually run them before submitting it to review? Madness. The explanation is that their focus is just on coverage and not on quality. Maybe a bit of arrogance and laziness.

This week I worked with a team to get code coverage over 80% (a corporate minimum). The problem with this effort: Code coverage can be gamed. Sure, low code coverage means there’s a lot of untested code. But, high code coverage doesn’t mean the code is well tested.

Cory House

You can add ExcludeFromCodeCoverage “attributes” to your code which tells the code coverage tool to ignore it. It’s a simple way of reducing the amount of lines that are flagged as untested.

Here’s one of our principal developer’s opinion on this attribute:

“Using ExcludeFromCodeCoverage is only good when the goal is 100% coverage. That should never be the goal. The goal should be a test suite that prevents bugs from ever going live. I’m happy never using it and just having coverage reports flag things that are not actually covered, it is a more realistic representation of what the tests cover and makes me much more cautious about changing them as I know I don’t have test coverage. Never add Exclude from Code Coverage, it’s just lying to everyone. Why suppress things that might be a problem, if they are, we need to fix them.”

Principal Developer

Personally, I think adding suppressions/attributes just clutters the code base. I’d rather just treat the stats as relative to each release. The numbers have gone up/down, but why? If we can justify them, then it’s all good. Chasing 0 code smells and a specific test coverage means you can just cheat and add the likes of ExcludeFromCodeCoverage to meet such metrics.

Another developer said:

I value a holistic set of metrics that help us understand quality in software development. Code coverage is a single metric that can be part of that set of metrics you monitor. No single metric can stand by itself, and be meaningful. Nothing is perfect, which is why we should value a toolbox. I don’t believe in gaming the system and “hiding” uncovered code to get to 100%.

You need engineering teams who are prepared and confident enough to publicly share their coverage reports. This sets the tone of the culture. Context is needed, always. There will be reasons why the coverage is as it is. Use tools that help engineering teams with confidence/delivering at pace and ultimately delivering customer satisfaction. You cannot compare reports from different teams or projects.

Useful Tests

You need to make sure your tests actually test some logic. Sometimes people end up seemingly writing tests that really test the actual programming language, but I suspect it is just so the Code Coverage metric is fooled. Code Coverage checks if lines of code are “covered” by tests, but the simplistic nature of the check just ensures that a line of code is executed whilst a test is running; rather than if there was a meaningful test.

So for example:

[Fact]
public void DefaultConstructor_ReturnsInstance()
{
        var redisMgr = new RedisStateManager();
        Assert.NotNull(redisMgr);
}

So there you are instantiating an object then checking it is not null. Now that’s how objects work in C#. You instantiate an object, and then you have an object. Now, I suppose an exception could be thrown and the object wasn’t created, but that is generally considered bad practice and also there was no other test to check a situation like that, so they haven’t tested all scenarios.

largeResourceDefinitionHandler.MissingResource = _missingResourceDefinitions;
 
Assert.NotEmpty(largeResourceDefinitionHandler.MissingResource);

Setting it then checking it is set. Unless the property has loads of logic which you could say is bad design, then checking it is set is really testing the “.net framework” but if you think you need this; that means you don’t trust the fundamental features of the programming language you are using. You are supposed to be testing the logic of your code, and not the programming language.

If there’s lots of setup then the Assert is just checking for Null, then it’s likely just to fudge the code coverage. Another classic that I’ve seen is loads of setup, then ends with:

Assert.IsTrue(true);

So as long as the test didn’t throw an exception along the way, then it would just always pass because True is definitely equal to True.

Those ones seem intentionally malicious to me, but maybe the following example is more of a case of a clear typo:

Assert.Same(returnTrigger, returnTrigger);

Whereas this following one looks like a typo, but it’s actually two different variables. Need to look closely (one is a single S in Transmission). 🧐

Assert.Equal(organisationTransmissionStatus.Enabled, organisationTransmisionStatus.Enabled);

What goes through people’s heads? How can you write code like that and carry on like nothing is weird.

Sometimes tests look a bit more complicated but on analysis they still don’t really test much:

        [CollectionDefinition(nameof(LoggerProviderTests), DisableParallelization = true)]
         public class LoggerProviderTests : IDisposable
         {
[Theory]
                 [InlineData("Verbose")]
                 [InlineData("Debug")]
                 [InlineData("Fatal")]
                 [InlineData("Information")]
                 [InlineData("InvalidLogLevel")] // Test with an invalid log level
                 public void GetMinimumLevel_ReturnsCorrectLogLevel(string logLevelSetting)
                 {
                         // Arrange
                         System.Configuration.ConfigurationManager.AppSettings["DistributedState.LogLevel"] = logLevelSetting;
var firstInstance = LoggerProvider.Instance;
                         var secondInstance = LoggerProvider.Instance;
// Assert
                         Assert.Same(firstInstance, secondInstance);
                 }
         }

So this sets a setting on AppSettings, presumably used by the “LoggerProvider”. However, all they are doing is testing that if you call the Instance property twice, it returns the same object  both times. So the setting of the different log levels is completely irrelevant. I mean, the log level could be completely wrong but you are comparing ‘is the wrong value of A the same as the wrong value of B’; and it will still pass.

Another common aspect is when you use a testing library like Moq, and you can use it to create objects and essentially say “when I call some code with these specific parameters, then give me this value back”. The thing is when developers use this as the actual thing they are testing, then you are testing Moq, and not your actual logic.

	[Fact]
	public void JobReposReturnsValidJobTest()
	{
		//Arrange
		ScheduledJob job = new ScheduledJob() { ScheduledJobId = Guid.Parse("e5ee8f7410dc405fb2169ae2ff086310"), OrganisationGUID = Guid.Parse("fbee8f7410dc405fb2169ae2ff086310") };
		_mockJobsRepo.Object.Add(job);
		_mockJobsRepo.Setup(e => e.GetById(Guid.Parse("e5ee8f7410dc405fb2169ae2ff086310"))).Returns(job);

	//Act
	var resultJob = _unitOfWork.JobsRepo.GetById(Guid.Parse("e5ee8f7410dc405fb2169ae2ff086310"));

	//Assert
	Assert.Equal( Guid.Parse("fbee8f7410dc405fb2169ae2ff086310"), resultJob.OrganisationGUID);
	}

“I think all this test is doing – is testing that JobsRepo returns an object that was passed into the constructor on line 22. The GetById is redundant, it will always work if it returns that object because the Moq was configured to return that value. That is testing Moq, and not our code. But then if you are just asserting a property returns an object, you are just testing that C# properties work.”

Me

“yes you are right , I am just testing if JobsRepo could return a value, so that it helps me in code coverage for get functionality of JobsRepo , as it is just set in the constructor of the class and there is no call for get”

Developer who wrote bad tests

So I think they are saying “I am just fudging the coverage”. Checks it in anyway.

There’s been loads of tests where you could actually cut out large parts of the method they are testing and the tests still pass. Again, sometimes you point this out to developers and they still want to check it in, purely for the statistics, and not for any benefit to any developer.

“do these tests add value? a quick glance suggests this is very dependent on your mock object. It might be the case that the production code can be changed without breaking these tests.”

Me

yeah, they’re kind of meaningless. Merging code to use as templates for future, better, tests.

Developer who wrote bad tests

Here is a rant I left on a similar review:

This name implies that it should always be disabled, especially because there’s no coverage for the case where it is true. However, these tests aren’t really testing anything. I think you’re basically testing that Moq works and the default boolean is false. I think the most you can really do is call Verify on Moq to ensure the correct parameters are passed into the GetBool call.

If you replace the contents of IsRequestingFeatureEnabledForOrganisation with return false, your tests pass which illustrate the coverage isn’t complete, or you aren’t guaranteeing the configuration manager code is even called at all. Personally, I don’t think it is worth testing at all though. All your class does is call the FeatureDetails class so you aren’t testing any logic on your side.

I think people are too concerned about getting code coverage up, so they insist on writing tests, even if it makes things more confusing.

I suppose it is up to you and your team to decide what you want to do, but I occasionally question people just to make them think if it is actually adding any value. I’ve seen tests where they simply assert if an object is not null, but it could literally return an object with all the wrong values and still pass (and the method always returned an object anyway so could never fail). If you see a method has tests, it gives you a false sense of security that you think it is going to catch any mistake you make, but it just always passes anyway

always think if your tests will add value and if it’s worth adding them. If you need to mock everything then they’re not very valuable, or you’re testing at the wrong level (too high), and you’re better off with integration tests than unit test. 100% code coverage is a really bad idea for complex software, massive diminishing returns in value the higher you try to push it. We change stuff all the time in our software too, so if everything has high-level unit tests then you spend more time fixing those tests.I tend to find you spend ages writing tests then if you change the implementation then you have to change the tests and you can’t run them to see if you broke anything because you had to change the test to run it.

Me

Test Driven Design (TDD)

There’s a methodology called Test Driven Development where you write a test first. It will then fail if you run it because there’s no functionality to run. Then you write the implementation and get it to pass. Then move onto writing the next test, and repeat. So you build up your suite of tests and get feedback if your new changes have broken previous logic you wrote.

I was recently listening to a podcast and the guest said that he always writes code first, then adds tests after. If he can’t write tests, then he will make a change to bypass code just for the tests. I wasn’t sure what he meant by this, maybe it’s like when people write a new constructor which is only ever called by the tests. But that’s bad design.

I thought he may as well just do TDD from the start, instead of always going through that struggle. He says TDD doesn’t often lead to good design because you aren’t thinking about design, you just think of how to make the tests pass. 

But doesn’t the design organically come from TDD? and his way of changing the design just for the tests is what he is arguing against TDD for. TDD often slightly over-engineers the solution with the likes of Interfaces. So then he is avoiding TDD, and instead writing the tests after; but his way adds “Technical Debt” via adding extra constructors that are only used by the tests.

“I’ll add tests in a separate change later”.

 5 reasons to add tests before merge:

1. Clear memory: Before merge, everything is fresh in my mind. I know what the code is supposed to do, because I wrote it. So I also know what tests I should write to assure it works. Every minute that passes after merge, I will understand the feature less, and thus, be less equipped to add proper test coverage.

2. More effective reviews: If I write the tests before merge, then anyone reviewing my code can use my tests to help them understand the code, and to watch the feature run.

3. Faster development: If I write tests during development, I can use the tests to accelerate my development. I can “lean” on my tests as I refactor. Faster feedback loops = faster development.

4. Better design: Writing tests during dev encourages me to write code that is testable. It makes me consider accessibility too since that tends to make automated testing easier by providing well-labeled targets.

5. Changing priorities: After merge, there’s no guarantee that I’ll have time to write the tests at all. I may get pulled away for other more “urgent” tasks.

Bottom line: The proper time to add tests is *before* merge.

Coty House

I recently saw the following conversation. A developer was basically saying he didn’t have time to write the tests, and it might end up in some drastic refactoring which would be risky. Then the plan is to rely on manual testers and get the changes released. Then the next part probably won’t happen (because important features will be prioritised), but his suggestion is that he then makes the changes for the next release with good unit test coverage.

Senior Developer:
This domain supports unit testing, you should be able to add tests to cover the changes you made to make sure it behaves as you expect

Developer
Currently there are no unit test cases available for the changes made class, and the class is tightly coupled. I have written some draft tests and will check them next month as a priority.

Architect
IMO, given the pressure, timescales and urge to complete this, I think we can defer for now and stress the testers to pay more attention to the areas that we have low confidence.

Senior Developer:
So instead of checking if it is correct by adding tests that we can be sure exercise the code changes, we just merge it and hope that the manual testers find any bugs over the next day or so, and if they do, then it is back to the dev team and another change?

Time In Unit Tests

Tests should be deterministic. If a test is run and passes, then if no changes have been made and we run it again, it should also pass (obviously). An unreliable test doesn’t give you confidence in code changes you make. It’s a surprisingly common occurrence when you make a change and an unrelated test breaks, and you are thinking “how can those changes break the test“? then you look at what it is doing, and it’s often something to do with time.

You see something like
data is "BirthDate":"1957-01-15T00:00:00"
And the test result says:
Expected "Age":"67y"
Actual: "Age":"68y"
Today is their birthday!

What you need to do is put a “wrapper” around the code that gets the current date. So instead of simply DateTime.Now, you create a class called something like DateTimeProvider, and in the production code, the class returns DateTime.Now. Then in your Unit Tests, you then create a MockDateTimeProvider and make it return a hard-coded date. That way, no matter when you run the test, it always returns the same date, and is a deterministic test.

I recently fixed some tests that were failing between 9pm-12am. I found that a developer had changed the MockDateTimeProvider to return DateTime.Now, completely rendering it pointless. Other parts of the test were adding 3 hours to the current time, and because 9pm+3 hours is tomorrow’s date, the date comparison it was doing then failed.

public class MockDateTimeProvider : IDateTimeProvider
{
        public DateTime Now { get { return DateTime.Now; } }
}

I think another red flag in unit tests is conditional statements. Logic should be in your production code, and not in tests. Not only does this following code have a DateTime.Now in it, it looks like they have put a conditional If statement in there, so if it would normally fail, it will now execute the other branch instead and pass. So maybe the test can never fail.


[Fact]
public void ExpiryDateTest()
{
        DateTime? expiryDate = (DateTime?)Convert.ToDateTime("12-Dec-2012");
        _manageSpecialNoteViewModel = new ManageSpecialNoteViewModel(_mockApplicationContext.Object);
        _manageSpecialNoteViewModel.ExpiryDate = Convert.ToDateTime(expiryDate);
        
        if (_manageSpecialNoteViewModel.ExpiryDate < DateTime.Now.Date)
                Assert.True(_manageSpecialNoteViewModel.IsValid());
        else
                Assert.False(_manageSpecialNoteViewModel.IsValid());
}

Other Bad Unit Tests

Maybe the most obvious red flag, even to non-programmers – is testing that the feature is broken. The developer has left a code comment to say it looks wrong!

Assert.Equal("0", fileRecordResponse.Outcome); // I would have thought this should have been -1

The One Line Test

How do you even read this. Is that actually one line? 🤔🧐

_scheduledJobsRepo.Setup(r => r.GetAllAsNoTracking(It.IsAny<Expression<Func<ScheduledJob, bool>>>(),
        It.IsAny<Func<IQueryable<ScheduledJob>, IOrderedQueryable<ScheduledJob>>>(),
        It.IsAny<int>(),
        It.IsAny<int>(),
        It.IsAny<Expression<Func<ScheduledJob, object>>>()))
        .Returns((Expression<Func<ScheduledJob, bool>> expression,
        Func<IQueryable<ScheduledJob>, IOrderedQueryable<ScheduledJob>> orderBy,
        int page, int pageSize,
        Expression<Func<ScheduledJob, object>>[] includeProperties)
        =>
        {
                var result = _scheduledJobs.AsQueryable();
                if (expression != null)
                {
                        result = result.Where(expression);
                }
                result = orderBy(result);
                result = result.Skip(page * pageSize).Take(pageSize);
                return result;
        });

When it is that hard to read, I wonder how long it took to write it.

Other Common Mistakes

I think tests can be unclear if you use a Unit Testing library but not understand what features are available. Like instead of using the ExpectedException check, they may come up with some convoluted solution like a try/catch block then flagging the test as passed/failed.

try 
{ 
        Helper.GetInfoArticles(articleName, _httpWebRequest.Object); 
        Assert.IsFalse(true); 
} 
catch 
{ 
        Assert.IsTrue(true); 
} 

Naming the tests can be tricky to make it clear what it does and to differentiate it from other tests. The worst is when the name says something completely different, most likely from a “copy and paste” mistake.

I’ve talked about how using DateTime can make tests fail at certain times. You can end up with tests that rely on some shared state, then the order you run the tests causes failure when the test expects data is set or not set.

Bank

Tests are even more important in certain domains. You know, like when money is involved. Commercial Bank Of Ethiopia allowed customers to withdraw more cash than they had.

“I’m really interested how a bank managed to deploy code that didn’t have tests for “can a user withdraw money when they don’t have enough balance.” Development teams at banks are usually conservative, process-heavy and slow-moving with changes exactly to avoid this. Wow”

Conclusion

Unit tests can be a useful and helpful tool to developers. However, there is an art to writing them and they have to be written with good intentions. If they aren’t written to be useful, fast, and reliable, then developers either won’t run them or won’t trust them.

Minimum requirements when logging defects

A colleague wrote a list of ideas to improve bug reports. The more detail a report has, the more stakeholders are aware of the impact and can triage it accordingly. When it comes to fixing it, software developers can work out the issue more easily…

It is important that all the required information is added to a defect at the time of raising it. This stops the need to refer it back to the original author for more information and therefore saves time.

It should also be noted that we report all defects raised to certain user groups, and it is important that they are able to understand what the issue is and how severe the issue is to them.

In order to meet the minimum requirement all bugs should contain the following:

Title

  • The title must be clear and concise
  • It should describe the problem
  • Avoid using technical jargon

Description

  • The description must clearly outline the problem being experienced
  • It should explain the impact to the customer
  • It should describe a workaround if known.

 Reproduction steps

  • Reproduction steps to recreate the issue
  • These should be clear and easy to follow
  • Include any required environment configuration
  • Include the actual and the expected outcome

 System info

  • Enter the version of software the bug was found in

 Other checks

  • Have the Area and Iteration paths been set correctly?
  • Are relevant examples present with the record? (record number, sample data, screenshots etc.)
  • Have the relevant error details been included? (stack trace, error message, error number etc.)
  • Has the bug been linked to the test case/user story etc?
  • Has this been linked to a relevant user story, requirement, or similar bug report?

 Root Cause Analysis (RCA)

If the defect is found during integration testing, the following must also be completed

  • Stage Detected In
  • Use Release Testing
  • Complete Release Introduced In field

Recorded Test Steps

In my blog How To Make Your Team Hate You #3, I wrote about Barbara, a Tester who I used to work with that caused a lot of conflict and was constantly trying to get out of doing work, whilst taking credit for other people’s work.

Recently, when going through old chat logs, I found some brilliant “dirt”, which, in hindsight; I could have probably used to get her sacked because it was fairly strong evidence that – not only was she not doing work; she was falsely passing Test Cases. When you are paid to check if the software is behaving correctly, claiming you have tested it is very negligent.

When running test cases, if you pass each step separately, and haven’t disabled the recording feature, Microsoft Test Manager would record your clicks and add it as evidence to the test run.

I think the feature worked really well for web apps because it can easily grab the name of all the components you clicked, whereas on our desktop app, it mainly just logged when the app had focus and read your keystrokes.

The bad news for Barbara, is that she liked going on the internet for personal use, and liked chatting using instant messenger as we will see.

The Remedy

Type 'Hi Gavin. ' in 'Chat Input. Conversation with Gavin Ford' text box
Type 'Hi Gavin. I've been telling everyone about this concoction and it really worked wonders for everyone that's tried it, myself included. This is for cold, cough and general immunity. 1 cup of milk + 1 tablespoon honey + 1/4 teaspoon of turmeric - bring to a rolling boil. Add grated root ginger (2 teaspoons or 1 tablespoon) and let it boil for another 5 mins. Put thru sieve and discard root ginger bits (or drink it all up if you fancy), but drink it hot before you sleep every night and twice a day if symptoms are really bad. Hope you feel better soon. 🙂 ' in 'Chat Input.

Pumpkins & Tetris

Type 'Indian pumpkin growing{Enter}' in 'Address and search bar' text box
Type '{Left}{Left} {Right} {Left}{Left} {Up}{Up}{Up}{Up}{Up}{Up}{Left}{Left} {Up}{Up}{Up}{Right} {Up}{Up}{Left} {Right}{Right} {Up}{Right}{Left}{Left}{Left}{Left} {Right}{Up}{Left}{Left}' in '(1) Tetris Battle on Facebook - Google Chrome' document

Click 'Amazon.co.uk:Customer Reviews: 100ml Bergamot Pure...' label
Click 'Close' button
Click 'Essential-Oil-Blends_Chart.pdf' label
Click 'Checkout' label

3 days Small Regression Pack


Me 11:26:
Barbara has been doing the Assessment regression pack for 3 days
she says there is only a few left in this morning's standup. There's 15 left out of 27
Dan Woolley 11:28:
lol
Me 11:29:
I don't even think she is testing them either. It looks like she is dicking about then clicking pass
Click 'Inbox (2,249) - [Barbara@gmail.com]Barbara@gmail.com - Gmail' label
Click 'Taurus Horoscope for April 2017 - Page 4 of 4 - Su...' tab
Click 'Chrome Legacy Window' document
Click 'Chrome Legacy Window' document
Click 'Close' button
Click 'Paul' label in the window 'Paul'
Click image
Type 'Morning. ' in 'Chat Input. Conversation with Paul' text box
Type '{Enter}' in 'Chat Input. Conversation with Paul' text box
Step Completed : Repeat steps 6 to 19 using the Context Menu in the List Panel
End testing

Next Day

Me 12:42: Barbara said this morning that all the Assessments test cases need running. She has just removed them instead

Greek Salad

Type 'greek salad{Enter}' in 'Chrome Legacy Window' document
Type 'cous cous salad' in 'Chrome Legacy Window' document
Type 'carrots ' in 'couscous with lemon and coriander - Google Search ...' document

Click 'Vegetable Couscous Recipe | Taste of Home' tab
Click 'Woman Traumatized By Chimpanzee Attack Speaks Out ...' tab

Marshall 11:50:
oh damn haha
these are things that were inadvertently recorded?
Me 11:51:
yeah
Marshall 11:51:
ha you've stumbled upon a gold mine
Me 11:53:
I don't think she is actually testing anything. I think she just completes a step now and then
the other day Rob went to PO approve an item and he couldn't see the changes because they hadn't even patched

Haven’t Been Testing From The Start

we are in Sprint 8 and Barbara suggested Matt does a demo on the project so we know how it works; it’s a right riot

Me. 4 months into a project

Bad Audits

I wonder if Barbara was inconsistent with how she ran the test cases, or realised by the end that it tracked you. So near the end of her time, she was just hitting the main Pass button rather than passing each individual step. Managers liked the step-by-step way because if you mark a step as failed, it is clearer what the problem is.

Me 16:15: 
Barbara called me. Matt is monitoring our testing!
Dan Woolley 16:15:
how?
Me 16:17:
looking at the run history
she said he was complaining it wasn't clear which step failed because we were just using the main pass button, and also bugs weren't linked when they had been failed
I told Barbara I linked mine, then she checked and said it was Sam that didn't. I checked and saw it was Sam and Barbara
so only the developer did testing properly 😀
you just can't get the staff

Obviously The Wrong Message

Me 09:12: 
Bug 35824:Legal Basis text needs to be clear
what's all that about?
Barbara Smith 09:12:
Charlotte asked me to raise it for visibility
We need to fix the text that appears on that tab
Me 09:13:
what's wrong with it?
Barbara Smith 09:21:
It says that on the Bug LOL
And with a screenshot (mm)
Me 09:22:
it says "needs to be clear" and has a screenshot with a part of it underlined. But it doesn't say what the text should be instead.

She rarely logged bugs because she did minimal testing. Then when she did log something it didn’t have enough info to be useful.

Karma

Barbara got well conned in the end. She was gonna take the entire December off but delayed it for the end of the project and then she has been told she has lost her job, so they are telling her to take the holiday now. She had just bought a house so would be relying on the money for the mortgage payments. Luckily for her she got accepted for a new job, but she was looking for a brand new way of getting out of it, as we will see below.

Tax Fraud

Type 'what if I don't contact hrmc about my tax{Enter}' in 'Address and search bar' text box
Sam 11:23:
Ha ha
You are savage
Me 11:24:
she is gonna get jailed for tax evasion

Maternity Fraud

Click ‘How to get pregnant fast | BabyCenter’ tab

Testing Stories

Go Play, Find Bugs

One of our Senior Testers wrote a blog detailing how she found an obscure bug. When I was a software tester, I often said that – even though you spend a large amount of your time writing Test Cases and running them; the majority of bugs I found were actually performing actions off-script.

The reason for this is that if you have a certain requirement, the developer writes enough code to pass that requirement as it is written. A better developer may even write some kind of automated tests to cover that scenario to prove that it works, and it won’t break in future. Therefore, running a manual test that describes that behaviour won’t find a bug now, and it won’t if you run that test in the future (during regression testing).

Being able to freestyle your steps means you can come up with obscure scenarios and experiment, and do way more testing than you would following a strict, heavily-documented process.

This was the main problem I had working as a Software Tester. Managers wanted the documentation and if you told them you had been testing without it, you sometimes got told to stop, or spend time writing Test Cases for ALL the additional scenarios you came up with. All that does is encourage people to be lazy and do the minimal amount of testing, which consists of just the basic scenarios.

You also get into scenarios where if there is a bug in live, it’s easy to make stupid claims in hindsight. I remember a colleague being absolutely furious with the criticism. They had done loads of testing but there was a bug in live in a very specific scenario:

“I’m disappointed in the level of testing” – Test Manager

Here is our Senior Tester’s blog:

I found a deliciously elusive bug last week. The feeling a tester gets when this happens is joy at your good luck, like satisfaction at solving a fiendish puzzle, and relief at preventing harm. We feel useful!

The bug was to do with online visibility data. My team is implementing the ability to right-click items and set Online Visibility. Sounds simple in theory - but data is complicated and the code base is large.

How was I going to approach this? It was an intimidating piece of work – and I was tired. My normal process would be to come up with some ideas for testing, document them, then interact with the product, make notes, fill out the report. But that day, I just couldn’t face doing the documentation and planning I would normally do before the testing. I decided to just test, not worry too much about documentation, and have fun.

I sought out a Record with a rich data set and played around, selecting things, deselecting them, selecting parent entries, child entries, single entry, multiple entries. I didn’t have any defined agenda in mind except to explore and see what would happen.

One minute in, I was rewarded with a beautiful crash!

I hadn’t taken a note of my steps – but I knew I could probably find the path again. I set up and recorded a Teams meeting with myself, as I didn’t want to have to pause to note down every step I took – that would take a long time and risk my mindset changing to a formal, rigid, structured view – which I didn’t want. I needed that playful freedom. The system crashed again! As there were so many variables at play, I didn’t know what the exact cause was, but I now had proof that it hadn’t been a magical dream.

I spent the rest of the afternoon trying to determine the exact circumstances in vain. I spoke to the programmer, and showed him my recording. He took the issue seriously, and tried to recreate it himself. We both struggled to do so, and decided to wait until the morning.

The following day, we got on a call and went over the recording again. What exactly had I done before the crash? I had selected the parent entry, then two child entries, right clicked but not changed anything, deselected the parent, selected another child, unselected it, selected a different child, selected the parent again and then right clicked and changed the Online Visibility - crash. We tried that again on the developer’s machine, on the same type of report, break points at the ready. Crash! Got it!

The developer eventually narrowed it down to two conditions: child entries had to have a GroupingDisplayOrder index beginning with 1, and the user had to select the parent entry after its child.

It seemed sheer luck that I had found this. But was it luck? No. It was luck by design – I had created a rich data set, and done lots of different actions in different orders, been creative and diverse in my testing. And it had only taken a minute to yield results!

So what did I learn? Reflecting, I noted my preference for highly structured documentation – of tables with colour highlighting, documenting each test in high detail, strictly in order, changing one condition at a time. The result of this was that I tested in a highly formal, structured way to fit the documentation, and only did informal testing as an afterthought. And yet I had most often found bugs during the informal testing!

I had made a god of documentation and lost sight of what mattered most. If you need me, I’ll be testing. And trying not to make too many pivot tables.

What Are Software Testers Really?

The same tester once came out with this quote

“testers are ultimately critics. Developers are artists. Testers are there to give information. What you do with that information is up to you.”

That’s quite an interesting perspective. I think it mainly comes from the idea that Testers can find 10 bugs but maybe you decide that you will only fix 6 of them, a few you might fix later, then 2 you think aren’t a problem, or so unlikely to happen – it’s not worth the effort and risk to fix it.

“we are the headlights of the car, driving into the darkness”

Software Testers In Game Development

“She was the one who taught me the importance of testers and how they are a critical gear in the machinery that makes-up making a game. Testers aren’t just unit tests in human form. They have a unique perspective on the game and poke not only at the bugs but also the design and the thought process of playing a game.”

Ron Gilbert, creator of Monkey Island

Another interesting discussion on the role software testers play is from Mark Darrah who has worked on games like Dragon Age Origins. He does seem to agree with this idea that the Testers are merely critics.

Mark Darrah – Don’t Blame QA

When encountering bugs during gameplay, it’s often misconceived that the quality assurance (QA) team is to blame. However, it’s more likely that the QA team identified and reported the bug, but it remained unresolved due to various factors. For instance, a more critical bug could have emerged from the attempted fix, leading to a strategic decision to tolerate the lesser bug. Additionally, project leaders may assess the bug during triage and conclude that its impact is minimal (affecting a small number of users), opting to proceed with the game’s release.

Such scenarios are more common than one might expect, and they typically occur more frequently than QA overlooking a bug altogether. If a bug did slip through QA, it’s usually not the fault of any single individual. The bug might result from a vast number of possible combinations (a combinatorial explosion) of in-game elements, making it impractical to test every scenario. Your unique combination of in-game items and actions may have simply gone untested, not due to negligence, but due to limited resources.

Complex game designs can introduce bugs that are difficult to detect, such as those that only appear in multiplayer modes. Budget constraints may force QA to simulate multiplayer scenarios solo (a single person playing all four or eight different players at once), significantly reducing the scope of testing. 

Furthermore, bugs can be hardware-specific, and while less common now, they do occur. It’s improbable that QA had access to the exact hardware configuration of your high-end gaming setup.

The term ‘Quality Assurance’ (QA) can often be a misnomer within the development industry. While ‘assurance’ suggests a guarantee of quality, the role of QA is not to ensure the absence of issues but to verify the quality by identifying problems. It is the collective responsibility of the development team to address and resolve these issues.

Understanding the semantics is crucial because language shapes perception. The term ‘QA’ may inadvertently set unrealistic expectations of the role’s responsibilities. In many development studios, QA teams are undervalued, sometimes excluded from team meetings, bonuses, and even social events like Christmas parties. Yet, they are expected to shoulder the criticism for any flaws or bugs that remain in the final product, which is both unfair and inappropriate.

Developers, it’s essential to recognize that QA is an integral part of your team. The effectiveness of your QA team can significantly influence the quality of your game. Encourage them to report both qualitative and quantitative bugs, engage with them about the complexities of different systems, and heed their warnings about testing difficulties. Disregarding their insights can lead to overlooked bugs and ultimately, a compromised product.

For those outside the development sphere, it’s important to understand that if you encounter a bug in a game, it’s likely that QA was aware of it, but there may have been extenuating circumstances preventing its resolution. Treat QA with respect; they play a pivotal role in maintaining the game’s integrity.

Remember, a strong QA team is the bulwark against the chaos of a bug-ridden release. Appreciate their efforts, for they are a vital component in the creation of seamless gaming experiences. 

Analysing Risk – A Story

Just like my last blog, this is based on  an internal blog that our most experienced software tester wrote. She seems to love Michael Bolton, but not the singer. Michael Bolton is also the name of a software tester that is the co-Author of Rapid Software Testing (see About the Authors — Rapid Software Testing (rapid-software-testing.com)).

Michael Bolton

She said that Michael Bolton was asked the following question:

Q: My client wants to do risk analysis for the whole product, they have outlined all modules. I got asked to give input. Do we have a practical example for that? I want to know more about it.

Tester

Michael: Consider the basic risk story –

Some victim will suffer a problem because of a vulnerability in the product (or system) which is triggered by some threat.

Start with any of those keywords, and imagine how it connects with the others.

Who might suffer loss, harm, bad feelings, diminished value, trouble?

How might they suffer?

What kinds of problems might they experience?  What Bad Things could happen?  What Good Things might fail to happen?

Where are there vulnerabilities or weaknesses or bugs in the product, such that the problem might manifest?  What good things are missing?

What combinations of vulnerability plus specific conditions could allow the problem to actually happen? 

When might they happen?  Why?  On what platforms? How?

Our tester stated “This is a brilliant definition of risk. It is also a somewhat intimidating list of questions. If you are looking at this and thinking, “That’s hard!” you’re absolutely right. Good testing is hard. It’s deep, challenging, exhausting. It will make you weep, laugh, sigh from relief. But it’s also tremendous fun.”

Test Automation Mess

Every now and then, there is a big initiative to focus on Automated testing. A manager will decide that our software is complex and too manually intensive to regression test in detail. Automation seems the answer but it’s never that practical.

Our main software, a desktop application, requires interaction through the UI which is incredibly slow and unreliable. We used to have a dedicated Automation team that maintained the tests but they would take several hours to run, would randomly fail, then eventually the team disbanded and declared them obsolete. There’s been times we wanted to replace them with the likes of CodedUI (which turned out to have the same issues), and more recently FlaUI.

When the last “drive for automation” was announced by the CTO, our most experienced tester wrote an internal blog which I thought had a lot of subtext to it, basically saying “it’s a bad idea”.

Communities of Practice around Test Automation

With all of the new Communities of Practice around Test Automation*, I wanted to share some thoughts on whether automation is actually a good idea. This comes from experiences over the years. I hope this saves some people time, and provokes conversations.

To automate or not to automate? That is question….

A common question in a tester’s life:  “Should we automate our tests?”

Which of course really means, “Should we write our checks in code?”

This will inevitably give rise to more questions you need to answer:

  • which checks we should automate
  • and which we should not automate
  • and what information running the checks gives us
  • and how does that information help us assess risks present in the code
  • and which is the best tool to use
  • and how often we should run the checks

Asking and answering these questions is testing. We have to ask them because no automation comes for free. You have to write it, maintain it, set up your data, set up and maintain your test environment, and triage failures.

So how do you begin to decide which checks to automate?

Reasons for automating:

  • The checks are run frequently enough that if you spent a bit of time automating them then you would save time in the long run (high return on investment)
  • The checks would be relatively easy to write and maintain owing to the product having a scriptable interface (such as a REST API)
  • They can be performed more reliably by a machine (e.g. complex mathematical calculations)
  • They can be performed more precisely by a machine
  • They can be performed faster by a machine
  • You require use of code in order to detect that a problem exists
  • You want to learn how to code, or flex your programming muscles(Even if you ultimately decide not to automate your checks, you may decide to use code for other purposes, e.g. to generate test data.)

Reasons against automating:

  • There isn’t a scriptable interface; the product code can only be accessed via a User Interface (UI automation is notoriously expensive and unreliable).
  • In order to have a greater chance of finding problems that matter, the check should be carried out by a human being as they will observe things that would matter to a human but not a computer (e.g. flickering on the screen, text that is difficult to read).
  • The checks would have a short shelf life (low return on investment).

Beware of the fallacy that use of code or tools is a substitute for skilled and experienced human beings. If you gave an amateur cook use of a fancy food processor or set of knives, their cooking still wouldn’t be as good as that of a professional chef, even with the latter using blunt knives and an ancient cooker. Code and tools are ultimately extensions of your testing. If your testing is shallow, your automation will be shallow. If your testing is deep, your automation can be deep.

Ultimately the benefit you derive from writing coded checks has to outweigh the cost, and to automate or not is a decision no one else can make for you. 

Testers in my Team

Most of the testers we employ aren’t that technical, and most aren’t interested in writing Automated Tests since that requires knowledge as a  developer since it is coding. One of our testers went on a week-long training course about FlaUI. One of the first things he says is “FLAUI is not worth its value”, which made me laugh. The course cannot have painted it and a good light !” 😂

He then got asked to move teams to do pure automation for a few months. Another tester had no interest at all, but was instructed to “try learn”. 

“writing the steps is fine, it’s just when you go into the code”

Joanne

There was no way she was gonna be able to learn it. She isn’t technical and the desire isn’t there at all. Being pressured by managers to move away from “manual” testing to “automated” just disrespects them as a tester. It’s happened before and they end up leaving. She eventually moved internally to be a Release Manager.

Automation Mess

The original decision to move to FlaUI was made by a group of Testers and they didn’t get input from the Developers. 

I think it would be logical to code using the Coding Standards that us Developers have followed for years. If Developers want/need to help write Automated tests, they can fit right in since the process and code style is the same. Additionally, after years of writing Automated Tests, maybe the Testers want to switch roles and be a Developer and so it would be a smooth transition.

Not only did they invent their own Coding Standards, which meant variables/methods/classes were named differently, there was a lot of duplicated code to perform basic actions like logging in, selecting a customer record etc. 

The process including a branching strategy was different too, and so instead of having a Master branch, taking a Project Branch for longer-lived changes, and standard User Branches for simple short-lived branches, they went for a more convoluted strategy where they had Development, Devupdate, Master. Then it became a disorganised mess when work wasn’t merged to the correct branches at the right times.

I can’t even make sense of this:

Before the start of Regression: 

  • 1) Lock the Development Branch (no PRs to be allowed to come in to Development till regression is completed) 
  • 2) Development, Devupdate, Master are up-to-date by syncing your local with remote branch and get all the commits into local branch
  • 3) Merge from Development to DevUpdate 
  • 4) Merge from DevUpdate to MasterUpdate 
  • 5) Set <updateTestResults> to true and <testPlanId>(from URL ?planid=12345) inProjectSettings.xml in MasterUpdate 
  • 6) Raise a PR from MasterUpdate against Master. Throughout step 3, step 4, observe that ‘commits behind’ are equal after the merge process to that of master. 
  • Once the above process is completed, observe that Master branch is 1 commit ahead of other branches 

After the end of Regression: 

  • 1) Development, DevUpdate, Master are up-to-date by syncing your local with remote branch and get all the commits into local branch 
  • 2) Merge from Master to DevUpdate 
  • 3)Change the <testPlanId>toxxxxand<updateTestResults> to false in DevUpdate 
  • 4) Raise PR from DevUpdate against Development After Step 2, observe that ‘commits behind’ are equal after the merge process to that of master. 
  • Once the above process is completed, observe that Development branch is 1 commit ahead of other branches 

Eventually, a few more technical testers were moved into the team and tasked with aligning the process and codebase with our production code – ie sort the mess out.

This is the classic case of managers thinking they can just assign “resource” to a team, and give them an aim “automate this”; and expect results. But you need the technical know-how, and a clear direction.

Test Specifications

Many  years ago, when I was a Software Tester, I remember when we had to write a Test Specification based on the work that the Developers had planned. This was for both Enhancements and Bug Fixes (so new features and changes to old ones).

It would be a Word document, with the Item Number, Title, and then a description of what you would test (the description would be a bit more high level than the step-by-step description featured in an actual Test Case).

You would spend weeks writing it, then you had to get it approved by all the developers, or the Dev Lead. The developer often then told you it’s nothing like you imagined so you had to rewrite it. 

Sometimes they would demo the feature to you so you had a better idea. If they had no comments, I often felt that they didn’t read it.

When there was a new Developer who wasn’t familiar with the process, he “rejected” a Test Specification because of some grammatical issues. I think it was something to do with the wrong tense, and not using semicolons. The Tester was fuming, but he was quite a belligerent character.

I think we often worked from the Bug description, or some comments from the Developer. However, quite often, the comment section would be the Developer writing something generic like “test bug is fixed”, “check data is saved“. If it was more detailed, sometimes you would paste the developer’s notes and change a few words – and have no idea what it meant until you saw the finished thing.

The Verdict

I think both Developers and Testers saw Test Specifications as a waste of time. The Developers weren’t enthused to read it, especially when most people rewrote what the Developer provided, and that might not be the full test coverage needed. The Testers should be adding more value by using their existing knowledge of the system to come up with additional scenarios to cover “regression testing”. 

I think the only advantage is to quickly verify that the Developers and Testers were on the “same page”, but that only works if the Tester has not used the developers words and tried to illustrate that they do understand the upcoming features.

I think it eventually got binned off for what we called “Cycle Zero Testing” which was where the developer quickly demoed their changes; which I think was still hated by the Developers but was easier to understand the value, and was a more collaborative between the two roles.