Increasing Productivity #3

As we head towards the release of our project, we have a massive suite of manual test cases that need running. The managers have looked at the numbers and worked out the average we need to run per day. We haven’t been meeting the targets.

The managers have decided to try and incentivise us with more rewards, be it money, extra holiday, or treats. We just have to hit the daily target of test cases ran, with a new reward available each day.

In the meeting, our Lead Developer exclaimed “We will hit those targets within 10 minutes”. What he meant was, he was just going to mark the tests as run without actually running them. As explained in “Increasing Productivity #1“; “you get what you measure”: If all you want to see is the tests have ran, then we simply mark the test as ran.

In the blog, I also explained that each test takes a completely different amount of time to run, and some are more difficult to run. So the way I see it; if we carry on just running whatever test case is next, then we will fail to meet the target every day, since nothing has changed. Instead, we could spend time looking at them, sorting them out in levels of difficulty, then all running the easy tests. This means we will meet our target and then claim a reward. The next day we can just focus on the hard ones. Sure we will massively fall short of our target, but at least we bagged a reward the previous day. The net result over the two days will be the same as it was without the rewards. So it will look like this:

	Day 1	Day 2	Total Tests Ran
Normal Approach	70	70	140
Incentivised Approach 100 = reward	100	40	140

When I was discussing this with the Lead Developer, he said that: had he been the one running the test cases, he would have kicked off in the meeting because the system is insulting. The way he saw it, people are trying to do their jobs, and managers are stating it isn’t good enough, and are taking the approach that they are only motivated by money, and not by anything else.

When he put it that way, I realised he has a point there. It reminded me of A-Level Business Studies, when learning about Frederick Taylor’s ideas of motivation; stating that workers are purely motivated by money. It’s quite an archaic viewpoint.

The thing is, if the reason why we are behind is incompetence, then why are we rewarding incompetence, intead of dealing with that problem? If we are doing as expected, then why throw more money at it? I don’t really see any advantage of this idea. It seems weird to complain about something that benefits me; but I just don’t understand it.

Increasing Productivity #2

In the early days of my Testing career, I found that when Test Managers looked at their metrics on how the project is progressing, and thought it appeared we weren’t on target; they would just throw more staff at it.

This seems a simple and obvious solution to the problem, but often they only assigned this extra person for a small amount of time, sometimes even just 1 day. So once this new person has been assigned to your project, you end up giving them a run-down of the project; explaining what the project aimed to achieve, some known bugs (so they don’t waste time investigating and relogging known issues), and then show them where the tests are to run.

So you may have taken an hour to do your introduction presentation, then they have bugged you several times throughout the day, causing time loss, and you lose your train of thought; slowing your progress down further. So at the end of the day, there may have been a net gain in productivity for the team, but it wasn’t twice the productivity that the Test Manager planned. It ended up being more like two people running at 60% effort. But then sometimes they ask you why you weren’t productive. “Erm, it was your fault for making me train someone up”.

Some days I’ve even gone through the who introduction to the project, only for the Test Manager to change their mind and take them back to their original project. All that does is increase stress for your team as you are closer to the deadline with nothing to show for your efforts that day.

Luckily, things have changed since then, and the team structure is more set in stone. Instead if the testers want help, the developers will chip in with running test cases. This means that they are already familiar with the project and functionality, and you can have them running tests as long as required.

Increasing Productivity #1

When I worked as a Tester, we didn’t have any kind of monitoring on our efforts; we were just left to get on with things. This meant some people were productive and did their work because that’s what they were paid to do, and others abused the system and did a bare minimum. Then one day my Test Manager came up with this idea that everyone needs to run 10 Test Cases a day in order for the team to be productive.

How long is a test case though? If each one took 40 minutes to run, then 10 would keep you busy for the day. But some can take that long, and others can take only a minute – so the metric is rather meaningless. Also, some tests often found bugs because they tested brittle areas of the system, whereas others never uncovered bugs. Some tests were fine to run with a few clicks, whereas others depended on certain configurations being set up.

The most annoying configurations relied on third-parties, so sometimes you would have to log a ticket with the Environments Team, then they would contact the third party and wait for a response. This could delay you for days.

So with a demand of “you need to run 10 tests a day”, how do people respond? Well, most people then started to cherry pick the simple tests. Tests that rely on third parties got pushed to one side, as did the tests that often find bugs (logging a bug means you need to spend time investigating, filling out the bug reports, then chasing up the appropriate developer). After speeding through 10 easy tests, people just spend the entire day chilling out. Some people ran 20-30 tests, then tried to boast to the mangers saying “look how good I am”.

People like me that actually cared about getting the release out on time would pick up the important tests. I’d be logging tickets with the Environments team, ensuring we had the correct configuration. I would be running the brittle tests, logging bugs and liasing with the developers to get the fixes. I was the one getting called into meetings, asked on why I was only running 5 tests and failing to meet my targets.

The closer to the deadline you leave things, the more chance the release will have to be delayed. Plus it’s not great telling a developer there is a bug, but they only have a day to get it sorted. It’s not great telling the Environments team they have to drop what they are doing to set up a Test Environment because you left an important test to the last minute only to find the Third-Party dependency isn’t working.

It’s almost like people forget what it is like being a Tester when they progress to management level. As my Lead Developer always says: “you get what you measure”.

Commit Messages

Now that we have moved to Git, our Lead Developer seems to be making more of an effort to promote good “commit” messages. Over the last few months, we have joked about the quality of some of the commit messages in TFS, and how useless and frustrating they are.

“Fix bug 1756”
“Address code review comments” 
“correct previous check-in” 
“fix crash”

All of these don’t describe what has happened, or why the code has changed. With any code change, you always know it has happened to fix a problem, be it: a crash, performance issue, or refactoring to make the code easier to read.

There’s plenty of times when I’ve been investigating a bug, and you track it down to a particular file. Then you view the history of the changes to try and understand at what point the bug was introduced. When you are greeted with any of the examples stated above, then you have to look at each change and look at the linked Work Item (Bug Report). It’s great when you can just glance at the commit messages and can jump straight to the most likely culprit.

Say you are investigating a issue with Threading and you see:

“Minor refactoring of the DisplayInfo method”
“Change threading to use a threadpool” 
“Add null check to prevent crash after clearing data”

You can then just jump straight to the change that clearly states there’s a change regarding threading.

I overheard the Lead Developer tell Derek to write a detailed commit method for his latest change and he says “seriously? I never write a detailed commit message; linking a Work Item is enough”.

I was face-palming. Surely, with a few months development experience, you will learn that good commit messages are very helpful. You could argue it only saves a few clicks and a few minutes here and there; but why make your job, or other people’s jobs harder?

Overtime

In a follow-up to my blog on Work Hours, I wanted to discuss Overtime. From my experience, Overtime has rarely been worth it for the employer, and my strong guilty conscience prevents me from taking advantage.

I haven’t looked into the topic, but the widely adopted “9-5 work hours, 5 days a week” must have come into play after years of theorising and research. Working past this limit will lead to burnout. After working for a few months in the 9-5 structure without additional time off, I do come to the point where I need time away. Working past this point will just push me to that point earlier.

Recently, my project has been approaching the final deadline, so overtime has been available. Some team members have taken advantage of this, whereas others have avoided it completely.

One day, I was helping out testing and was finding appropriate Test Cases to link to the Work Item. I saw that some tests I had previously wrote now were incorrect. I checked the history, and the tester that was doing overtime (I’ll call him Brian) had changed them on Sunday, late at night. He turns up late to work on Monday, looking really fatigued, and several times during the day, he calls me over to explain Work Items to him; which was unusual. So basically, the employer paid him a full days work on Sunday, and then he fails to perform on a Monday when he is actually contracted to work. So it seemed that we gained a day, but then lost it straight away.

There was another Work Item I was looking at, and I saw there was an Overtime Task from Colin assigned to it. Weird, since the tests were written by Brian, someone else wrote the fix, and it hadn’t been tested yet. I check the history, and Brian had removed all the tests that Colin had written, and rewrote them during work time. Colin was claiming 7 hours overtime on it. A full day at the weekend for no benefit.

There was a day where I overheard Colin talking to the Product Owner, saying that he had lost his work over the weekend. Now, I didn’t hear his reason, but I was thinking “how the hell can you lose your work?”. Let’s say it was a hard-disc failure; well, you wouldn’t be complaining that you lost your work – you would be moaning about losing everything. Let’s say it was a server failure and his Shelveset was gone; many other people would be complaining. So the only thing I can think of is that he didn’t create a Shelveset, then undid his changes; which is just incompetence. The only other alternative, is that he had nothing to show for his several hours of development, so came up with a “dog ate my homework” excuse to try and get paid anyway.

Yesterday, I was sat with Colin discussing the work we had left. He then told me that he was sick of the project and felt overworked. Since we are roughly on track, he was gonna take a couple of days off work. He then asks me why I don’t do overtime. Well, I think this proves my point.

We were seemingly behind, he does all this overtime and the end result is that we’d probably be in the same position if there was no overtime available, but now we have fatigued, demotivated team members.

I don’t know what the end result is going to be, but there was a time were promotions were dished out, but I didn’t get one despite it being a widely accepted fact that I was among the highest performers in the team. In a meeting with my line manager, he said the reason why Tim was picked over me, was that Tim did overtime when the company really needed it. I was like “WTF, I’m contracted to do a job, and I do it in normal hours, or stay behind and get it done. Tim wastes time doing person shopping on Amazon, then when his work isn’t done, he volunteers to do overtime at an extra cost to the company”. Nice guys finish last.