The Changes #1: Attempt To Replace Manual Testing

When it comes to Testers, we have hired some fairly non-technical people as “Manual” Testers, but then on certain recruitment drives, we have hired “Technical” Testers which performed manual testing but also have the scope for Automation testing.

When we announced that we would soon be starting projects that will build our next product, managers stressed the importance of defining new processes and upskilling with new programming languages. Moving into the “new world” where you expect things to be automated; there isn’t much need for non-technical people.

I did warn one of my Tester friends about this – that his days could be numbered, but he didn’t listen. He waited for several months, and then seemed to think “damn, I’d better get learning because there is literally no work otherwise“. Meanwhile a few people requested a promotion so they can be a manager, which is unfair really; but there’s always people that benefit from restructuring.

Anyway, another year has gone by and the projects have been a disaster. I don’t think they really went through with their “no need for manual testing” approach, although some people did move back to support the existing product to do manual testing on a familiar product.

In comes a new CTO and a new Head Of Development, and now, yet again, their new idea seems to be 100% Automation.

I expressed my concern to my manager. It didn’t go to plan last time, and you may find that some manual Testers simply leave the business since they have no plans/desire, or might not have the ability – to do automation.

Earlier in my career, I used to do manual testing and it soon became a tedious and boring job. If you actually find people that do it for a career, then they are probably worth keeping around. In my opinion, you always need manual testers.

Another thing is that you are assigning an inappropriate job title to those people. The automation testers have the weird job title of “Developer In Test”. If you are currently a non-technical, manual “Senior Tester”, how can you suddenly be a “Senior Developer In Test”? This implies that you not only have technical skills, but you have achieved Senior level in it. It’s a joke. Demoting the Senior to the basic “Developer In Test” would make sense but would probably cause them to be disgruntled and leave.

When I previously wrote about job titles, I used Colin as an example. I’d say he was an underperforming Developer, but got promoted to Senior. Simply calling him a Senior doesn’t mean he has better skills. Then another couple of years pass, and somehow he gets promoted again to Principal Developer. I wasn’t happy about that, and questioned my manager. He could only explain it as “we had a vacancy and a strong need for a Principal in the team”. But if Colin was already in the team, what good does promoting him to Principal do? He doesn’t level-up like a computer game character and gain extra productivity, coding skills, code reviewing skills and leadership. We needed to recruit externally to add another person to the team, but also increase the skill level.

So with these new changes, I asked my manager, “why are we changing job titles?” and he said “we are following a new Agile framework that the CTO is implementing”. Surely we can still follow the framework without copying job titles word for word? All it does is cause some staff to be disgruntled.

When another manager was explaining the team structures, he said

“there’s a thin line between “Software Engineers” and “Developer In Test”
Manager

He said that means that Developers can help Testers but then seemed to struggle how to justify how Testers can help Developers – because it turns out they aren’t actually the same.

Conclusion

New managers like Heads of Departments or CTO’s love making changes.
100% Automation is a dream. I think you always need Manual Testers.
Changing job titles can cause people to become disgruntled.
1. It could feel like a demotion.
2. It could feel like more responsibility for the same pay.
Assigning a job title to someone automatically doesn’t give people those new skills.
Testers don’t have the same skills as Developers.

Test Coverage

The “Test Governance Lead” announced that all developers will need to report on Unit Test Coverage. We explored similar demands several years back, but we soon realised it was a rubbish metric. How do you measure Unit Test coverage? There’s a few ways and they all have limitations on their accuracy and usefulness. Teams were also adding “attributes” to the code (ExcludeFromTestCoverage) which also manipulated the figures.

My opinion is that it’s okay to report on as long as you just look at the overall trends and don’t overreact to the actual figures. If a figure says 50% and there’s never any problems with the code, and the figure is still 50% in 3 months time; then it is all good. The problem is that managers just look at the figure then start putting pressure on developers to do something about it because they want the number to be higher.

An experienced developer responded to the “Test Governance Lead” with a blog post which opens with the line

“Let me use one of my son’s toys to explain…”.
https://danashby.co.uk/…/code-coverage-vs-test-coverage/

What a great opening line for a blog. It’s like a passive aggressive statement from the developer there. He didnt directly insult the “Test Governance Lead”, but indirectly called her naive/dumb by using the blog.

Years ago, on multiple occasions, I witnessed a developer talking to one of the high-ranking managers, boasting about his 100% test coverage. He was obviously trying to manipulate the manager’s impression of him to increase the chance of promotions.

Not so long ago, there was a team that kept on boasting about their 100% test coverage, yet developers using their code library were telling them that major functionality was broken.

Requesting Help – But Not Providing Enough Info

Becky is a Software Tester with maybe 15 years testing experience, so you think she’d know what she is doing.

When I was a Tester, I quickly learned what the developers wanted in order to help you. The Message and Stack Trace is mandatory if the software crashed, but it’s always great if you can explain what you were trying to do to “set the scene”:

What you expected to happen,
and what actually happened.

Then, if possible, state specific steps to consistently recreate the issue. This is what Developers need to fix a bug, but the same information is great if you want help after failing to configure a server/feature etc.

In this situation, Becky had what sounded like a straight-forward task, and under normal circumstances, you could just log in to the software, fill in a few fields and click save.

She posts on Slack for help, and states she thought this task would be simple but it’s “not the case”. Then says with “Alan’s help, I’ve changed a config file”, and she was trying to use a configuration program but was “unable to connect”.

I’m reading her message and thinking “why has she changed this config file?”, and “what has the connection error got to do with anything else she said in the message?”. She should be able to do all this using our software.

So I message Alan to translate what she said. He did explain it wasn’t that simple, and so they were doing this configuration an alternative way. He said the current problem was just connecting to the server, and he had told her to log a ticket with the Networks team. He said after that, he would carry on helping her.

So I reply to her message on Slack, stating that Alan is still helping her.

She then says that she “knows there is knowledge within the team and didn’t want to take up any more of Alan’s time”.

I thought she was just wasting other people’s time by trying to get other people unnecessarily involved. Alan had helped until this blocking issue occurred; which Becky needed to get the Network’s team to sort out. There’s no point wasting other people’s time.

Since I had already invested some time into it, I decided to ask her some questions. I wanted to know the IP address of the server she was having trouble with, the IP address of the system she was initially configuring, and a database ID so I could actually see if she had the data in the correct tables.

She only answered 1 of my questions, and her response was a slight rephrasing of the thing I questioned. So she wants help, but won’t give me the info in order to actually help. At no point did she say that Alan had instructed her to log a ticket, so she wasn’t even following what Alan told her.

I find that there are a lot of Software Testers that fail to give you enough information to do your job. Somehow they think you can magically work out what they intended to do, and work out what the problem is with barely any information.

Gibbon Testing

In my first programming job, I worked for a small company (under 10 employees in total). We had no Software Testers, and we definitely didn’t have any testing process. I don’t recall any automated tests either; I certainly didn’t know how to write them at the time (not even Unit tests).

When I finished my work, I informed the Senior developer, and I would upload the software to a touch-screen Terminal. He would casually play about with my new features. If he was happy, then the CEO would do similar tests; just casually explore and verify I had done what I claimed. Once he seemed happy, he did his final test: “The Gibbon Test”.

You could say this is some kind of performance testing, and also testing out multiple features in a short amount of time. What did the testing involve? Well, just frantically bashing on the screen.

I’m not sure how serious he took the testing. I mean, obviously it was a joke and we all laughed watching him do it. However, on reflection, I think it is actually a good idea – definitely a good one to do last after you perform all your serious tests.

With certain applications, the user could be frustrated and hit the screen randomly. Your program should be resilient to such abuse. I think a good example would be some kind of gambling game. When the user loses their money and hits the screen in frustration – the application shouldn’t crash, or take ages to respond. It should be ready for the next person to use.

Tester of One

Penelope, a Software Tester was really disheartened. A bug had been discovered and she felt that she could have found it. Since she was on the project by herself, she holds herself to blame, rather than sharing blame in the team.

Penelope said that she had raised the concern that she was the only tester on a 6-month project but it was ignored by management.

I said that if she wanted to take annual leave during that time, then the team would be impacted. She said that actually happened and the release was delayed by 1 month. Management have to take part of the blame here.

I also said that even though she is the only tester, the Developers do need to share accountability too because they are all working towards the goal of quality and can’t just rely on the Tester.

I did question if anyone was actually blaming her anyway, or if it was the case of Penelope blaming herself which it did seem to be. I did understand how she could feel like she could have done more, but that negative outlook is bad for your mental health.

I stated that she should think of the bugs that she did find. If she found 50 bugs, then what’s the point being down over the 1 bug that was found by users? She has justified her job by finding the other bugs.

I was watching a video recently where former England and Arsenal football goalkeeper David Seaman was recalling the time he conceded a long-range Ronaldinho freekick against Brazil in the 2002 World Cup. He felt he had made a mistake (by being far off his line), and from that very moment was thinking about the backlash and criticism he would then receive from the media and fans – even during the game. He was putting his hopes on the team scoring a couple of goals to redeem him. As he was explaining this story, I wasn’t thinking “yeah that mistake cost us the World Cup”, I was actually thinking “he was such a great keeper, he probably was a major factor in England even getting that far”. A goalkeeper is such an important position, because a mistake often leads to a goal, whereas an outfield player makes a mistake and there’s a few teammates plus the goalkeeper to redeem him.

Really, the mistakes are hopefully rectified before it gets to the last line of defence, i.e. before it reaches the Software Tester/Goalkeeper. If they save the day, then great. If not, then the team has failed. You can’t blame the last line of defence.

As it goes, we could easily blame Paul Scholes, he made no attempt to get the ball and gave Brazil a cheap free-kick. 😁

How To Make Your Team Hate You #3

I was thinking about the colleagues I’ve legitimately despised (and it’s not just my opinion), and most of them have something in common: they have either been promoted, or told they will get promoted if they prove themselves, resulting in a change of behaviour. So here is story number 3, and hopefully the last one since I don’t have any more drafted up.

The Situation

I was once on a project where I was the only software tester. Usually, when it’s a proper project where it is going to span 3 months or more, then we would have at least 2 testers assigned.

I was well excited to show that I could handle this responsibility. The project went well, and I was well proud. That project was actually a prerequisite for something bigger, and yet again, I was looking forward to the challenge of the follow-on project.

Collectively, both projects were going to be a replacement for part of our software, but it was going to be phased over time. Therefore, it was important that the old version still worked.

So I looked at what the current system did and wrote all the features down. I then spent ages looking through all the Test Cases, comparing these to the features that I documented. Then I had to write the Test Cases that were missing. This probably took around a month of preparation.

Since the new version needed to do what the old one did, but might have done it in a different way, I basically had to duplicate every test, then start changing the steps accordingly.

Having this 100% complete test coverage worked well, and I knew people would be thankful for it in the future. Anytime a feature is changed, you know there are tests that you can run for your manual regression tests. I think every tester has been annoyed at some point; when they think a test should exist, but it doesn’t, so they have to begrudgingly write it.

I’ve done the work for them. I don’t think any other area of the system had this level of test coverage. No one can be annoyed about this can they?

As we moved onto the second project, I was told another Tester, Barbara was joining the team. I’m not sure if I was told explicitly, or if I just heard rumours, but I’m pretty sure I knew it was the case that – if Barbara did well on the project, then she would be promoted to Test Lead. Based on my performance over the years, and based on the project I had just finished, I think it should have been obvious to promote me to Test Lead. Maybe I was slightly annoyed, but I don’t think that had any affect on what happened next…

The Conflict

So Barbara joins the team and looks at the test cases. She wasn’t happy. Despite having that 100% test coverage, the tests were “too granular”. She says that when it comes to testing a particular feature, we would end up adding maybe 10-20 tests from my pack. She only wanted to run 1 or 2.

So she starts complaining at me, telling me I’ve wasted my time and I need to rewrite them. She likes the idea of “end-to-end tests” where one test actually tests several features and could describe what a user would do over a 5 minute period.

For example, I may have written a test that describes how to create a document, then another test how to print a document, then another test on how to email a document.

Her way of testing would be to include all the instructions for these 3 features and put them in one test – since a user may log in, create a document, email it to someone, then print a copy out for themselves.

I argued that my way is better, because if there is a bug involving email, you can just pick out the short test involving email. I’d be annoyed if I had to test printing it when it has nothing to do with the email bug fix.

“But that’s not how we did it in my old team!”

Yeah, but you aren’t in your old team. You’ve joined this one, and this is our process.

She used this statement multiple times about different aspects of how we work. I kept on explaining how if we work “agile”, we are supposed to come up with a process that suits us, and agree as a team. This process suited us, or at least it worked perfectly well before she joined our team.

Yet she comes in, kicks off a massive fuss to try and put her stamp on the team, and bypasses the correct channels for these decisions; which would have been the Retrospectives. I would have been happy to debate it in a meeting, rather than her dictating what I should do in front of everyone.

What Barbara did next was – run off to her manager, who happened to be my manager as well. But because my relationship with my manager was simply a professional one, and Barbara was very chummy with her, then our manager instantly backed Barbara.

So then I had to plan out which features made sense to use after each other. I spent 3 or 4 days planning this out. I spent more days writing them. I only wrote about 7. How many test cases did she write?

Zero.

Didn’t practice what she preached.

I was fuming.

When it came to testing the new features, she often volunteered to test the easier items, and gave me the harder items. I didn’t mind, because if she wasn’t on the project, I would have just done them all anyway.

What was infuriating, was that the next day, she would chat some nonsense about how she was too busy so didn’t have time to test it, so then asked me to test it instead. So I’d end up doing all the work. It’s just that it would take longer because I’d finish my work and have nothing to do, then the next day, I’d have to do her work that she was supposed to do. So the result was that we had 1 extra tester in the team, but productivity had halved.

She did this for weeks. It wasn’t a one off. Often, I’d look at her monitor and saw she was sending personal emails, doing online shopping, or just looking at news.

Then when it came to the stand-ups, where we stated to the rest of the team what we had accomplished; she then claimed that she did the work.

I was fuming. How could she say that when I was standing there with her? It’s plagiarism, and absolute lies. I complained to my manager. She refused to do anything. I was so tempted to walk out.

I’d never had that feeling of not wanting to go to work before, but some days I just didn’t want to go. I actually started looking for new jobs.

Sometimes when Barbara was away from her desk, I’d be ranting to my team. We were a close bunch on the initial project, but it just didn’t feel the same way. Soon I was brought into a meeting with the project manager who seemed to suggest I should apologise and try and get on.

I don’t get it. We were all close and worked well together. We delivered a project successfully on time. A new team member joins. Testing productivity is halved. There’s massive friction. Who do you think is the problem? You could easily verify my claim that she wasn’t doing the work. The test case runs are audited. My name would be on every one.

It seemed that Barbara had just sweet-talked everyone and they were determined to promote her regardless. As a Test Lead, you are supposed to have testing skills, people-skills and some management skills. Barbara wasn’t showing any of it. She was refusing to run test cases and she was causing friction. Productivity of the testing side was lower than when it was just me on my own.

Epilogue

So what happened in the end? Well, I was a bit unsure what I wanted in my career. I initially wanted to move up the ranks as a Tester, but it wasn’t happening. I did get the craving to be a Developer, and considered becoming an Automated Tester. I requested a change of role to a Developer since the Automated Tester role didn’t exist in the company. I’d be in the same department, but I knew I’d switch teams so would be away from Barbara. I just had to put up with another couple of months working with someone I despised.

I did get my internal move in the end. Barbara got her undeserved promotion. Years later she got made redundant. Better late than never.

For more in the series, see: How To Make Your Team Hate You #1 and How To Make Your Team Hate You #2

Software Testing – Can developers test?

A software tester created a presentation about “the role of a tester”, and discussed whether “software developers” make good “software testers”. I thought the presentation was a bit confusing, because in some parts it sounded like she was saying they can, but in other places, it sounded like a categorical NO.

I found this quote recently from Ron Gilbert, creator of computer games like Monkey Island and Thimbleweed Park. It covers “game development” specifically:

Don’t believe the myth that programmers can test their own code. They can’t. Programmers will test for all the conditions they can imagine, but a good tester will imagine many many more and players will try things programmers never even considered. Knowledge of their code is the achilles heel of programmers. I’ve had code I was 100% sure was rock solid, only to have one of our testers reduce me to tears. A good tester excels at poking your code in places you never considered. I’m not talking about your unit tested sort routine, I’m talking about complex puzzle logic and odd UI uses. It’s the stuff unit tests will never catch, but a good tester will. Testers are the unsung heroes of your team, treat them very well.
Ron Gilbert – https://grumpygamer.com/delores_dev

This is the way I see it:

There’s the concept of being “close to the code”. So the developer that wrote the code will test exactly how they implemented it – and not test the cases they missed; because they never thought of those scenarios in the first place.

A tester inside the company could be influenced by their relationship with the developer; have some unconscious bias about not wanting to find a flaw in their code because they don’t want to upset them.

A tester for a third-party has no connection to the code or the staff at the company. They can test it without any bias. Finding all bugs is a good thing, because it enhances their reputation and should help with repeat custom. This is a good rationale for why some companies choose to outsource the Testing aspect entirely.

When I have worked in small teams and there is a testing backlog, sometimes the developers will help out with the testing, but they won’t test their own work; they will make sure it was done by another developer. This helps alleviate the above point slightly.

I have heard many developers say “they aren’t testers”, “I don’t know how to test” etc and for the most part; I think that’s nonsense. If you understand code and how to use software, then you understand how to test it. Sometimes it is directly transferable skills, but I think sometimes you need to take a step back, “put on another hat” and try to look at it from another perspective. For example, you then think “If I was a user, what would I want to do?”, “if I hated the company and wanted to find a bug in the code, no matter what it is, what would I try?”. You poke it in different ways. I think if you fail in the skill of being able to “put on a different hat”, then Ron Gilbert is 100% correct.

Personally, I have experience as a software tester, and as a developer. I do think I test my own code better than many other developers – ones without that experience. However, I don’t think I test as well as I used to when I was testing someone else’s code. It mainly comes down to the point I mentioned earlier – the fact that you only test what you know you have implemented and don’t test the things you haven’t. I mean, if you did think of it earlier, then you would have implemented it.

Report API fail

A tester says they are running a Test Case where they need to generate a report using our software. This report utilises a third-party service. The report fails to generate.

There should be errors displayed when you view the details, but it’s completely blank. So if they have correct data, then it’s a bug because it doesn’t work. If they don’t have correct data, then it’s a bug because it doesn’t tell the user what the problem is. It’s a bug either way.

Becky, a Senior tester:

Asks him if he has the correct data. Completely misses the point.
Then she posts a follow-up question asking him if he has gone directly to the third-party website to get the information. Completely misses the point.

No, Becky, the functionality is that our software contacts the third party API to retrieve this information. Manually going to the website and filling a form out by hand isn’t helpful, or acceptable. It isn’t going to make the test case pass is it?

“ok, we will pass the test because the website works. Nothing to worry about.”
Becky’s mind

Sometimes I think she tries to be dumb on purpose.

Disable all the services

I was talking to one of our Test Environments Engineers and he was absolutely raging. He said he had spent the entire day looking into a ticket someone logged that stated they could no longer connect to their server.

The Engineer said he was panicking because he thought they had been hit by a malicious virus. All the services on the server had been disabled which is why you couldn’t connect to it by standard means.

After a bit of investigation and quizzing the person that logged the issue, they admitted to some key information. The server had been slow, so they had disabled every single service on the server in an attempt to speed it up…then rebooted it. But since many of the services are vital for the server to function, or at least connect to the server; then the server was pretty dead.

The Engineer was raging that the person that logged it failed to mention this on the ticket, which would have been vital to work out what was wrong and how to fix it. Surely he realised it was his fault that the server was dead, it’s just that he wanted to try and cover up his mistake.

User has been deleted!

In previous blogs, I’ve told a few stories about how bad a software tester, Becky, is at communicating. Sometimes, it seems like she is trolling. Here is another story:

“My user has been deleted when I clicked the refresh button, how do I undo?”.
Becky

Well, I think you should have panicked and logged a high priority bug if that was the case.

There’s obviously more to this than she is stating. Your user doesn’t mysteriously vanish by clicking a button which says something other than “Delete”. What makes her think she can undo this action? Why does she think this is normal?

I replied, asking her for more information: What was she trying to do? What does she remember clicking before it happened?

Becky ignores my requests and directs her messages to a select few software testers.

What are the testers going to do? If there’s a serious bug, it needs to go to a developer to investigate, and promptly!

Why was she ignorant of my attempt to help? It made me think if she doubted what she was writing, and deep-down; she knew her user hadn’t been deleted afterall.

I decided to play around with the software and click some random buttons, but I noticed something peculiar pretty quickly.

It turns out there was a bug in the displayed list of users. Her user existed, it just didn’t show it unless you selected a different filter button; then reselect the initial filter.

She didn’t bother logging a bug. It’s literally her job as a software tester.