Using the Build Server to test your code

I was having a nosey at Pull Requests, and I saw one that was titled “Code to get the rest of the team up and running”.

I have a look, and after he sent the initial request, there were 2 other additional fixes along the lines of “fix broken build”. The next day, I checked back and there were 7 other additional fixes, again along the lines of “fix broken build”.

9 attempts.

Clearly, he wrote the code, and didn’t test it at all. Didn’t even build it. He just submitted the code, and let the Continuous Integration build run, and fail. Then run and fail, then run and fail… repeat until we all go insane.

They do say the definition of insanity is to do the same action multiple times and expect a different result.

9 attempts.

Still can’t get over it. At what point do you realise it’s best to actually test your changes?

I guess in his mind, he wanted to check in the code as quickly as possible so he can “get the rest of the team up and running”, but surely you have got to realise your approach isn’t working. When your “quick change” has been spread over 2 days; it is no longer quick.

I love it when people cut corners to save time…but in the end it takes 5 times as long as it would have done if they had done it properly.

9 attempts.

But wait, there’s more. Well, there will always be more because he doesn’t learn from his mistakes.

His next set of changes was to add support to handle a new message. His colleague reviewed his code and left a comment like “this line of code will prevent the message from being sent”. Yes that is right, he is adding a new message, and the message won’t even send. Yet he is sending it for review. Obviously didn’t test it; just coded it blind like before.

So he makes the change, makes some other smaller suggested tweaks. The build fails with Code Analysis errors which he would have found if he had actually built his changes locally. You know, step 1 when you begin to test your changes, and precisely the problem he encountered the week prior with his previous change.

The reviewer makes another comment, doubting some other part of his code would work. He replies:

“I think I will have to abandon this review and get this tested.”
Insane Developer

You know what makes this worse? He used to be employed as a software tester.

Apprentice Insanity

An apprentice sends a code review with the following code comment in it:

“Need to check this logic…but right now; I may be going insane. Wait, WTF is this?”

Instantly, I doubt how good the rest of the code is. I wasn’t assigned to officially review it, but I knew the Lead Developer would tear it apart. Not just for that comment, but the Apprentice had written this as a disclaimer:

“There are no new unit tests because this will be submitted later. This work needs to be quickly checked in and tested.”

Even though he had already technically missed the deadline. Maybe he got special permission from the Product managers, but if not, he had at least 2 weeks to work on it for the next release.

Cutting unit tests probably cost him time in the long run. I wouldn’t imagine he wrote all the code and just tested it once. It would be a case of implementing the solution in stages, making tweaks and repeating the same manual tests with each change. If he had written the Unit Tests early, he could have run the tests as he is refactoring and adding features. Then in the end he would have fully tested code which works. Instead, he has untested code which we aren’t confident if it works properly, and wants to check it at the last minute with less time for the Testers to run manual tests against it.

3 weeks later, he still hadn’t checked it in. Probably a good job this change got rejected by a Lead Developer for not having Unit Tests.

I’ve seen this kind of scenario before where people cut tests or a feature that was “nice to have” in order to meet a deadline. They say they will finish it after it is checked in, but usually something with higher priority comes along and they do that instead. If it wasn’t important enough for the original deadline, then I guess it is deemed as extremely low priority by any manager. Then it never gets done.

It’s funny how many code comments you see along the lines of “//Check what this does”, “//Finish this”, “//check the performance”, and then you wonder:

How did this get checked in? (well, probably fast-approaching deadlines like I’ve explained)
What feature is missing? Is there a bug with this code?

If you write unit tests early, then you achieve a higher quality solution earlier on, and you gain confidence in the code.

The worst scenario is: complicated code with no tests, written by an apprentice, who makes a statement which expresses his uncertainty if the code actually works.

New Tester

I didn’t think we really needed a tester, but months ago we put in a request for one. Now we finally have a new Automated Tester!

He is off to an amazing start…an amazingly bad start.

He did a bit of manual testing to understand what we have so far. He logged two bugs which are probably the same issue. On the stand-up, he says they are “minor”, so I assumed minor cosmetic issues.

When I encountered the issues, I would categorise them as major cosmetic. One of them was that the menu panel appears at the opposite side of the screen! How does that even happen? The second issue was that it sometimes didn’t appear at all!

The control is completely broken then. I’d say it’s pretty urgent given how little features we have.

He then starts writing automated tests, and the first one he submitted seemed to be backwards. I told him the test didn’t make sense, he kept on saying “it’s just a sample”. I didn’t understand what he meant, it should be a valid test. It was on the lines of :

GIVEN I navigate to the LogIn Screen
AND I enter valid credentials
WHEN I click Log In
THEN I am taken to the Home Page

although what he wrote was

GIVEN I navigate to the Home Page
AND I enter valid credentials WHEN I click Log In THEN I am taken to the LogIn Screen

I told him the page names were the wrong way around. He was demanding I let him check it in because the test passes; it is fine. I told him if that test passes, I don’t trust the framework.

He then explains that the test finds the title of the webpage is “Home Page”, then you are redirected to the LogIn Screen. After you type your credentials and click Log In, you are taken to the Home Page. However, the check doesn’t wait for the page to load, so at the time it checks, it finds “Log In Screen”.

So the test is crap basically, but instead of fixing it, he specifies the wrong page titles to get the test to pass. The functionality is correct, but the test is Invalid but Passes.

I eventually (reluctantly) agree to let him check it in anyway on the promise that when he adds more tests, he also fixes that. A week or so later, he sends an update. You will definitely think I’m making these stories up.

GIVEN I navigate to the Home Page
AND I enter valid credentials
WHEN I click Log In
THEN I am taken to the Home Page
WHEN I click logout
THEN I am taken to the Home Page

Yeah, that’s right, we are always on the Home Page no matter what we do. How can that pass this time? I’m out of ideas.

Enforcing The Quality Process

If you find a bug, you are supposed to log it. Simple logic. However, people get annoyed when they log a bug, then the developer just tells them it’s a simple configuration option they have missed, or they don’t even have the latest version. It can take time filling in the Bug report, and it’s annoying when you have to almost instantly close it.

So then you get the attitude where people will rather email the problems, and hold off logging it until a developer confirms that it’s probably a valid bug. If the developer doesn’t actively check, then bugs can go unreported; which can be a disaster.

The other day, a Senior Tester posts a massive stack-trace in Slack. I have a quick look and realise it is a bug. As I was about to reply, another Senior Tester, (who is about to be promoted into some kind of Quality Process Manager or some nonsense; or maybe that is a rumour), tells him to repost it on Microsoft Teams. But not just one post, she wants it posted across two channels for “maximum reach”.

Or maybe maximum annoyance.

It’s really dumb, we have this process, and someone that is supposed to enforce the process – and definitely someone that will be enforcing the process in her new role – is coming out with nonsense like posting in informal channels.

If there is a Bug, then log it. It can always be closed with a valid explanation; which can help others in the future if the same problem crops up again.

Doing Admin

Beavis, a software developer, was supposed to be helping out running manual tests since the Testers were behind on their work. That day, he was working at home and posts on Slack that he has “network issues” so was unable to test. He could have picked up some development work instead, but he didn’t bother.

So for his stand-up update, he says he was just doing “general admin” which I reckon meant he was chilling out at home.

A Tester explained to him that when they work at home, they remote onto their work PC, then connect to the test system from there. It’s a bit of a workaround but it’s what everyone does.

Unhappy with this workaround, Beavis decides to log a ticket with our IT Helpdesk.

A few days later, Beavis is supposed to be helping out the Testers, but is working from home again. Yet again, he posts on Slack that there are still “network issues”. A Manager challenges him on it, and he says he has logged a ticket, but it hasn’t been addressed – so he can’t help. The Manager tells him to remote onto his work PC, which is a known workaround.

The next day, for his stand-up update, he said he ended up doing “general admin” again like tidying up his C drive, and he was also doing “personal development”. So completely defying the Manager and other Tester’s advice.

He isn’t the first person to use the “doing admin” excuse. It’s hilarious when you think about it. How can you claim you spent 7.5 hours creating new folders, or sending files to the recycle bin? Even more ridiculous when you have already made that claim in the days prior.

Playing Dumb

Derek always annoyed me when he would play dumb when doing non-programming tasks. So if he was helping Testers by running test cases – they would tell him to open a test case, and he would be like “what software do I use?”, “what do I click now?”, “do I click the Pass button when I’ve finished?”

It just wastes time and frustrates people. I always thought he was being as awkward as possible so they would just tell him not to bother. If he asks all those questions, then it’s just gonna waste a tester’s time. If they are answering questions, then it means they aren’t running test cases. The idea of a developer helping out means there are two people running test cases. But an awkward developer just means there is a developer slowly running test cases whilst supervised by a tester who could have just done the job quickly.

I’m introducing a new character, Beavis; because he always acts dumb, or at least appears oblivious. Also, he has a very Beavis-like laugh.

Beavis was doing the exact same technique. I heard him ask many dumb questions like that. Later, he calls me over to help set something up. I tell him his account doesn’t have enough permissions.

So he logs in with the admin user, goes to the configuration module and then says:

“how do I find my user?”

“click the Find button”

“where is that?”

“In the menu, first button”

he clicks it. A dialog appears. “How do I then find my user? Do I just type in my name?”

“No, you have to type Pi to 10 decimal places!”

I didn’t really say that, but it was hard to keep calm with the questions that are just wasting my time.

He then says “sorry, I’ve never used this program before”. Even though that was clearly a lie.

I know he hasn’t used it much because he used to work on a completely different software product, but he has used it and was helping a Junior set it up the very next day. But when you have 20+ years of experience in the software industry, how can you not be comfortable using basic features? Many features are intuitive. If you wanted to find some text, it doesn’t matter if it is Internet Explorer, Microsoft Word, Notepad, OneNote, Visual Studio; you would always press Ctrl+F. Want to Save? Look for a “Floppy Disk” icon, or the word “Save”.

Full Regression

Several months ago, a project was completed (let’s call it Project X) that must have contained the biggest impact to our software. What I mean is that the scope impacted a lot of features, and therefore to sign it off, the testers ran an insane amount of regression tests. The Project X team ended up running these tests with the help of other teams, and many hours of overtime was needed.

A release was being planned that involved more bug fixes than usual. I think one of the Test Managers had suggested that we run a full regression test to ensure the changes made by Project X hadn’t been effected. Marcus, a tester involved in Project X stated that this was an “unreasonable request, and we need to do a more focussed, targeted test”.

The next day, a Test Manager came up to Marcus and said she had asked someone to give estimates of a full regression and she had been quoted 6 weeks. She said it was absurd and wanted Marcus to confirm it. Marcus stated once more that he had already highlighted it was an unreasonable ask. He explained that she is requesting that 4 people run the tests over a two week period, when the original team had three weeks and around 20 people were involved. This is why Marcus rejected the proposal in the first place.

Another day passes, and another Test Manager announces that, after much debate – running a full regression isn’t feasible, so they will do a targeted regression.

No doubt there were one or more meetings to discuss this, when Marcus had already told them how unfeasible it was a couple of days earlier. It’s just that people with the authority to make the decisions, and the people that actually have the knowledge of how things work – are completely different people.

Gun ’em down

A tester had created loads of alerts and hadn’t disabled them. Jack sent a mass mail to rant about it, stating that any tests that can annoy others should always be cleaned up. He was doing some important testing and having alerts spam his screen was slowing him down and stressing him out.

Later, I sat next to Jack in a meeting and decided to wind him up about it.

timeinints: “Hey Jack, what are you gonna do about this testing situation? This happens a lot and we need to stop it.”

Jack: “What I’m gonna do, is march down to their office and gun them all down.”

The meeting host then points out that members the other office were on the conference call.

Well, that’s awkward. 😀

Care

After I write some code, I always test and inspect it thoroughly. I’ll write Unit Tests, running manual tests, constantly reading and re-reading the code. Any slight change I make, I then end up repeating the process. Even when I’m about to send it to review, I get paranoid that I’ve missed something, so end up looking at the files again to make sure.

It’s often rare that I get any (serious) feedback to change my code, and it’s rare that Testers will find anything wrong with it. But then sometimes I wonder if I’ve spent a little too long in the development stage.

However, Colin and Derek seem to take the opposite approach and end up coming out with something more quick and dirty.

Derek always seems to come up with a solution close to a hack which I’ll tear apart in the Code Review. After, it will end up being rejected by Testing and he will put in another bodge.

Colin spends time writing Unit Tests but often misses some scenarios. His methods often need refactoring down more, and methods/variables/classes may not have the best names. His solutions still come across as rushed, and he seems to cut more corners when the deadline approaches. When I point out his mistakes, he often says “the deadline is tomorrow night, so I just have to get it checked in”.

But surely it’s useless if it doesn’t work? In a similar fashion to “it’s better late than never”, you could say “it’s better that it is late and works, rather than getting it early and broken”.

He got some comments last week to refactor some of his methods. He quickly changes them and submits his new method for review. It was a one-line method, and somehow there were 3 mistakes in it. The thing is, he had Unit Tests covering every possible scenario of that method; he just didn’t run them. Unit Tests take a several seconds for a large batch of them to run; why would you cut corners so much that you don’t even run the unit tests? If you had full coverage, it will allow you to cut manual testing, but to cut testing completely? Madness.

Anyway, his code was something like this:

public string DisplayInfo(bool isScotland)
{
 return DisplayInfo(Configuration.Scotland==Configuration.Scotland); 
}

The 3 things that are wrong:

he is passing in a parameter “isScotland” then not using it
where he should be using “isScotland”, he is calculating it…but wrongly. He is comparing a value to itself; so it always evaluates to “True”
The method just calls itself, which calls itself, which calls itself…until it crashes with a StackOverflowException because it’s an infinite loop.

Absolute Junior-level coding.

Testing Fail: The Config File

A Tester asked me to send them a Test Harness tool which I had been using the previous day. I thought I can either send them it, or remind them how to download it from Source Control and build it themselves. As it goes, I already had the Test Harness in my shared folder, which I’m pretty certain it was from last time when they asked me for it.

I also remembered that was also the time when they had checked out a folder which had several branches in it, and was building the code from one branch, and looking for the output in another.

Anyway, I send them the link to it, then thought it would be best if I actually give them the config file to point to the test system they are using. So put the file into the Instant Messaging app along with the text “this is the config file I used for the test system”. They accept it and thank me for my help.

Ten minutes later, they tell me the Test Harness crashes and asked if I knew why. I tell them that the Test Harness is a bit crap and doesn’t appear to have any error handling whatsoever. However, when it crashed for me, it was because it didn’t have the correct connection details to the server. So I ask them:

TimeInInts: “did you use the config file I sent?”
Tester: “No”.
TimeInInts: “Why not?”
Tester: “I wanted to use the config file you used, because that one works”
TimeInInts: “but the config file I sent you was the one that works”
Tester: “Oooooooooh right, I thought you were just saying that’s what the config file looks like”

HOW CAN YOU BE SO DUMB? If I was telling them where the config file was, why wouldn’t I simply send them a direct link to the file within the appropriate subfolder? Why would I make them download a file which is a copy of what they have already copied to their machine?