Performance Tales: SQL performance

Batches

Performance of SQL database queries is an interesting topic that I never understand. When dealing with a large amount of data, we are often told it is beneficial to process the results in batches. So if you are wanting to update/delete rows based on some calculations, for example; we may be told to iterate through groups of 1000 rows.

You can’t just put your query in a batch then assume it is efficient. There was one attempted data fix where the reviewing Database Expert reckoned it would take 2h 46 mins to execute.

“It doesn’t matter if you select 1 row or 1m rows – it still takes 10 seconds. So 1000 batches x 10 sec = 2h 46min processing time just working out what to delete. We need to change the proc so that it gets all 1m rows in one hit at the start into a temp table (should take 10 seconds) and then use that list to work from when batch deleting the rows”

Database Expert

Performance Tales: LINQ to SQL
  

Although we mainly use SQL Stored Procedures to retrieve data from the database, some of our code uses LinqToSQL which uses code to dynamically generate SQL. Sometimes it seems that handcrafting your own database query (Stored Procedure) can give better and consistent results since the SQL is the same every time. Sometimes the performance issues isn’t with the actual query, but possibly that the columns don’t have an appropriate Index applied to them to allow fast lookups on that column.

There was a problem where our default 30 second timeout was been reached for some users. So if they attempted to load a large record, it would take 30 seconds then error out. One developer suggested to increase from 30 seconds to 5 minutes.

ā€œi’d like the customers to still be able to see the user’s record.”

Developer

The record would eventually load for the user (presumably), if the user has actually waited for 5 minutes and not closed the application, thinking it has crashed (often the user interface in this example would just show “Not Responding“).

This idea doesn’t fix the root problem though, and the software still seems unusable for the user’s point of view.

An Architect stated:

“there is something very wrong with the query if it’s taking longer than 30 seconds, particularly when the use case of using LINQ2SQL is commonly for small batches of data.

Software Architect

The default is usually a good indicator that something is wrong, but if we increase the timeout across the board we will miss out on such problems.

The reason the timeout is being breached needs to be investigated:

  • Is it general: applying to any query seemingly at random?
  • Is it during the connection to SQL Server or during the execution of a query?
  • Could a particular problem query be optimised, be made asynchronous, or the timeout be altered for that individual query?

The Database Expert stated:

Nothing by default should take over 30 seconds – it’s a good default. If we have particular problems with SQL taking over 30s it should be investigated and addressed. It is possible that some features should be allowed to take over 30s (e.g. not user facing, known to take a long time). Allowing >30s means more chance for blocking and wider impacts.

Having queries run longer than 30s increases the amount of time they are running in SQL – this could lead to more blocking, more CPU/memory demand which could make the entire server unreliable, so we go from 1 user complaining to many.

LINQtoSQL can be optimised, we’ve done it many times over the years. The simplest is to replace the queries with Stored procedures – but it is possible to improve bad LINQtoSQL too. It depends what is wrong. None of daily operations should be anywhere near that. 

SQL can do near 0ms operations on many more rows of data than we have. It isn’t a problem because we have more data, it will be something with that query. Poor plan choice or Blocking.

Database Expert

Performance Tales: Limit User Input

A developer looked into an issue related to our Searches module which allows the user to query their data and generate tables and charts to create their own reports. The Developer claimed their new changes gave performance “up to 10 times better

The Database Expert looked at his changes and stated:

“I think that this might not be the right way to approach this issue in isolation. The issue with this procedure is not that it is slow in general, it is that our software allows you to call it in a huge variety of ways, some of which can only produce terrible execution plans as it stands. We should be analysing what kinds of searches that users do, and see if we can restrict some of the sillier searches that they can do. For example, users can right now do an audit search filtered on “anything before today’s date” with no further filters.

For example, see this snippet of a live trace done right now: <shows screenshot>

Clearly, some calls are very well optimised and others are terrible.

We should be looking at restricting what users can do, for example requiring people to search on a more focussed timeframe.

The way I would approach this issue would be to look at trace data and look for two main things:

-The styles of queries that are run most often

-The styles of queries that run the worst

Both need focus. The ones that run most often, even if they are relatively quick, can yield a big improvement simply from scale alone. The ones that run the worst improve the overall user experience as well as the impact on the system.

Of course improving a query might not involve any SQL changes at all, instead they might involve app changes to prevent silly queries from being run.”

Database Expert

Keep lowering the number with each MI

There was a Major Incident regarding a feature. The stored procedure used already had a limit on the amount of data selected in one go. A developer changed the number from 50 to 10. Since it is a very small number, I couldn’t understand why such small numbers made a difference. There was a code comment next to this limit to say it was there to “improve performance”. I looked at the file history to see if I could find any more information about why this line was added in the first place. I saw a very interesting trend, where that line was the only line to be changed in the previous 2 revisions:

original: select top 250

24 Sept 2015: select top 100

2 Oct 2015: select top 50Ā 

25 May 2022: select top 10Ā 

The developer did explain that the data is passed to another process that takes 90 seconds to process 10 instances. It’s essentially a queue that is polled every 10 minutes and would only have a small number of tasks each time.

The concerning thing is that the number keeps being lowered and the performance is not deemed good enough. Maybe the overall implementation needs revising.

Infinite Scroll

An Automated Tester posted the following

“I hope you want to test infinite scroll by scrolling down to the last element. You can use mockIsIntersecting(dom.getByTestId(“datatestId”), true) from library react-intersection-observer/test-utils to test this.”

This made me think deeply:

Is it using infinite scroll if you can scroll to the last element? šŸ¤”

Me

Maybe in most situations, an “infinite scroll” feature isn’t truly infinite unless it is dynamically generating content. But in some cases, it’s near infinite, like in Twitter where your feed could just go on for seemingly forever because there is far more content than a human can read.

However, in that case, you could never do a test on real data that scrolls to the bottom. If you had mock data, then you could test that scrolling with limited data does reach the bottom.

I told a colleague about this and got him thinking about other claims of infinite. He said it’s like the grains of sand idea. There’s a massive amount but it’s not really infinite. Or the size of the universe; constantly expanding, but at any given time; is finite.

Mike: A line on a graph that extends out to infinity, it doesn’t though; it extends out as far as you graph.

Me: Buzz Lightyear can go beyond infinity (“To infinity, and beyond!”).

Growth Mindset

Recently, our CEO has become obsessed with the idea of “Growth Mindset” which she seems to crowbar into all her company updates, and we received a talk from an external speaker. The actual talk seemed like a lot of waffle to me, but the general idea sounds like the type of life lessons that Simon Sinek says (he calls his philosophy “The Infinite Game“). Although I normally think ideas/mentality like this are pretentious, I respect Simon and I think there’s probably something in this way of thinking, so shouldn’t be put off by the external speaker’s presentation. Many colleagues stated they were aware of the idea of Growth Mindset and cited the book “Mindset” by Dr Carol Dweck.

Join us on the latest and most exciting stage of our journey with Growth Mindset. We have been working with the NeuroLeadership Institute on embedding a growth mindset culture in order to increase employee engagement, equip our people with tools and techniques for personal and professional development and to help everyone in the organisation navigate change. We have been running Growth Mindset sessions with our leadership team since September last year and we will be making a set of incredible tools and resources available for everyone to benefit from.

Company announcement

Growth mindset is thinking that every day is a learning experience and you don’t know everything. When you change your perspective, it changes outcomes. You have to allow for failure which is a learning opportunity. Don’t strive for perfection. In Software Development, you may never release software if you always strive for perfection. 

“Growth Mindset can be applied to all aspects of your life. It’s easier to think about what a growth mindset is by thinking about what it isn’t. So if you think about a fixed mindset, a fixed mindset says this is the way it’s always done and this is the way it will always be. I can only do this, I can’t do that. I don’t want to share. I don’t want to learn. I’m in my swim-lane. I’m not getting out of it, and I’m not interacting with anyone else. A Growth Mindset is the opposite of all of that. It’s like, we’ve done it this way, but there are other ways to do it. I can learn to do anything. I believe that I can learn to do anything that I put my mind to. It’s genuinely sharing in other people’s success, making it not just about you. The fact that other people have achieved something based on what you’ve kind of started or an idea that you might have had. Let yourself be happy about other people’s success.

NeuroLeadership Institute

The application of a growth mindset is not confined to any single aspect of life. It can be equally impactful at work, in personal relationships, or in pursuing personal hobbies and interests. The concept of a growth mindset is a transformative and powerful approach to personal development. It’s the belief that one’s abilities and intelligence can be developed through dedication, hard work, and perseverance. While we may have inherent talents, our abilities are not fixed. Unlike a fixed mindset, which limits potential and discourages risk-taking, a growth mindset thrives on challenge and sees failure not as evidence of unintelligence but as a springboard for growth and for stretching existing abilities.

I believe the growth mindset is an antidote to cynicism. I hate cynicism – it doesn’t lead anywhere and worse than that it spreads. Nobody in life gets exactly what they thought they were going to get. But if you work hard, smart and you’re kind, amazing things can happen!Ā 

Joshua (colleague)

A new scheme, which I expect to be fairly short-lived is the idea of mentorship. Several people across the business put themselves forward to be a mentor and one person could ask them to be their mentor. To put yourself forward, you had to create a profile of your skills to show what you were offering mentoring support on. One colleague came up with the following:

Key skills, experience and behaviours:

  • Growth Mindset
  • People Management
  • Interpersonal skills development
  • Commercial awareness and negotiation
  • Presentation creation and delivery

What sort of person puts ā€œGrowth Mindsetā€ at the top of their list of skills? Surely number 1 should be your top skill. Seems like he is just sucking up to the CEO.

“Temporary” Check-ins

In Software Development, sometimes you have to be pragmatic and choose a sub-optimal solution in order to get some functionality/bug fix out the door urgently. In this case, you sometimes log a work item with a better idea to implement. However, because there’s a lack of ā€œbusiness valueā€, other work gets prioritised and this work never gets done.

XKCD

XKCD joked about how this seems to be a common phenomenon. You think the feature is temporary or low use, and it becomes a thing that the product relies on. Sometimes the opposite scenario happens where you expect your code to be reused/enhanced/adapted so come up with a great design, using some appropriate ā€œdesign patternsā€ and ensure it is well tested and documented. However, it then doesn’t get used like you thought.

https://xkcd.com/2730/

Dave Plummer

Dave Plummer, a Microsoft developer, wrote this story about how his Disk Format dialog was meant to just be temporary. After all these years, it still remains in use.

I wrote this Format dialog back on a rainy Thursday morning at Microsoft in late 1994, I think it was.

We were porting the bajillion lines of code from the Windows95 user interface over to NT, and Format was just one of those areas where WindowsNT was different enough from Windows95 that we had to come up with some custom UI.

I got out a piece of paper and wrote down all the options and choices you could make with respect to formatting a disk, like filesystem, label, cluster size, compression, encryption, and so on.

Then I busted out VC++2.0 and used the Resource Editor to lay out a simple vertical stack of all the choices you had to make, in the approximate order you had to make. It wasn’t elegant, but it would do until the elegant UI arrived.

That was some 30 years ago, and the dialog is still my temporary one from that Thursday morning, so be careful about checking in “temporary” solutions!

I also had to decide how much “cluster slack” would be too much, and that wound up constraining the format size of a FAT volume to 32GB. That limit was also an arbitrary choice that morning, and one that has stuck with us as a permanent side effect.

So remember… there are no “temporary” checkins šŸ™‚

Bing

I gave the following prompt to Bing to write a blog post

ā€œ write about how in Software Development, you may come up with a temporary solution, but it ends up being used for years.ā€

The Permanence of "Temporary" Solutions in Software Development

In the fast-paced world of software development, the term "quick fix" is often synonymous with a temporary solution intended to address an immediate issue. These solutions are typically devised under the pressure of looming deadlines or critical bugs that need swift attention. However, despite their intended short-term application, these "temporary" solutions often become permanent fixtures in the codebase, lingering for years beyond their expected lifespan.

The phenomenon is so common it has its own adage: "There is nothing more permanent than a temporary solution." But why does this happen?

Firstly, there's the factor of immediate relief. A temporary fix that resolves a pressing problem can reduce the urgency to replace it with a more robust solution. Once the immediate pain is alleviated, the incentive to revisit the issue diminishes, especially when other, more pressing issues arise.

Secondly, there's the issue of risk versus reward. Modifying a working system carries inherent risk. The fear of introducing new bugs or the potential downtime that might come with refactoring code can deter developers from replacing a temporary fix that works, albeit imperfectly.

Moreover, as time passes, the context in which the temporary solution was implemented may be lost. Documentation may be lacking, or team members may move on, taking their knowledge with them. This loss of context can make the thought of unraveling a temporary solution even more daunting.

Another contributing factor is the nature of business priorities. Often, the push for new features and visible progress trumps the less glamorous task of maintaining and improving existing code. This can lead to a culture where temporary fixes are not just commonplace, but expected and accepted as the norm.

However, the reliance on temporary solutions can lead to technical debt, a metaphorical term that reflects the implied cost of additional rework caused by choosing an easy solution now instead of using a better approach that would take longer. Like financial debt, technical debt accumulates interest, manifested in the form of compound complexities that make future changes more costly and time-consuming.

To mitigate the permanence of temporary solutions, development teams can adopt several strategies:

- Prioritize technical debt: Treat it as an important factor in the backlog, not just an afterthought.

- Improve documentation: Ensure that the reasons for temporary solutions and the intended long-term strategy are well-documented.

- Foster a culture of quality: Encourage practices that balance the need for progress with the importance of code quality and maintainability.

- Regular refactoring: Schedule time for refactoring and improving the codebase to prevent temporary solutions from becoming entrenched.

In conclusion, while temporary solutions are sometimes necessary in software development, awareness of their tendency to become permanent can help teams take proactive steps to manage them effectively. By doing so, they can maintain a healthier codebase, reduce technical debt, and ultimately deliver better software.

SonarCloud, the Static Analysis Tool

SonarCloud is a static analysis tool. It runs on your codebase and points out where code can be written to ā€œbest practicesā€, possible bugs, and code coverage.

When my employer bought a SonarCloud licence, there was a large emphasis that they wanted existing errors fixed, and no errors in new code. I knew this would be a bad idea to emphasise because of the classic ā€œyou get what you measureā€. If you tell people the goal is to not have errors, then they will do whatever they can to not have errors. The idea of a tool like SonarCloud is to improve code quality, but since the metric is a number, then the number becomes the goal rather than code quality. However, the aim should be to improve the quality of the code, optimised for readability, maintainability, scalability, and code coverage.

As more people began working with SonarCloud, we saw more changes with the title along the lines of ā€œFix Sonarcloud issues“.

Human Analysis

SonarCloud is just a static analysis tool, it can only spot problems and suggest pre-defined improvements. These improvements are not always in the best interest of the code base. You have to objectively ask yourself “Does the change I’m making make the code better or worse?” Before you can do that, you need to understand why Sonar thinks there is a problem with the code. Then you can decide if Sonar is right or not. Some of the rules are not applicable everywhere.

I have seen a number of changes where the code is actually made worse by the changes to satisfy Sonar.

Example 1: 7 Argument Limit

A constructor or method that contains loads of parameters/arguments can be a sign of bad design, and maybe some of the parameters can be grouped together inside an object. Once you reach 8 arguments, Sonar will flag this. A simple fix is just to create a class and throw in a couple of parameters in there. It then satisfies the rule but it doesn’t make logical sense unless the parameters are in fact related. Adding a new class when it is single use could just make the codebase more cluttered and seemingly more complicated. I would rather see a class with 8 values, than some complicated class with extra types in the way “Just because of Sonar”. Mark Seemann has a good blog post about Curbing code rot with thresholds.

Example 2: TryParse

Another example, I once wrote some code with the following:

var ok = int.TryParse(a, out var x) & int.TryParse(b, out var y);
//something with x and y

Sonar complained about the use of the bitwise & and it was confusing and suggested I use &&. However, if I did that then “y” wouldn’t always be defined because of the short-circuiting and the code wouldn’t compile. I was about to reject the Sonar issue as “Suggests code that doesn’t compile” and just keep my version.

Then I thought, “if Sonar cannot understand my code to make a decent suggestion, maybe other developers can’t either“. I was trying to be too clever with my code.

Instead I changed the code to:

var ok1 = int.TryParse(a, out var x);
var ok2 = int.TryParse(b, out var y);
var ok = ok1 && ok2;
//something with x and y

It wasn’t as terse as my original version, but it was certainly easier to read and understand, and Sonar didn’t have a problem with it any more either.

Example 3: Cyclomatic Complexity

When a method has loads of If statements, this creates loads of permutations that can be executed which means if you are aiming for true 100% test coverage, you have loads of tests to write. It can easily make the code hard to read and understand too. At a certain point, Sonar suggests breaking the methods into smaller methods. I have seen people take this extremely literally and you end up with a design that looks like

  SetPersonalDetailDifferences1(returnList);
  SetPersonalDetailDifferences2(region, returnList);


So there is no logical grouping for what goes in part 1 and part 2, it’s just that enough code gets placed in the first method then everything else in the second. Now the original single method is half the size with no logical reason other than to satisfy the Sonar rule.

Me (rhetorically)
Are these fields logically grouped or is this just to satisfy sonar?

Brijesh
It is just to satisfy sonar

Example 4: Making The Code Worse

Nullable DateTime

DateTime always has to have a value. However, You can declare a nullable DateTime in C# by appending a question mark like DateTime?

The existing code was checking a standard DateTime for null, which can never happen.

if (startDate == null)
   throw new ArgumentNullException("startDate");

The Code Analysis report was correctly flagging this code as completely unnecessary. Instead of removing the code, the developer then changed it to

if ((DateTime?)startDate == null)
    throw new ArgumentNullException("startDate");

The method was still accepting the startDate as a non-nullable DateTime so could never be null. But then it was being cast to a nullable DateTime so the check against null is technically valid.

Me:
Why are you casting to nullable datetime?

Chandeshwar
dateTime is value type , it can never be null value.
if check dateTime == null, it's always return false .

Me:
Yes, that is correct Chandeshwar. That’s why you can delete the code completely. Your code is always false too.

Sometimes, this type of scenario leads to multiple attempts. Either

  1. because they make some changes and they still get the same Sonar error,
  2. they introduce a new problem,
  3. or maybe the reviewer dislikes the changes and wants them to change it back.

So there’s plenty of files where our Change History looks like this

Fix Code Smell Issue <- doing what they think Sonar wanted them to do
Additional Static Analysis Fixes <-they messed up, so tried again
Addressing code review comments <- I pointed out it still wasn’t correct

Example: Magic Numbers

Magic numbers should not be used

A magic number is a number that comes out of nowhere, and is directly used in a statement. Magic numbers are often used, for instance to limit the number of iterations of a loop, to test the value of a property, etc.

Using magic numbers may seem obvious and straightforward when you’re writing a piece of code, but they are much less obvious and straightforward at debugging time

That is why magic numbers must be demystified by first being assigned to clearly named variables before being used.

-1, 0 and 1 are not considered magic numbers.

Vignesh changed code like

dosagesCount == 1

to

dosagesCount == Constants.Single

Me:
Constants are for a different purpose! They are not for replacing all the numbers in the codebase. It's a good idea to give numbers clear names, and if the same value is used multiple times, it means if you need to change the number, it will update in all places.

Vignesh
This is the rule we follow (quotes the magic number description shown above)... and I got comments from my senior level.

WHAT DOES IT SAY, VIGNESH?
“-1, 0 and 1 are not considered magic numbers”

And you have replaced 1 with the word “SINGLE”

Example: Lack of pragmatism

It frustrates me so much that so many developers even don’t agree with Sonar, or don’t understand the change, but still attempt to change it anyway. Their sole goal is to remove the error that they lose sight of the true aim to write great code that works.

Dave
OK, but why did it suggest it, and why does it make the code better? Did you understand the suggestion or just blindly do what it said?

Pavel
I think I understood it. The comment was: "Add the default parameter value defined in the overridden method". Default arguments are determined by the static type of the object. If a default argument is different for a parameter in an overriding method, the value used in the call will be different when calls are made via the base or derived object, which may be contrary to developer expectations.
That's why it suggested it. But in my opinion this change is redundant and doesn't make the code better.

There was some code which had loads of if statements. It was checking the types of “nodes” in a tree structure, and so attempted to cast the type. If successful, it would go into that code block, else it would attempt to cast to a different type.

Even though the developer didn’t need to change this code, he did change it to attempt to resolve Sonar issues with regards to the casting. However, he only updated some lines and not others which meant it was inconsistent so the codebase is more confusing, and still contains Sonar errors. He also took some lines out of the if statements and then performed all the casting at the top of the method. So now it was actually more inefficient because once you have found a match, you do not need to keep attempting to cast.

  {
            var creterionTag = row.Tag as LinkedCriterion;
            var relationshipTag = row.Tag as Relationship;
            var attributeTag = row.Tag as Attribute;
            var resultSetRuleTag = row.Tag as ResultSetRule;
            var conceptOrderingTag = row.Tag as ConceptOrdering;
Me
why change to "as casts" for most, but then not for Table, Concept and SharingDisplayValue?
I'm not even sure if there is an advantage to doing it this way. We now have several variables where only 1 will be set, and the rest are null.
Might be better spending more time refactoring it out to get rid of the branching.
Pattern matching is probably the neatest way for now. https://docs.microsoft.com/en-us/dotnet/csharp/pattern-matching

Kalya
I just simply resolved some existing SonarQube issue which not raised because of our changes. It is a kind of help to resolving existing issues, It is very difficult to resolving all the issues as of now

So he tried to help, was too difficult, so gave up, but still decided to submit the changes for review, despite making the code worse.

Example: Just Suppress it!

Of course, you can still get rid of the error without actually fixing anything. But merely hide it from the managers that are only looking at the figures:

#pragma warning disable S1075 // URIs should not be hardcoded
public const string InfoUrl = " http://med.info";
 #pragma warning restore S1075 // URIs should not be hardcoded

Conclusion

The goal isn’t to make Sonar happy, the goal is to write good clean code, sonar is a guide to help you do that, but it doesn’t guarantee success.

Me

Employer of Choice: Lack of Progress

Employer of Choice Programme

A few years ago, my employer came up with this idea of ā€œEmployer of Choiceā€ which sounds like they wanted to be classed as a ā€œBest Employer to Work Forā€, but it’s more of a self-award. At its core, it’s about listening to employee feedback and implementing it, but I never think there’s that much scope really. Surely no matter what the pay is like, people are always gonna want more. Same with benefits, amount of annual leave etc.

ā€This is a business critical programme with a number of Senior Leadership-sponsored work-streams that are focused on making us an employer of choice; where we attract, recruit and retain the people we need, enabling everyone to be their best self and deliver high performance.ā€

During all the development updates, they were hyping up the changes they made no matter how trivial. There were some beneficial ones which some people really benefited from, such as improving maternity and paternity leave conditions.

“50 colleagues are working behind the scenes in the employer of choice programme”

Comments are useful

I did critique how the survey was a simple scale of 1-5 but comments were optional. Surely the comments are where the key data lies, so they should be mandatory. Despite the managers claiming that they learned this from the first survey, the next survey was completely unchanged. They also said it was difficult to collate the feedback, since they had hundreds of free text entries and had to try to categorise it. Or that was their excuse not to act on it. Yet they still claimed they did implement a lot, and just looking at the feedback was ā€œinsightful and empoweringā€.

“Cynicism around such surveys and their purpose is understandable when there is little to show for the amount of feedback submitted but, as I say below, you can see how powerful the data can be through the series of enhancements driven through the Employer of Choice programme. I also know that it has helped initiate change locally with managers finding the personalised dashboards particularly insightful and empowering.

The more responses and open comments we get, the richer the data set will be. This will ensure that the Employer of Choice programme can continue to focus on the right things and make informed decisions based on your experiences, suggestions and insight, and managers have the data to share with their teams and agree on local actions together.”

Starlight Session

There was a meeting with the pretentious title of ā€œStarlight Sessionā€. This sounded like it was specific to the Development department. Even though they hyped it up, they didn’t disclose what was actually discussed or implemented.

ā€œYesterday I had the privilege of hosting the second in our wider leadership team strategy session. This session brings together the Senior Leadership team in the Development department, and their direct reports to tackle challenges that we face in the department.

Whenever I work with this team I’m reminded of the depth of expertise and commitment we have in our business. I feel upbeat and positive about the things we’re doing and have achieved together.ā€

Wizard of Oz style

One of the managers leading the Employer of Choice changes was talking about making the Benefits Portal more helpful: “we trialled the use of chat bots….however that process was very manually intensive”

How can the process be manually intensive? was it the classic “Wizard of Oz style” where it was actually a person pretending to be a computer!?

Employee Reps update

We were supposed to give suggestions to our Employer representatives who then met with the managers to discuss the feasibility of our requests. We haven’t heard from them in months despite them promising to provide monthly updates.

So over the last 3 months, here are their achievements:

(On the subject of Optician Vouchers, they gave this incoherent rambling)

ā€œWe contacted the voucher provider who confirmed they will accept vouchers that are from opticians not listed on their portal if they accept them. We also changed the wording on the portal to advise staff should contact their opticians to see if they will accept the voucher.ā€

is the process that the voucher provider gives you a voucher. You give it to the optician. They then claim the money from the voucher provider. So we needed to know if the voucher provider would accept the vouchers from unspecified sources. But then surely the optician needs to contact the voucher provider if there isn’t a pre-existing agreement between them, otherwise the optician will just reject it. so I think what that post is saying, is that you should just ask the optician if you can use it, which is pretty standard really.

ā€œIncreased visibility of employee repsā€ – they attended a company event…which was mandatory. And they didn’t present anything, they just attended. We are being completely fobbed off here.

One employee rep won a raffle at the company event.

We are really clutching at straws here. Literally nothing has happened over this time.

Old Employee Forum

I found some notes on when we had a suggestion box, and occasionally got updates on these. Someone suggested we have newspapers in the canteen “preferably a range of broadsheets”. This got declined

  1. due to the high turnover of staff (not staying in the canteen for extended periods of time)
  2. additional weekly costĀ 
  3. WIFI available to use smart phones

Surely the amount of staff that go in means the newspaper is a good idea. Why would they reject it based on cost. You’d only really need about 5 newspapers which is a trivial amount of money considering how wasteful they are in other areas. It just shows you that there isn’t much intent to listen to staff.

Team of the Year

When it came to our annual awards ceremony, the Employer of Choice team won one of the awards. That seems a little biassed to me. The directors created this team, and it’s the directors that are dishing out the awards. There could have been a chance that they deemed it a failure, but it seemed predictable that they would win: who made the biggest impact? Well, maybe the team we designed to create the biggest impact. 

Apart from the fact they didn’t really make an impact, like I discussed.

To Dos

When writing code, some developers like to write notes to themselves in the actual code using ā€œcommentsā€. A special type of comment is the ā€œTo do ā€ which simply starts with the prefix ā€œTODOā€. Once the feature is complete, you should have no TODOs left in the code.

In a project, sometimes you may split up the requirements in a way that you may need to leave the TODOs in for now to indicate this will be covered in the upcoming weeks, and maybe another team member will write the code there; removing the TODO when complete.

When a feature is released, you should definitely have no TODOs there. For some reason, this wasn’t the case a few decades ago. There’s still loads of TODOs in our legacy codebase. It seems a common occurrence though when you read what other codebases are like.

/// <summary> 
/// TODO: Whoever wrote this please document 
/// </summary> 

What use is a “TODO: Whoever wrote this” style comment?Ā 

Surely it’s likely that the original developer won’t update it, and surely it isn’t hard to find out who wrote it if you just check source control. Now the TODO will stay around until someone is bothered enough to delete it, and they probably won’t ā€œdocumentā€.

“I’ll do it later” is one of these lies you tell yourself. Have you ever seen a kanban card saying “review TODOs”? Nope. It never gets fixed.

Vincent Deniel

Cory House

Cory House’s advice is the following:

// TODO: Thing we’ll never get around to doing. If it’s worth doing, do it now or open a ticket. Why a ticket?

  1. TODO comments are often forgotten.
  2. A ticket makes the issue more visible.
  3. Tickets can be tagged, categorised, sized, and prioritised.
  4. TODO comments create a second system for tracking work. Now we have no easy way to see our backlog in one spot.

Sonarcloud 

The static analysis tool Sonarcloud flags up TODO comments because it obviously means something is missing and needs to be addressed. However, some developers have the mindset to remove Sonar errors by doing the minimum to make the error go away. So to fix a TODO, it’s easier to remove the comment rather than fixing the actual issue (if there even is an issue to fix). 

However one developer just removed the word TODO so then Sonar would no longer flag it, but the comment still remains, and nothing is actually fixed.

Me
why delete part of the comment? delete it all if it doesn't add value

Kalya
Yeah, SonarQube identify only TODO text to remove, so that did like. Now I just removed entire comment

Me
Sonar doesn't tell you to remove the TODO text:
"TODO tags are commonly used to mark places where some more code is required, but which the developer wants to implement later.
Sometimes the developer will not have the time or will simply forget to get back to that tag.
This rule is meant to track those tags and to ensure that they do not go unnoticed."

Kalya
Yeah you are correct, But we should not move any incomplete code to master with TODO identification right. Anyway it is there in master, so in this case what shall we do whether can we remove or keep as it is. Kindly suggest. This is also opt for all the incomplete methods as well.

Me
Yeah, new TODOs mean you haven't finished your project.
It's not a big deal to leave the existing TODOs as they are.
If you did want to address them, for each existing TODO, you would really need to think if it is a problem or not. They have probably been in the codebase for years, so most likely are safe to remove the TODOs entirely. If the TODO is in a blank method, then you can just remove the method if you wish. Sometimes the methods may be part of an interface though so would need to remain. But then Sonar will complain if you don't have some kind of comment in it, so maybe just leave those with the TODO comment in, unless you can think of a good code comment to replace it with.
It is possible that some of the TODOs could point out possible bugs or limitations so could be helpful as a standard comment. In that case, you may need to reword them a bit so they make sense .
In general, for any kind of refactoring, you need to consider if it is worth changing. The more changes you make, the more risk: increase testing scope, and more code to review (I think most of my comments have been for Sonar changes). Deleting the TODO comments has no impact on the behaviour.
I tend to make changes just to the methods I am modifying due to my bug fix or enhancement. If the testers are planning on doing a full regression test, then I'll consider making more dramatic changes.

Goodbye Slack

For the last several years, we have used Slack as our primary way of communicating in the Development department. However, company-wide we have Microsoft Office 365 licences, so other departments use Teams. I always thought it was a dumb decision to use Slack due to essentially paying twice for a communication tool. Slack isn’t that expensive on the lower tiers but it adds up when you have a large amount of staff. Plus, due to stricter security policies, we wanted to use single-sign on so had to upgrade to the Business+ licence which didn’t seem to be worth the cost.

As time goes on, we keep “improving security” which I often think is just an excuse to get rid of certain software. How do you really determine which software or companies are secure anyway? They could tell you they use certain security practices or have some accreditation but if your data is exposed in a data breach is another story.

“not sure what you can hack via Slack. Just over reacting like everything these days. 2FA all the things!”

me

On Slack’s Enterprise licence, they boast even more security features and with our new strict security policies, the management decided that we would have to pay significantly more to keep using Slack, or just get rid of it. So they decided to get rid of it.

To be fair, Teams has improved a bit over the years, and although I prefer the way Slack looks, and its excellent emoji support (you can add custom emojis!); I can’t justify the cost.

why is slack not secure as opposed to teams? probably just nonsense. Where does the data go when it is lost? surely doesn’t leak out onto the dark web!

Rob

We somehow had over 900 members according to Slack Analytics but I reckon that was every historic user since we started using it. Scrolling down the list and roughly estimating, we seemed to have around 300 which could reasonably be called ā€œactiveā€. Then looking at the Business+ costing, it should cost $45,000 per year. Enterprise is one of those tiers where it says “contact sales for a quote”. One manager reckoned it would cost $250k a year to use which doesn’t sound right. How can you justify such an expense for a chat application? Even if it did cost that much on paper, surely you can haggle that down significantly. I’m sure Slack won’t want to lose us. Surely charging $60k is good profit for them.

I often think the way companies charge for software licences doesn’t make sense. They often just charge “per user per month” but there will be times where people don’t actively use the licence due to the work they are doing, or maybe have annual leave to take. Then there’s people that join temporarily, then people just naturally join/leave the business over time. So who really tracks what the accurate amount you need to pay. Companies just end up overpaying for licences they don’t need. Slack seem to suggest they charge just for active users. But what happens if you just send a few messages for 1 day in the month; is that an active user for the month? I often think the best approach would be to charge for a certain amount of users, but then give out an extra 25% keys for light usage.

One thing that I found interesting when looking at Slack Analytics is that most people seemed to be sending as little as 20 messages per day. I think that they are either super focussed and just work independently, or they are chilling out. It’s hard to believe that you can work well in a team, or even have a good relationship with them if you only send 20 messages. I find that some people use instant messaging by sending a sentence per message, which will inflate the message count which makes the numbers even more surprising. For example, they could send 4 messages for this interaction:

Hi

Are you free?

I was wondering if you can help me work out this error

I have just got the latest code but am unable to log in

The decision to remove Slack was disappointing for some, but the bizarre thing is that we got told by our manager on the Wednesday, it was formally announced on Thursday, and gone by Friday 4pm. If you were on annual leave that week, you would be confused when you could no longer access Slack on the following Monday. There was some great information that we had on there, and was great to search for common errors and find solutions to them. We didn’t have enough warning to try and extract the information.

“Has the cost of the loss of productivity and collaboration been factored into the decision to remove slack?”

Sad developer

One developer had a crazy idea of  developing our own solution:

“We are a software development company. If we’re that desperate, can’t we write our own messaging system, protected to the security standard we want?”

Ambitious developer

The thing is, we already made a chat application for our users. I never understood why users would want a native chat app when they could use something more widespread. Since we already have a chat app, then it could actually make sense to add more features to it; then use it internally.

Making your own tools isn’t as cheap as you would think. If a developer’s wage is Ā£35k, then paying only 1 developer to develop and maintain it each year is Ā£35k. You may as well just pay for Slack then. But if we are using it and selling it to our users, then it does make more sense.

The weird thing is, for our upcoming software, we originally used Okta for the login functionality but it was decided it was too expensive, so a few developers got together and made their own solution. That seems bonkers to me because that is about security, so surely you should leave it up to the company that specialises In security. So the fact that we do make custom authentication makes the idea of making a chat app even more realistic.

However one of the architects working on this upcoming software ironically replied:

“We need to move away from homegrown solutions, especially based on the presentation happening now from our Head of Security”

Hypocritical software architect

Another architect supported this claim:

“This is about minimising home grown solutions when an off-the-shelf solution would do just as well”

Software Architect

Does that mean he should be bringing Okta back?

Learning at Work Week

Each year we have a ā€œLearning At Work Weekā€ where there’s usually a few optional meetings where people either do presentations about their department, or some ā€œsoft skillsā€ like management advice, or some philosophy like Growth Mindset. Then you are encouraged to use resources like LinkedIn Learning.

I never get involved and just carry on doing my work as usual. This year, I received an email:

Good morning, We are in the process of putting together a social media campaign around Learning at Work Week, and we’ve seen that you’re a regular user of LinkedIn Learning. We wondered if you’d be happy to share your experience and answer the following questions for us to use as a quote in marketing (e.g on the website and on our socials)?

  • What is your name and job title?
  • How often do you use LinkedIn Learning and what do you use it for?
  • Why is it important to you that you get the opportunity to continue learning and development at work?
  • Feel free to add any further details you want to include!

If you are happy to be involved, please can you send me your written answers to the above questions by the end of the week, along with a photograph.

I watched some of the playlists they put together that were on the topic of our company values. I watched the playlists that were compiled by the Development department about Cloud Technologies. Then I watched some about woke issues they also told us to watch, like on Unconscious Bias. Our company is massive and I know there are plenty of people that will want any excuse to chill and not do work, so I was extremely surprised that I was one of the top users. I would say I have rarely used it, and often forget we are even signed up to it. If I am one of the top users, then that means we must waste loads of money on it and should probably just get rid of it.

So my answers would be:

  • Senior Software Development Engineer
  • Never really use it. Just watched a few courses on wokeness for the bants. Then a few on Cloud Technologies and Entity Framework.
  • LinkedIn Learning is not important, and mainly for chillers.

 I asked Bing Copilot to write me a humorous response. This is what it came up with:

Good morning,

Thank you for considering me for your Learning at Work Week campaign. I’m flattered to be mistaken for a LinkedIn Learning aficionado! While I may not be the power user you envisioned, I’m all for learning and development. To be candid, my engagement with LinkedIn Learning has been more of a casual flirtation than a committed relationship. I dip in occasionally, like a cat circling a fishbowl, pondering whether to take the plunge.

As for my job title, let’s go with ‘Aspiring Knowledge Sponge’ – it has a certain ring to it, don’t you think? The truth is, if I’m one of your top users, it might be time to audit those subscription numbers. It seems there’s more ‘linked out’ than ‘LinkedIn’ in our company’s learning stats!

Nevertheless, I believe in the power of continuous learning. It keeps the brain nimble and the skills sharp, like a Swiss Army knife in a world of plastic sporks. And who knows, maybe this email will spark a renewed zeal for e-learning in me. I’ll aim to send you a more detailed response, along with a headshot that hopefully doesn’t break your camera, by the deadline.

Warm regards,

[Your Name]

Aspiring Knowledge Sponge

Minimum requirements when logging defects

A colleague wrote a list of ideas to improve bug reports. The more detail a report has, the more stakeholders are aware of the impact and can triage it accordingly. When it comes to fixing it, software developers can work out the issue more easily…

It is important that all the required information is added to a defect at the time of raising it. This stops the need to refer it back to the original author for more information and therefore saves time.

It should also be noted that we report all defects raised to certain user groups, and it is important that they are able to understand what the issue is and how severe the issue is to them.

In order to meet the minimum requirement all bugs should contain the following:

Title

  • The title must be clear and concise
  • It should describe the problem
  • Avoid using technical jargon

Description

  • The description must clearly outline the problem being experienced
  • It should explain the impact to the customer
  • It should describe a workaround if known.

 Reproduction steps

  • Reproduction steps to recreate the issue
  • These should be clear and easy to follow
  • Include any required environment configuration
  • Include the actual and the expected outcome

 System info

  • Enter the version of software the bug was found in

 Other checks

  • Have the Area and Iteration paths been set correctly?
  • Are relevant examples present with the record? (record number, sample data, screenshots etc.)
  • Have the relevant error details been included? (stack trace, error message, error number etc.)
  • Has the bug been linked to the test case/user story etc?
  • Has this been linked to a relevant user story, requirement, or similar bug report?

 Root Cause Analysis (RCA)

If the defect is found during integration testing, the following must also be completed

  • Stage Detected In
  • Use Release Testing
  • Complete Release Introduced In field