A test professional’s take on “if all you have is a hammer, every problem looks like a nail”

What made me write this

One of the things I do at work is attend a weekly engineering post-mortem for high severity production issues that have occurred to lend my experience in looking at whether there were gaps in testing that could have prevented the issue. As any experienced engineer knows, not every production issue can be avoided “if we just had that test at pre-deploy” or is it always cost effective to implement that test (by cost, I include opportunity cost of delaying the release of…

A step-by-step approach

Introduction

As engineering managers, we are always looking for ways to measure team productivity, identify obstacles team members are facing, coaching our teams on sound software engineering principles, developing engineering craftsmanship and keeping an eye on software quality. The larger the team, the bigger the challenge.

In the last decade or so, software for managing development have transitioned away from shrink wrapped or in-house hosted to SaSS (software-as-a-service) based. Tools such as Github, SonarQube, Jira have become managed services where you pay for what you use and you don’t have to incur the cost of installation, setup and…

Authors note: by Engineering here I mean Software Engineering as I am not qualified to speak on other schools of engineering.

Introduction

In a previous article [1] I explained why just catching bugs is not going to lead you to a high quality product. I also explained that what level of quality you need in a product depends on what phase the company is in and that there’s a “Maslow’s hierarchy of needs” when it comes to product quality. More recently, I’ve come across an excellent graphic from ProductStack that describes this somewhat:

Introduction

In a previous article, I detailed how Amazon’s BlazingText could be used to build a Jira bug classification engine. In there, I also referenced the corresponding Gitlab repository. In the README.md there, I very briefly mentioned about word2vec, word embeddings and the vectors.txt file that was generated as output to the word2vec algorithm.

In this article, we are going to be using this same vectors.txt word embedding to build a text similarity engine to find similar bugs in a given corpus of bugs.

What is a word embedding?

I’ll probably not be able to do sufficient justice in explaining what a word embedding is but…

My first encounter with the power of Observability

Way back in 2015 I was involved in the first ever internet live streaming of an NFL game (see [1]). I was in charge of quality for the front end portions ie. the iOS, Android and Web experiences. We were testing almost everyday and on weekends using real live events with significantly lower viewership. This went on for weeks.

On one such test, the engineers discovered a bug in the playback for our Web player. I naturally volunteered my team to support testing for the corrective fix. To my surprise the engineering team said they did not need it as…

Introduction

If you ask any software quality professional what they do, most will tell you (I should know, I’ve been working as a QA/QE/Test manager for the last 12 years) their job function is mostly centered around testing. It comes in various forms: manual testing, automated testing, exploratory testing, functional testing, performance testing, load testing, mobile device testing etc. All these activities have the primary purpose of finding bugs. …

Author’s note: in this article I assume that the reader has a decent understanding of what a modern code review is like and what unit tests are.

Background

Test and Quality professionals often believe that to achieve quality, you need lots and lots of end-to-end tests to validate all the user requirements. I disagree. Sure, we need tests but the bulk of them should be unit tests. Also that code reviews, together with unit tests are the very foundation of software quality.

Incidentally, in the “2019 state of code review” from Smartbear (see [1]), participants were asked what was the #1…

A step-by-step approach

Background

Google Forms is a really useful tool for creating and conducting online surveys. It’s also free and pretty easy to use. However, there are times when you need to perform additional processing to the responses eg. filter out certain responses or aggregate them. After that, you need to visualize the data. For that, Jupyter Notebooks are really useful.

In this article, I cover how to use a Jupyter Notebook, in particular we are going to be using Google’s Colaboratory, which is Google’s implementation of Jupyter Notebook (with many enhancements), to accomplish this.

The files I reference in…