What I look for in a Test Engineer
I conceived of this little article because one of my LinkedIn professional contacts had posed a question about whether, as hiring managers of Quality Engineering or Quality Assurance teams, we look for certifications in the area of software quality. I said “no” and went on to list the qualities I do look for.
But before I start rambling about what I do look for …
I look for Test Engineers, not Testers
Why?
I firmly believe that in order to understand how to test a piece of software well you need to know how it is built and how each component fails or is likely to fail. In most modern software systems, the test surface area is huge. How does one look at the entire set of requirements, the design of the software and figure out what are the likely areas it will fail? That takes an engineer’s mindset.
Also Engineering is about problem solving . A test engineer has a ton of problems to solve all the time, from figuring out which aspects to test more, setting up the rig to test, designing a test framework, building it etc. Also it takes an engineering mindset to look at how to get more done with less effort. Quality teams are small if they exist at all (and rightly so — subject of another article).
Lastly, Test Engineers have to be able to run as fast as the development teams they support. Ask any developer what frustrates them the most about their QA team and it’s usually the fact that they have to explain the technical details of the feature implementation ad nauseum or they don’t understand the complexity dev is trying to wrangle or the trade-offs.
(I once worked with a QA lead in a large company and I was helping her set up Selenium to run tests with the Android emulator on her own laptop. She then asked me “what server are you connecting to?”. I was taken aback by that question for obvious reasons. I then proceeded to ask her what she meant by “what server”. She pointed to the configuration of “http://127.0.0.1:8080")
Intellectual Curiosity
A really good test engineer is usually intellectually curious. Ask any seasoned test engineer and she’ll tell you she could be bouncing from a front-end project one week and a database project the next. It would be challenging to cope without an innate intellectual curiosity. Also in the career of a test engineer you will move from domain to domain with a high frequency. For myself I’ve cycled through the domains of Telecom, Internet/Mobile, Digital Agriculture, Fintech, Ecommerce, LegalTech in just 20 years. With each switch, you have to very quickly pick up the salient aspects of the business in order to understand which tests matter more than the others.
Many companies also have a multi-national reach. Having the intellectual curiosity to learn about cultural nuances is critical. (I once worked on a stock app that showed the market ups and downs. When it came time to localize the app for Taiwan, we were surprised that the requirement for the up and down arrows was flipped — red was up and green was down. Turns out red is usually associated with fortune hence the switch)
Incisiveness
A really great test engineer is able to see the needle in the haystack. The term “Shift-Left” testing is well known in the industry but what would be more Shift-Left than a test engineer, during scrum or a briefing on the product requirements be able to raise his hand and say “but that does not make sense because … “? That’s being sharp and incisive.
As mentioned above, the test surface is often large. The ability of the test engineer to zoom in to the most likely aspects of a project to cause issues is absolutely critical — a skill that pays huge dividends and highly valued.
Tenacity
A great test engineer is tenacious. She should not be a leaf blowing in the wind. Your findings will be disputed and dismissed as not important. Be comfortable with standing your ground and using data to prove your point. At the end of the day, product, developers and ops all want the same thing: a great customer experience. We do not always agree what that means and sometimes personal agendas are at play but overall that is the case. Earn their respect.
Another aspect of tenacity is the “can-do” attitude and not giving up too easily. Test teams are small for a good reason: we’re a cost center (never forget that). There is always more work than there are test resources. If as test engineers we do not gradually push the boundary of software quality and give up at the first sign of resistance then quality will never improve.
Technical Competency
As mentioned above, I look for test engineers, not testers. This does not mean having a technical qualification but rather mindset. I’ve worked with great test engineers who do not have an engineering or CS degree but exhibit the problem solving mindset that exceed a lot of those supposedly schooled in the sciences.
Part of being technically competent is the hunger to learn. Ask any working software engineer who’s been in the game for 10 years: they’ll tell you ~ 20% of their free time is spent learning a new technology or reading about and mucking around with the new technology. Every 5 years or so the technology landscape shifts: the shift to PCs from mainframes, the internet, mobile, IoT, crypto now it’s AI. If you do not have the hunger to learn new technology, you’ll be obsolete very soon.
What if you’re not a geek and technology absolutely bores you? Well there’s a minimum bar of technical competence eg. you should be comfortable with software tools and simple modification of test code etc then be really good (and I mean really really good) at the domain (key terminologies, standards, regulations), have a good eye for detail, understand the regulatory compliance processes eg. HIPAA, PCI. Be a subject matter expert.
Spine
As I’ve mentioned previously, your test findings will at times be disputed or dismissed as not being relevant (“no actual user will hit this problem” is a common refrain). Use data, use observability (see [1]) to make your case but have the nerve to stand your ground if you believe and have strong evidence to demonstrate you are right. However also note that at the end of the day, the company you work for is a business and business priorities rule. It is possible to allow for bugs to be fixed post release provided that the business objective is met.
So does a software quality certification help?
While I’d say it isn’t necessary I would say it wouldn’t hurt. It’s just like asking if a CS degree is necessary for you to be a software developer. A lot of really good software developers I know do not have a CS degree. Most of the really good test engineers I know do not have certification. Most do have a background in the sciences or just have the qualities described above. But certification does help in that it covers the basics of what is Software Quality, what the role entails etc which in reality varies from organization to organization and one can learn on the job.
References
[1] What every test engineer needs to know about observability, Heemeng Foo, Dec 2020, Medium, https://medium.com/swlh/what-every-test-engineer-needs-to-know-about-observability-654f757f6622
