The point of this post is to help you with the process of choosing the test automation tools that suit you the best. First, we’ll go through five important aspects you need to evaluate before making any choices. After this, we’ll end with a pros and cons comparison between open source and commercial tools. Before or after reading this post, you might want to check this one out: Five things you should evaluate before proceeding into test automation.

Okay, you’re in a situation where you feel that test automation could be suitable for your application. In many cases, the next step is choosing the tools to be used. Tool selection is one of the most important factors leading to a successful test automation solution. So how do you begin the process?

Checklist before selecting your tools for test automation:

1. Budget

This can already make certain tools unavailable due to their licensing costs. There are many good open source tools out there (more about these later in this post), so if you have a tight budget, think carefully whether you want to invest part of it on the tools or not.

2. Research

Don’t underestimate the importance of the right tools. You don’t want to jump into conclusions with this one. Without proper research of the subject, you can’t know which set of tools will suit your situation. Do research on each possible candidate and gather information. If you’re short on time or prefer to use your hours into something more productive, there are companies that can help you (like VALA for example).

3. The situation as a whole

Understand the situation in hand – make sure that the people in charge of making the decision, really understand the needs and the challenges involved. You need everyone involved to be active, and most importantly, the decision makers need to be thoroughly educated about the various aspects of your test automation tools.

4. Skills of the team

This can suggest whether to go for open source or commercial products. Check your team, what kind of skills and knowledge they have. Do you have skilled programmers in your team? Are they capable of learning new things fast? Do you have the possibility to expand your team with the right people that could handle the challenges of using open source solutions? If you answered “no” into all these questions, you might want to steer clear of open source solutions.

5. POCs

If you’re a quality assurance professional, you understand the importance of testing in software development. It’s the same thing with tool selection, you can’t just go with a particular tool because you heard or think that it’s a good one, you need to test it. Use proper proof of concepts, rather than just picking a tool without evaluating it.

Comparison: open source vs. commercial tools

As you know, there are free and paid solutions available. Next, we’ll help you with the choice between commercial and open source products. Here’s our list of the pros and cons of commercial and open source tools.

Commercial products, PROS and CONS:

PROS:

– May offer a head start and faster progress in the beginning of the project

– Built-in features that usually help

– Good technical support

– Might be easier to use

– Might allow non-technical people to automate simple scenarios easily (although this might not always be a good thing)

CONS:

– Extra costs from licensing

– Implementation of missing features is done by the provider of the solution

– Not that flexible – some solutions only offer support for single programming language

– Might not be well suited for future test automation needs

– No access to source code

Open source products, PROS AND CONS:

PROS:

– Flexible – with enough skills you are in control and can implement whatever features needed, whenever needed

– Some solutions are widely used and communities/support are very good

– No extra costs for licensing

– Access to the source code

– Most of the tools support various programming languages

– Some tools (like Selenium and Appium) are quite mature. Also, both major mobile OS developers provide official testing support libraries/frameworks for application testing. These frameworks include Espresso (for Android) and XCTestFramework (for iOS). Their full power is unlocked by those who are familiar with the codebase under test.

CONS:

– Skills of the team (more technical people required – this might actually be a good thing)

– No support – you are on your own (might be ok with good enough skills)

– Relies heavily on developers support (actually this too can be seen as a positive thing)

– Not so many built-in features available out of the box

– Fixing of bugs or changing the open source solution might be needed

– Extra costs for maintenance (most likely will be less than paying for licenses)

As you can see from the pros and cons list, open source products are usually just as good (or even better) as commercial ones. The biggest con of open source tools is that you most probably need skilled technical people. But if you can tick that box, it’s hard to argue why using commercial products would be better choice in the big picture.

Conclusion

Finding the best tools for your situation might not be as easy as you had thought. As any important decision, it needs some ground work to be successful. Don’t be lazy and jump into conclusions, do your research.

The good thing is, that now you know where to start the process. Use our checklist, and by all means, evaluate other things too that are important in your situation (the checklist in this post covers just the most important things in our opinion). Finally, use the pros and cons list to figure out whether open source or commercial products are more suitable for you. Later on, we will post a blog with the best tools for different situations so stay tuned for that one too.

Main photo: Matt Artz, unsplash.com

• Blog
|
• Software testing
|
• Test automation