VALA was again strongly represented at Robocon with more than 10 test automation specialists  participating. To give you a glimpse to the conference, in a easy to digest form, we asked three VALA people to write about a speech/topic they thought was most interesting. 

The topics were: 

– The Pros and Cons of Generative AI in Robot Framework (David Fogl)

– RF swarm (Dave Amies)

– How to create a successful Robot Framework automation project (Liviu Avram)

Let’s get to it!

The Pros and Cons of Generative AI in Robot Framework (David Fogl)

With the emerging popularity of ChatGPT, it’s nearly impossible to attend a conference without running into a talk about AI these days. This year’s RoboCon was no exception, with David Fogl giving a presentation about using generative AI in the context of Robot Framework. During his presentation, he introduced ways to utilize generative AI for enhancing test automation developers’ work while also addressing its current limitations.

Test data creation is one area where generative AI can really shine. By varying inputs such as text prompts, images, HTML files, and different types of specifications, AI can create test cases, keywords, locators, and valid and invalid inputs to be used in tests for different kinds of testing scenarios; for example, input validations, database validations, and API testing. 

This will not only increase test automation developers’ productivity but also help testers who may not be that familiar with Robot Framework. AI-generated test cases can act as examples of how to create automated tests when the tester doesn’t know where to start. 

In terms of helping junior developers, ChatGPT can be used to teach Robot Framework and for example answer questions about using Robot Framework libraries. As for documenting your own projects, AI can help by creating a README for your GIT repository.

Another interesting idea for integrating AI with Robot Framework was using it together with Robot Framework’s Listener API. In this scenario, a failing test case would trigger a listener to get the source of the failing page and send it to AI along with details about the failing test to be analyzed. The AI would then make suggestions for fixing the problems, and a maintenance ticket would be created automatically via integration with Jira. Test automation developers would then receive a task for fixing the test with suggested fixes. There was a more hands-on demonstration of this earlier in the RoboCon workshops.

In the presentation AI was described as our new intern with 24/7 availability. While this is true, it is still currently in its early stages and has its cons and limitations. Probably the biggest drawback of using AI is that the output it generates may not necessarily be correct or exact. Therefore, it still needs to be overseen by a person. 

Critical thinking is needed to evaluate whether the generated output is usable or not. Tests generated by AI may also be overly simplistic. Without proper context, it is difficult for AI to understand the system under test and to know what needs to be tested. AI may not, for example, know what happens after a successful login to an application.

As a drawback to using AI, it is also worth noting that some people may not be able to use AI in their work-related tasks due to restrictions. Companies may have concerns about how their inputted data is handled. However, these kinds of restrictions are expected to diminish.

– Jan Nyman, Test automation engineer at VALA

RF swarm (Dave Amies)

RF Swarm is a performance and load testing tool that is part of the Robot Framework ecosystem. Robot Framework is more commonly used in functional testing, which assesses whether a software application performs its intended functions correctly. 

Performance testing evaluates how well the application performs under specific conditions, such as high user load or limited computational resources. RF Swarm enables testers to use Robot Framework test cases for these performance testing purposes.

RF Swarm is written completely in python, so it has the same installation processes and prerequisites as Robot Framework. As a result, the tool can be easily integrated with CI/CD systems.

Tool Structure

RF Swarm is structured into three distinct parts: the Manager, Agents, and Reporter.

• Manager is the central server component of RF Swarm, this is where you plan, execute and monitor your performance test. This component orchestrates the test, managing the distribution of tasks to the agents.
• Agents are the components which actually run the Robot Framework test cases assigned by the Manager. They simulate the actions of users or systems interacting with the application under test, generating the load or performance conditions required for the testing.
• Reporter collects and compiles the results from all agents, after the tests are completed. Reporter has a graphical user interface for customizing and generating different kinds of reports based on the tests that have been run.

Why RF Swarm Was Created

The creation of RF Swarm was motivated by the observation of significant duplication of effort in functional and performance testing within software projects. Traditionally, these types of testing were conducted using different tools and often also different programming languages. 

By developing RF Swarm, the aim was to reduce this duplication by enabling both functional and performance testing to be carried out using the same scripting language and test cases.

Creator’s Goals

The creator, Dave Amies, set clear goals for RF Swarm from the outset:

• To be an open-source tool, ensuring accessibility and community support.
• To allow the use of the same scripting language for both functional tests and performance testing and reducing the learning curve.
• To avoid duplication of code between performance and functional testing and saving time.
• To provide reporting capabilities through integration with Robot Framework’s reporting features. Good reporting tools have been missing from many other performance testing tools.

Current Status of RF Swarm

RF Swarm has largely achieved its initial goals. It is being used fairly extensively, with the Manager component alone having around 800, and Client around 2000 monthly installations. This indicates a strong interest and adoption within the community, suggesting that RF Swarm is fulfilling a need for more integrated and efficient performance testing practices.

– Sami Tonttila, Test automation engineer at VALA

How to create a successful Robot Framework automation project (Liviu Avram)

I attended a talk in Robocon24 called “How to Create a Successful Robot Framework Automation Project,” and it gave me some useful insights. As a test automation engineer, especially working with the Robot Framework (RF) tool, I found the presentation really helpful.

The talk stressed some important principles that are crucial for the success of any RF project. These principles can really improve how efficiently and effectively a new RF project runs.

One big point was the importance of having a reliable, consistent project without any false positives. This means having a well-structured project from the beginning that’s easy for everyone to understand and can grow smoothly as the project moves forward.

Looking back at my past experiences, especially when I had to take over projects started by someone else, I see how sticking to these principles right from the start is key. When you plan well and think ahead, it’s much easier to handle any challenges that come up and keep the project going smoothly.

One thing I took away from the talk is how important it is to prepare thoroughly before starting to write any code. Understanding the documentation, testing before automating anything, and thinking about future changes are all really important steps to take before diving into the project. In my past roles, I’ve seen how skipping these steps can lead to delays and problems down the line.

The talk also stressed the importance of following good practices, like keeping things organized and breaking down the code into smaller, more manageable pieces. Sometimes, when you inherit a project, it can be a mess with tangled-up code and unclear structures. This makes it hard to work efficiently and keep everything running smoothly.

By using practices like writing independent test cases and using keywords for different functions, you can make the project much easier to understand and work with. These practices not only help the project move more smoothly from one phase to the next but also make it easier for the team to adapt to any changes that come up.

In the end, the insights I gained from the presentation on creating successful RF automation projects are really valuable for my professional growth. By following these principles in my work, I’m becoming a better tester and test automation engineer, able to start successful projects and help others do the same.

– Tommy Johansson, Test automation engineer at VALA

• Blog
|
• Robot Framework
|
• Test automation