by Eran Kinsbruner, Chief Software Evangelist

The Complete Guide to

Codeless Test Automation
Today, innovative technologies are fueling digital transformation across all
industries. While these are exciting times, from a DevOps standpoint, these
innovations are also posing new challenges for both dev and test teams.

From a continuous testing (CT) perspective, there is still a huge gap in

implementing test automation that runs in a reliable, stable and efficient
way throughout the DevOps pipeline. From talking to our own customers,
we found that more than 40 percent of failed automation attempts were a
result of scripting issues, ultimately creating barriers for achieving CT.

To address this issue head on, we recently released Perfecto Codeless, an

AI-driven codeless testing solution that allows teams to automate the process
of writing test scripts regardless of skill level. For teams that want to move
faster with automation, our solution provides them with the tools they need
to generate quality test scripts more quickly and to effectively maintain them.

Although codeless test automation has become a top priority for testers, as
well as the developers that are now focusing more on reducing time in test
creation and maintenance to increase software release velocity, many teams
are struggling with how to best implement it for increased output with very
little trial-and-error.

Let’s get a better understanding of what codeless test automation is, what
tests it can be used for, and how it can help your team.

In this eBook, we’ll break down codeless testing into four sections:
Part One: What Is Codeless Test Automation?
Part Two: Selenium vs. Codeless Automation: Which is Right for Me?
Part Three: Why Codeless Automation is Better in the Cloud
Part Four: Five Ways to Get Started with Codeless Selenium

THE COMPLETE GUIDE TO CODELESS TESTING perfecto.io

Part 1
What is Codeless Test Automation?
Codeless test automation is testing that requires zero coding skills throughout Codeless test automation relieves development teams of the extensive
the entire testing pipeline. responsibility of coding – giving them time back to focus on greater
advancements for their business and the industry as a whole. In terms of skills,
Over the last few years, these “codeless” platforms have grown in popularity codeless testers don’t need to know how to code – but they do need a high-level
due to their ability to help your average Joe whip together a website or blog understanding of testing, as well as deep insight into the application they’re
— no coding required. In an effort to take DevOps to the next level, we’ve testing, its business objectives and pain points.
seen an increased adoption and release of codeless test automation solutions.
This is due to a decline in DevOps maturation.

As development teams struggle to find new, innovative ways to speed up each

phase of the SDLC (software delivery lifecycle) in the face of increased demand,
advances in continuous delivery (CD), continuous integration (CI) and CT have
come nearly to a halt. What’s more, with more apps, features, iterations, users,
and ultimately quality assurance activities, to tend to than ever before, teams
are under immense pressure.

THE COMPLETE GUIDE TO CODELESS TESTING perfecto.io

Part 1
WHICH TESTS ARE BEST FOR CODELESS TEST AUTOMATION THE FUTURE OF CODELESS TEST AUTOMATION
When looking to implement a codeless test automation platform, it’s important Codeless test automation is still in its infancy and has a lot of maturing to do.
to keep one thing in mind: it will never completely replace all manual testing. Similar to any emerging technology, there is much to be learned about best
There are always going to be tests that need the “human touch,” or to be practices, challenges and areas of growth. In particular, artificial intelligence
monitored much more closely. Traditionally, this includes tests that have third (AI) is one area where we will see a lot of growth in the next few years.
party dependencies and complex prerequisites from a setup perspective.
While our solution currently leverages machine learning (ML) capabilities to
However, there are several types of tests that can – and should – be operating do things like error classification and self-correct object maintenance issues
codeless. For example, manual tests that are either hard to implement or require within code, allowing scripts to continuously run and fix themselves without
greater in-house skills should be the first that make the cross-over, in addition disrupting operations, in the coming years we will see AI and ML capabilities
to “flaky” tests that continuously lead to inconsistent results (e.g. pass and fail evolve further. It won’t be long before these capabilities will give codeless the
for the same configuration or result in false-negatives). legs it needs to take over all aspects of the testing process.

Typically, a lot of these “flaky” tests are those that look at web pop-ups for Until then, codeless test automation can significantly cut down on the time
unconfigured image elements or the effects of mobile behaviors, such as and resources put into monotonous tests that eat away at developers and testers
certain devices being blocked by certain applications. time. It gives them the opportunity to be much more productive than they are
today – and more invested in the DevOps cycle.
In order to successfully implement codeless test automation solutions into their
pipeline, organizations need to keep in mind that they will have a combination With codeless test automation, testing teams have a renewed opportunity
of code and codeless tests that need to work together. The best approach is to to shine.
exclude the tests that will operate codeless from the actual code itself, apply
the codeless test and then bring it back into the sequence.

THE COMPLETE GUIDE TO CODELESS TESTING perfecto.io

Part 2
Selenium vs. Codeless Automation: Which is Right for Me?
While Selenium automation testing has been the de facto choice for web testing, TEST AUTHORING AND SKILLSET
it is notoriously complex and can only be performed by a developer or automation
engineer — or someone with coding skills. Developers who run Selenium automation testing must have superior coding and
code development skills. Because these tests typically take longer to develop than
Codeless automation, meanwhile, is growing in popularity amongst business testers standard tests — about six hours per test — and are more complex, developers need
with varying levels of coding skills as well as developers who are focusing more on to have a more advanced skills set. Typical test authoring that is performed with
reducing time in test creation and maintenance to increase software release velocity. Selenium automation testing includes:
So which approach is best for your team? • Defining manual test scenarios and user story styles.
• Creating scripts in Java/JS from within IDEs (IntelliJ, Eclipse).
Let’s explore the answer to this question as it relates to test authoring and skillset; • Define Objects using Object Spy tools, DOM viewers, and more.
test execution and maintenance; and tool maturity of both test solutions. • Page-based test step creation.
• Inserting visual validations and assertions.

In contrast, teams of all skill levels can perform codeless automation testing. Test
reusability is simpler with codeless automation testing, meaning team members do
not need to continuously write new test scripts for different scenarios.

What’s more, test scenarios are typically less structured and more exploratory based,
and the time to author is much shorter. Whereas tests can take up to six hours with
Selenium automation testing, with codeless automation testing, it takes about an
hour on average per test.

For QA teams, codeless cross-browser testing takes the complexity out of writing test
scripts, so they can create and execute fully-automated test scenarios without coding.
For developers, codeless automation testing makes test creation and execution
substantially faster — even for the most complicated test scripts — allowing developers
to save time and focus on product development and innovation.

perfecto.io
Part 2
TEST EXECUTION AND MAINTENANCE TOOL MATURITY
When conducting Selenium automation testing, execution is either done locally Because Selenium testing has been in the market for over 10 years, the tools needed
via continuous integration (CI) or through CI in a cloud-based environment. The to carry out such testing are highly mature. Solutions typically include samples and
environment is configured using tools like TestNG Data Provider. Tests are also have a standard set of best practices and a means of documentation.
managed and maintained in a SCM tool such as GIT or Perforce, and any changes
to the tests need to be done manually as the website changes. In contrast, we find that codeless automation testing tools are less mature since
they are quite new in the market. However, these tools are powered by emerging
Similar to Selenium automation testing, codeless automation testing can be technologies such as artificial intelligence and machine learning. It is these
done locally via CI or through a cloud-based solution. However, a stark difference capabilities that make it possible for Perfecto Codeless to automatically self-correct
between Selenium and codeless automation testing is how tests are maintained errors in code and complement testing areas, where Selenium would either require
and managed. longer time to address the error, or cannot deal with it.

A key benefit of codeless automation testing solutions is that the tools support Today, codeless automation testing solutions for web testing are much more mature
self-healing. This means updates and changes to test scripts are made agonistically. and developed when compared with codeless solutions for mobile testing. While we
Scripts can continuously run and fix themselves without disrupting operations. are only seeing basic integrations for mobile testing, we expect this to change in the
coming years as solutions continue to evolve.
Codeless Selenium tools are able to learn the website DOM object tree, meaning
any changes to objects are caught by the tool and can be resolved in the
background without impacting ongoing and scheduled test execution.

THE COMPLETE GUIDE TO CODELESS TESTING perfecto.io

Part 2
BOTTOM LINE
Both solutions offer different advantages to developers. When determining which In addition, organizations should remember that codeless automation testing
solution to use, it’s important to consider the skillset of the team running the tests will never completely replace all manual testing. There are always going to be
and the end goal. Codeless Selenium offers a great opportunity for both business some tests that need the “human touch,” or to be monitored much more closely.
testers and developers to close test automation coverage gaps and complement the It is the combination of code and codeless tests that will ultimately result in teams
Selenium test code that is already in place. achieving success and meeting velocity goals.

To be successful in adopting codeless and embedding it into the DevOps pipeline

testing activities, teams ought to identify the right test flow candidates that can be
done via codeless, ensuring that they are stable and that they do not duplicate
existing Selenium code. They also need to be sure to include them in the existing
CI/CD activities.

THE COMPLETE GUIDE TO CODELESS TESTING perfecto.io

Part 3
Why Codeless Automation is Better in the Cloud
In general, codeless test automation solves a huge test automation creation
and productivity problem for business testers and developers — enabling business
testers to create and execute fully-automated test scenarios without coding;
and freeing developers up to focus on product development and innovation.

With solutions like Perfecto Codeless, machine learning and AI capabilities

eliminate precious time in creating and maintaining scripts — and complement
Selenium test automation written in code.

But what are the differences when it’s done on-premise versus in the cloud?
And which is better for your team?

Let’s dive in.

LEGACY CODELESS COMES UP SHORT

Codeless test automation is typically done with on-premise technologies, meaning
legacy codeless solutions are downloadable software development kit (SDK) solutions.

By nature, this technology creates a huge headache for individuals using the
technology and IT – who manage software versions to ensure the solutions are
properly installed per each type of machine and always up to date.

Test automation by definition is very complex and requires constant maintenance.

It is also being executed continuously. This means that test automation must run
on a robust foundation that can scale as needed, stay up and running, remain highly
secure to meet enterprises’ strict requirements, and offer strong computing power
and storage capabilities.

While you may be more familiar with on-premise codeless test automation, the
technology can’t meet many of the aforementioned requirements.

THE COMPLETE GUIDE TO CODELESS TESTING perfecto.io

Part 3
THE BENEFITS OF CODELESS AUTOMATION IN THE CLOUD COLLABORATION
Continuous testing in DevOps is all about individuals working together within
Meanwhile, codeless test automation in the cloud allows users and enterprises squads and feature teams. Sharing test data and test scenarios is essential to
to meet a number of robust requirements across scalability, speed, collaboration, reduce overhead in recreating tests, as well as having one single pane of glass
quality and security — freeing up team time to focus on product innovation. for everything that is related to quality.

SCALABILIT Y When comparing a collaborative cloud environment to several local desktops,

Managing a Selenium grid locally on a monthly basis when a new browser version it is very clear that the benefit of cross-team collaboration disappears or declines
is being released is close to impossible, expensive and, in most cases, delays test significantly. The cloud empowers multiple users to leverage existing resources,
executions. In addition, scaling a grid with hundreds of permutations locally is flaky, seeing other teams’ executions on-demand in a simplistic and web-based manner.
unreliable and very hard to do compared to a cloud SaaS solution.
QUALITY THROUGHOUT SDLC
Cloud services, on the other hand, enjoy the unlimited scalability of thousands of Continuous testing in the cloud is not just about test creation, it’s also about
on-demand desktop VMs and mobile devices. There is zero setup required for the the entire process of mitigating software risks by performing relevant testing
team executing, and it enables teams to meet strict project timeline requirements and delivering fast feedback to the developers upon any code change.
and testing coverage.
Cloud-based solutions are best positioned to support this objective via the four
SPEED pillars of success in the cloud: creation, execution at scale, a smart lab, and test
Cloud computing solutions enjoy the benefits of strong machines, robust reporting and analysis. When you compare a cloud that supports all four capabilities
infrastructure, and networking that can boost test automation execution time with an on-prem test creation tool that is disconnected from the pipeline, it is clear
and performance significantly. Cloud-based solutions also offer easy access to that the cloud is the best path to success.
platforms in various geographies via globally-distributed data centers.
SECURITY
This is something that contributes to the performance and speed of testing On-prem and cloud solutions also differ in the level of test data security they each
Local solutions fall short in supporting multiple geographies that are, in many provide. Cloud security is far more advanced when compared with on-prem.
cases, a must in web and mobile testing.
A local software installation that runs on a test engineer desktop lacks the necessary
level of test data security given, preventing teams from meeting strict requirements.

THE COMPLETE GUIDE TO CODELESS TESTING perfecto.io

Part 3
KNOW BEFORE YOU GO
Codeless automation testing has clear benefits. To be successful, teams need
to identify the right scenarios that will fit their projects and will be a good return
on investment for the developers.

In addition, organizations need to understand the clear differences between

cloud-based and on-premise from the perspective of creation, maintenance,
documentation, maturity and more.

Ultimately, codeless testing will mature in the future into near 100 percent
self-test automation creation, and while in the cloud, it will boost software
delivery cycles by shrinking the overall testing time per software iteration.

The cloud, in that regard, will serve as a single hub that manages the entire
DevOps activities from development, CI pipeline, through testing and monitoring.

THE COMPLETE GUIDE TO CODELESS TESTING perfecto.io

Part 4
Five Ways to Get Started with Codeless Selenium
If you want to get started with codeless testing, look no further.

Codeless automation testing is perfect for manual tests that are either hard
to implement, require greater in-house skills, or are “flaky” and continuously
lead to inconsistent results. Codeless Selenium is ideal for teams that are
continually running into stability issues within their testing pipeline.

Teams that have a hard time meeting their iteration testing timelines and have
a low percentage of test automation in their pipeline due to limited coding skills
should also explore codeless Selenium.

As more teams look to implement codeless Selenium, let’s explore the benefits
it offers organizations, areas that pose challenges, and our five best practices
for getting started with codeless Selenium.

Keep reading to learn more.

THE ADVANTAGES OF CODELESS SELENIUM THE CHALLENGES TEAMS ENCOUNTERS WITH CODELESS TESTING
Codeless Selenium offers teams a variety of advantages. For example, it empowers Although the benefits of codeless Selenium are numerous, we do find that teams
better collaboration across all testing personas. This includes business testers, struggle to implement this technique. For example, they run up against questions
developers and Software Development Engineers in Test (SDETs). like how to embed codeless Selenium into CI and their overall pipeline, and how to
connect codeless Selenium to the existing software delivery process.
It also increases test automation coverage, which in turn reduces any risks that
could impact quality. Other challenges to overcome include determining what test cases to include in the
codeless testing suites vs. the one to leave running within Selenium, and when to
Codeless Selenium also expedites software release cycles. By reducing flakiness perform refactoring and maintenance for the codeless testing suite.
and allowing teams to automate the majority of manual error-prone test cases,
it stabilizes not only the entire pipeline but also continuous integration (CI) and
overall testing activities.

THE COMPLETE GUIDE TO CODELESS TESTING perfecto.io

Part 4
BEST PRACTICES FOR GETTING STARTED WITH CODELESS SELENIUM
While these challenges can seem overwhelming, teams should not fear codeless
Selenium. To get started, identify the tests you’ll apply it to and the people within
your organization that will be in charge of getting codeless off the ground. This is an
imperative step to ensure the implementation process is successful.

One of the big mistakes we see organizations make with codeless Selenium is that
they go all in from the start – looking to apply it to all test cases throughout their
pipeline. Instead, start small with a quick and solid “win” to prove value.

Starting small also enables teams the opportunity to evaluate from the start the
pieces that can be reused across the codeless test suite. Many steps are repeatable
and can be used as prerequisites for other cases.

These should be identified and made available for other teams to use, ultimately
making suites more stable and speeding up the process of building new cases.
Then you can start embedding it into CI and grow the suite with additional test cases.

Once you have those bases covered, follow our top five tips for embedding
codeless Selenium:
1 Identify manual and/or flaky test steps.
2 Mark test as “codeless” to disconnect them from the code-based
automation suite. This helps to avoid duplications.
3 Record the test in a “smart” manner. This includes:
· Flow-based and exploratory recording.
· Reusable components.
· Robust object locators.
· Tagging for reporting and tracking.
4 Connect codeless to CI and the larger pipeline for continuous testing.
5 Identify maintenance and examination points for both suites.

THE COMPLETE GUIDE TO CODELESS TESTING perfecto.io

Part 4
BOTTOM LINE
Codeless Selenium offers organizations a number of benefits.

However, for those teams that are newer to the concept, they should
approach it with caution. Start small and grow the suite carefully to
ensure value is added to the product. Maintaining the suite should
depend on upcoming test cycles and critical releases.

Similar to traditional automation, you need to ensure you build a solid

foundation so it can grow, scale and become the master suite for future
testing. Teams that take a gradual approach to codeless Selenium will
ultimately benefit from stabilized pipelines, better collaboration between
teams and increased test automation coverage.

NOW WHAT?
Now that you’ve learned all about codeless test automation and what it
can do for you, be sure to check out Perfecto’s own cutting-edge solution,
Perfecto Codeless. It’s next-generation codeless testing that allows you to
create tests up to six times faster.

And be sure to check out our on-demand webinar, How to Embed Codeless
Automation into DevOps to see codeless testing in action.

READY FOR A DEMO? SIGN UP NOW

THE COMPLETE GUIDE TO CODELESS TESTING

ABOUT PERFECTO ABOUT ERAN KINSBRUNER

We enable exceptional digital experiences and help you Eran Kinsbruner is the chief software evangelist at Perfecto, the
strengthen every interaction with a quality-first approach leading cloud-based mobile application testing and automation
for web and native apps through a cloud-based test company. He is also the author of “The Digital Quality Handbook”
environment called the Continuous Quality Lab. The CQ Lab and “Continuous Testing for DevOps Professionals.”
is comprised of real devices and real end-user conditions,
giving you the truest test environment available. You can find Eran on Twitter @ek121268, LinkedIn, and his
professional blog, mobiletestingblog.com. Eran also writes for
More than 1,500 customers, including 50 percent of the the Perfecto Blog and speaks on a regular basis at events such
Fortune 500 companies across banking, insurance, retail, as QUEST, and AppsWorld StarEast and StarWest.
telecommunications, and media rely on Perfecto to deliver
optimal web and mobile app functionality and end-user
experiences, ensuring their brand’s reputation, establishing
loyal customers, and continually attracting new users. For
more information about Perfecto, visit www.perfecto.io, join
our community follow us on Twitter at @PerfectoMobile.