End-to-end testing is a technique that tests the entire software product from beginning to end to ensure the application flow behaves as expected. It defines the product’s system dependencies and ensures all integrated pieces work together as expected.

The main purpose of End-to-end (E2E) testing is to test from the end user’s experience by simulating the real user scenario and validating the system under test and its components for integration and data integrity.

Software systems nowadays are complex and interconnected with numerous subsystems. If any of the subsystems fails, the whole software system could crash. This is a major risk and can be avoided by end-to-end testing.

Benefits of End-to-End Testing

End-to-end testing has been more reliable and widely adopted because of the following benefits:

Expand test coverage
Ensure the correctness of the application
Reduce time to market
Reduce cost
Detect bugs

Modern software systems allow subsystem interactions through advancements in technology. Whether the subsystem is the same or different from the main system, within or outside the organization, subsystem failures can cause adverse effects throughout the entire system.

System risks can be avoided by performing the following:

Verifying the system flow
Increasing test coverage areas
Detecting issues associated with the subsystem

E2E testing broadly appeals to many groups:

Developers benefit from it because most of the testing is offloaded to others.
Testers find it useful because writing tests that verify real-world behavior help them avoid problems and achieve a greater sense of accomplishment.
Managers like it because user simulation tests let them know how a failing test would impact the user.

End-to-End Testing Lifecycle

End-to-End Testing | Infographic depicting end-to-end testing framework | Katalon

An end-to-end testing lifecycle consists of four components: test planning, test design, test execution, and results analysis.

Test planning: Specifies key tasks, associated schedule, and resources
Test design: Test specifications, test case generation, risk analysis, usage analysis, and scheduling tests
Test execution: Executes test cases and documents testing results
Results analysis: Analyzes test results, evaluate testing, and perform additional testing if necessary

Types of End-to-End Testing: E2E Test Methods

Horizontal E2E test

A commonly used method occurring horizontally across the context of multiple applications and easily takes place in a single ERP (Enterprise Resource Planning) application. Here’s an example. A Web-based application of an e-commerce system includes accounts, product inventory status, and shipping details.

Vertical E2E test

This method refers to testing in layers, meaning that tests happen in sequential, hierarchical order. To ensure quality, each component of a system or product is tested from start to finish. Vertical testing is often used to test critical components of a complex computing system which does not typically involve users or interfaces.

End-to-End Test Automation with Katalon

Katalon Studio is an excellent automation tool that can make your end-to-end tests less complicated. There are three core features in Katalon Studio that you can use to apply E2E testing: recorder, built-in keywords, and custom keywords.

Recorder

Different use cases are often mixed together in different orders and variations. But we can call each grouping of use cases a user journey. Technically, a user journey is a collection of steps in which each step corresponds to a user action. Collectively, they represent a typical user session.

Katalon offers the Recorder feature on Web and Mobile to help you accomplish the task of creating user journeys without any hassle. The Recorder essentially watches and records all your movements on the application so that you can focus on creating the journey itself.

Built-in keywords

A user journey usually consists of hundreds of steps. When a test case – which represents a user journey – fails, it may be easy to pinpoint superficial causes (e.g. a change in a Web element’s property). However, it is harder to diagnose the actual cause solely based on the fact that the test has failed. It could have been due to the current page is not the same page that was recorded because at some points the test case went off the right track.

To ensure that the user journey has to go through certain milestones, test oracles are necessary. Test oracle refers to a mechanism to ensure that the test case is being executed as expected. In Katalon, a set of built-in keywords that implement this concept is provided. You can assert or verify a web element against certain text that you expect it to contain, or its properties against some expected properties and many other types of expectations check.

Custom keywords

Depending on the business logic of your application, there may be behaviors that occur across different pages but differ only in some known characteristics. In such cases, it is a best practice to capture these behaviors in a behavior template that can be filled with specific information when necessary.

In Katalon, a custom keyword can be defined to represent such a behavior template. Once defined, it can be reused in different test cases and even in different test projects. You can even share your custom keyword as a plug-in through Katalon Store so that others can benefit from it.

End-to-End Testing Best Practice: When to Apply

There are many scenarios to apply end-to-end testing. Here’s an example:

Before testing, designers and developers create a list of which UI, functions, or features need to be performed. Data flow should be tracked between systems to find dependencies, flaws, and inaccuracies. Teams also have to prepare the conditions – factors that may change any interface options – before testing.

E2E testing is regularly conducted on finished products and systems, making each review a test of the completed system. A second test will take place if the system does not output what is expected or if a problem is found. In this case, the team will have to record and analyze the data to determine the issue’s origin; then fix and re-test them.

End-to-End Testing Best Practice: Metrics to Use

Some of the many metrics used for E2E testing are:

Test Case Preparation Status: Used to determine the specific position of the test cases that are under preparation, compared to planned test cases.
Test Progress Tracking: Test progress should be tracked on a weekly basis. This step will provide regular details of test completion percentage, such as passed/failed, executed/unexecuted, valid/invalid test cases, and so on.
Defects Status and Details: It gives a weekly percentage of open and closed defects. Also, weekly defect distributions are based on severity and priority.
Environment Availability: The actual number of operational hours and hours scheduled per day for testing.

End-to-End Tests vs. Functional Tests

End-to-end tests are not just several unit tests and functional tests strung together – they are more complex and carry more risks. We’ve listed the main differences between functional and E2E tests to illustrate this further.

Functional Tests	End-to-End Tests
Testing is limited to one single piece of code or application.	Testing crosses multiple applications and user groups.
Ensures the tested software meets acceptance criteria.	Ensures a process continues to work after changes are made.
Tests the way a single user engages with the application.	Tests the way multiple users work across applications.
Validate the result of each test for inputs and outputs.	Validate that each step in the process is completed.