What Is Synthetic Monitoring? Definition, Guide, and Best Practices

Modern applications are expected to be up 24/7, and that puts certain pressure on the performance testing team. One of their goals is to continuously monitor real users’ interactions with the site and develop a performance benchmark based on that information. 

However, there’s a catch: real user monitoring doesn't really allow you to observe the system's behavior in extreme scenarios (such as sudden spike in traffic or abnormal user behaviors). After all, those are extreme scenarios, and they don’t happen that often. That’s why synthetic monitoring is created to fill in the gap that real user monitoring can’t provide. 

In this article, we will dive deep into the concept of synthetic monitoring, present the key components of this approach, as well as best practices you should follow to better implement it. You can even jump straight to our comprehensive guide on how to do synthetic monitoring.

What Is Synthetic Monitoring?

Synthetic monitoring is a process where testers create automated scripts simulating real users’ interactions with the application or website to proactively detect performance issues and assess overall system health. These automated scripts are called synthetic transactions, and they help testers gauge the system’s performance when real user data is not yet available, or too challenging to capture.

Benefits of Synthetic Monitoring

Real user data doesn't always tell us the full picture for the software’s performance. It is admittedly great to have data captured directly from real users to provide actual user experience metrics, but what if the feature is still under development and not yet released for real users? How can you proactively capture the data to build a performance benchmark for it?

Even if the feature is already released, user activity tends to be quite limited in the early stage. Performance testing teams have no choice but to passively wait until enough data has been gathered so their test results can be statistically significant.

Similarly, to ensure the highest level of coverage, testers also need to look at more extreme cases, such as sudden traffic spikes or server outages, and you don’t usually have data for such outlier scenarios. To know how the system behaves under such pressures, testers can create synthetic transactions to mimic large volume users and send them to the website and document the results.

When it comes to dependencies, the need for synthetic monitoring becomes even more pronounced. There are so many variables at play that can mess with the final test result. Having a controlled environment exclusively built for the testing of that specific feature is a must if you want the test results to be more reliable. It also reduces the risk of unintentionally breaking other features. 

In short, you need synthetic monitoring for two primary reasons:

Real user data is not yet available, or difficult to acquire.

Testers want to have greater control and less risk over the testing process.

There are so many other benefits of synthetic testing, not because of its synthetic nature, but rather because of its automated nature:

Scheduled testing: You can schedule your synthetic transactions to run at specific times, preferably during peak usage periods, maintenance windows, or any time periods suitable for your test strategy.

Improved analytics: Historical data generated by synthetic monitoring allows you to dive deep into the testing effort and uncover underlying patterns which you can turn into valuable insights for future decision-making.

Customization: Synthetic monitoring can be tailored to your specific testing requirements, allowing you to send traffic on specific journeys through the website to address your unique testing needs.

Synthetic Monitoring vs. Real User Monitoring

Have a look at the table below to better understand the differences between synthetic monitoring and real user monitoring:

Examples of Synthetic Monitoring

Have a look at some examples for synthetic monitoring below. You should notice a pattern: all of them have a fairly specific set of requirements that would be quite challenging to capture if you go with real user monitoring:

Regularly test the load time of your website's homepage from different geographic locations.

Simulate user searches for products on your e-commerce site to confirm that search results load quickly and accurately.

Automate the registration process by creating test accounts to verify that users can successfully register and log in.

Test the submission of forms on your website to ensure that data is processed correctly and validation messages appear when necessary.

Simulate the entire e-commerce checkout process from adding items to the cart, selecting shipping options, to completing a purchase.

Create synthetic tests that send HTTP requests to specific API endpoints to measure response times and identify potential issues.

Automate user logins, providing credentials and checking that the authentication process is secure and efficient.

Automate the launch of your mobile app and simulate various user interactions like screen transitions, selections, and form inputs to ensure a smooth user experience.

Generate specific error conditions to verify that error pages are correctly displayed and provide useful information to users.

Monitor images/videos to ensure they are available and load quickly for users.

A Comprehensive Guide to Do Synthetic Monitoring

Before you start, make sure you choose the right approach:

Write a test script with an automation testing framework (Selenium is a good example. You also need to install Node.js and WebDriver for the specific browser you want to run the test script, such as ChromeDriver).

Write test scripts with the help of an automation testing tool. Installing such a tool is more straightforward, and you can choose from the list of the top automation testing tools on the market here. 

Each tool comes with their own advantages that you should take into consideration. Writing your own test scripts grants a great degree of customization, but the downside is that you have to update all of them when updates are rolled out. Automation testing tools, on the other hand, often come with robust features that support you throughout the entire testing life cycle, but you need some initial investment to get access to the tool.

CTA: Calculate the ROI of an automation testing platform for your team

Synthetic Monitoring With Selenium

Scenario: In this example, we'll create a Selenium test script in JavaScript to automate the process of checking a website's homepage for its availability and response time.

Dependencies:

Node.js: Download here

Selenium WebDriver: You'll need the Selenium WebDriver library for Node.js, which you can install using npm (Node Package Manager). Open your terminal or command prompt and run the following command:

npm install selenium-webdriver

ChromeDriver: Also run the following command in your terminal:

npm install chromedriver

Test steps:

Open a web browser.

Navigate to the website's homepage.

Measure the response time of the homepage.

Verify that the page loads successfully (HTTP status code 200).

Close the browser.

Test script:

The following script performs the test steps we mentioned above. Make sure to change the URL from https://www.example.com to the website you want to test.

const { Builder, By, Key, until } = require('selenium-webdriver');

const assert = require('assert');

(async function () {

  // Set up the Selenium WebDriver

  const driver = await new Builder().forBrowser('chrome').build();

  try {

    // Navigate to the website's homepage

    await driver.get('https://www.example.com'); // Replace with the URL you want to monitor

    // Measure the response time

    const startTime = Date.now();

    // Wait for the page to load (e.g., by checking for an element on the page)

    await driver.wait(until.elementLocated(By.id('someElement')), 10000); // Adjust the element and timeout as needed

    const endTime = Date.now();

    const responseTime = endTime - startTime;

    // Verify the page loads successfully (HTTP status code 200)

    const statusCode = await driver.executeScript('return window.performance.timing.responseStart');

    assert.equal(statusCode, 200, 'Expected HTTP status code 200');

    // Log the response time

    console.log(`Response time: ${responseTime} ms`);

  } catch (error) {

    console.error(`Test failed: ${error.message}`);

  } finally {

    // Close the browser

    await driver.quit();

  }

})();

Make sure that you understand the pros and cons of synthetic monitoring with Selenium:

Synthetic Monitoring With Automation Testing Tools

For automation testing tools, the process of synthetic monitoring is much more straightforward. You don’t have to script much thanks to all of the low-code test authoring features (such as Record-and-Playback, which essentially records your on-screen activities and turns that sequence into a code snippet that you can freely execute on environments of your choice. 

Automation testing platforms take this to an even higher level when centralizing all stages of the testing life cycle, so that you can plan your synthetic monitoring, write scripts based on that plan, schedule them to be executed locally or on-cloud for web, desktop, mobile, and even API, then generate detailed reports to help you understand your testing effectiveness, all in one place.

Here’s how you can do synthetic monitoring with Katalon Platform:

Create a new test case or test suite in Katalon Studio.

Use the built-in WebUI automation capabilities to create scripts that simulate user interactions with your web application.

Set up the tests to run at scheduled intervals, such as every hour or every day.

Use Katalon Analytics to view the results of your tests and identify performance issues.

Integrate Katalon Studio with APM vendors.

Step 1: Sign up and download Katalon Studio. This will primarily be where you create your test cases.

[CTA]

Step 2:  Once you have downloaded Katalon Studio, launch it. You should see the Katalon Studio interface as below. You can then click the “Create new Test Case” button.

Let’s call this test case “synthetic_monitoring”. After you name it and write some description, click OK.

You are now ready to create your test case. Katalon offers a wide variety of keywords (which are essentially automation code snippets for specific actions) that you can use to craft a full on test case. Here’s a preview of our keyword list—quite plenty for all of your synthetic monitoring needs. You can even create your own custom keywords or leverage the Record-and-Playback feature.

But the great thing is not just about quickly creating test cases for web (and also desktop, mobile, or API), without having to write any code, but also about scheduling, executing, and viewing reports for them in one place. Here you can see how to schedule a specific test case and test suite to run on specific elements, at what time, and even what intervals.

After that, you can check the results in Katalon Analytics, with detailed information on passed/failed tests, before diving deep into the granularity. 

[CTA]

Read More: Enabling Synthetic Monitoring with Katalon

Challenges of Synthetic Monitoring

Limited visibility: Be aware of your assumptions for user behavior. Synthetic monitoring is usually based on speculations on what your users do, especially in earlier stages of testing where usage data is not yet available. Can you be sure that they are representative enough for real-world usage? Is it too simplistic? Also, you should note that user behavior is not static: it evolves over time. That is why you need to complement synthetic monitoring with some real-user monitoring to add some realism.

Adjustment for growth: As your website grows over time, it is essential to also adjust your performance benchmark accordingly. What was seen as “traffic spikes” in your outdated test script can actually become “normal traffic” after a long enough period of time. If the assertions in your test scripts are not frequently updated, you will see false positives being reported. These script maintenance tasks can occur alongside infrastructure upgrades.

Scalability: As your organization grows, your synthetic monitoring infrastructure also needs to evolve accordingly. This introduces additional costs and technical challenges.