Vibe coding web frontend tests — from mocked to actual tests

Part 3 of building an AI-assisted pipeline for a real-world web app

Introduction

This article is Part 3 in a series where I build up a pipeline for the Simple Books Store web app described here.

• In part 1, I described vibe coding a mock server and API tests
• In part 2, I described vibe coding synthetic tests

In this installment I describe building the Front end tests: (a) with a mocked server then (b) transitioning to testing against a pre-prod backend (in this case we will be using the actual backend for simplicity). The reason why I wanted to do it this way is to see how we can use Cursor to transition those tests built when the actual backend is not ready to when it is.

Code repo: simple-books-store-pipeline under the tests/frontendfolder.

Switching to use the mock server

First step was to see if the frontend would grok with the mock-server that was built in my last article. That was done by creating a separate .env file that pointed to the mock server and then running the frontent.

But when I tried to run the frontend built during my original vibe coding article there were issues and so I got Cursor to attempt to fix the problem. It made a bunch of changes that led the frontend to be in a bad state so I got Cursor to rollback all the changes. This it did by checking out the current version of “frontend” on git.

It turned out that between the time I had built the frontend (in the original article) till now there were underlying dependencies used by NextJS (or the other frameworks used with it) that changed. Maybe I could have version pinned some of the dependencies 🤷 . I am a serious noob when it comes to the framework so that could be the MO. Happy to take feedback from any serious NextJS developers!

Building the Mocked Server Frontend Tests

Prompt:

In the folder “tests/frontend/mocked-tests” add the following Playwright UI tests:

a) hit the main page, select “Non-Fiction” and all the books (from the main page) show up

b) Click on the “View Details” button on the first book and it redirects you to the “Book Details” page with the following fields: Name, Author, ISBN, Price and Current Stock and all these fields have values

c) when you click on the “Order this book” button, you redirect to the “Book Order” page with a “Customer Name”, input field and “Order book” button

d) when you click on the “Order book” button with the input field filled up, a pop up that says “Order successful” is displayed

e) On the main page there is a “View Orders” button. When you click on it you will be redirected to a “Orders” page with 3 orders.

f) if you click on the “View Details” button of the top order it will redirect you to the “Order Details” page with the following fields displayed: OrderID, Customer Name, Book Name, Quantity, Order Date and “Delete Order” button.

g) if you click on the “Delete Order” button, you will get a pop up that says “Are you sure you want to delete this order?” and when you click on “OK” you will see a pop up that says “Failed to delete order. Please try again”

Cursor proceeded to build the tests. But then:

Prompt: all the tests are failing. Can you investigate?

Cursor tried by asking to check to see if the NextJS dev server and the mock server was running and even added the config to run both when the tests were to be run. That was not the issue cos I know they are running 🤦.

Prompt: ok let’s check it test by test

Prompt: for the test “should display non-fiction books on main page” the failing step was “await page.waitForSelector(‘.book-card’);”

Cursor reviewed the test code and made some code change suggestions and the test passed.

Prompt: Can you apply the same reasoning to the rest of the tests?

Cursor: Yes, I’ll update all the remaining tests to match the actual implementation. Looking at the code, I’ll update each test to use the correct selectors and structure (this is where I 🤦 )

This time about half the tests passed but I took a deeper look — Cursor was using XPath to identify the UI elements. This creates very flaky tests. A better approach would be to use data-testid (see this).

Prompt: ok I see a problem — you are using XPath to identify the UI elements. That could cause flaky tests as the UI gets changed. Are you familiar with using “data-testid”?

Cursor acknowledged that and proceeded to make changes to the UI markup to include the data-testid . I had to ask Cursor to see if it had updated the other tests as well (🤦 ) to reflect the change and then it also made those changes.

There were still test failures.

Prompt: ok there are test failures. Let’s troubleshoot one by one. Here’s the first: in the “should navigate to book details page” the test failed at await page.click(‘[data-testid=”book-card”] [data-testid=”view-details-link”]’);

Cursor reviewed the code and figured out the problem and fixed it.

Prompt: ok can you review the other tests and see if there are similar issues that need to be fixed?

Cursor went ahead to do that there were still failures on 3 tests (I gave up at this point that Cursor knew what to do) so I went in and got all the information for it.

Prompt: Ok 3 tests are still failing. In “should display orders page with 3 orders” it is failing on this step: await page.click(‘[data-testid=”view-orders-link”]’). In “should navigate to order details page” it is failing at await page.click(‘[data-testid=”view-orders-link”]’);. In “should show confirmation and error popups when deleting order” it is failing at await page.click(‘[data-testid=”view-orders-link”]’);

Cursor went and figure out the issue and fixed them.

Building the end-to-end preprod tests

First thing to do is set up the folder for the tests. You don’t want to mix the mocked tests with those of the actual backend since the behavior would be slightly different.

Prompt: create a subfolder in the tests/frontendcalled “preprod”

Prompt: copy the playwright tests in the “mocked-tests” into the “preprod” folder

Cursor did that and then asked to see if it should make modifications to account for the differences in the envs (not bad!)

Prompt: What are the changes that you propose for the tests in the preprod env?

Cursor:

For the preprod environment, I propose the following changes to the tests:

1. Base URL Change:

• Instead of using the mock server, we should point to the actual preprod API endpoints
• We should use environment variables for the base URL to make it configurable

2. Authentication:

• Add authentication handling since preprod might require it
• Add login/logout test scenarios

3. Data Handling:

• Remove mock data setup since we’ll be using real data
• Add cleanup steps to ensure test data is removed after tests
• Add data verification against the actual API responses

4. Error Handling:

• Add more robust error handling for network issues
• Add retry mechanisms for flaky operations
• Add better logging for debugging

5. Performance Checks:

• Add basic performance assertions (e.g., page load times)
• Add API response time checks

Not bad but I had already planned to use the .env file to switch the backends and the frontend should just work so did not take up what Cursor had recommended.

Prompt: Ok don’t make any changes yet. Let’s try to run the tests as-is and see what happens.

Cursor went ahead to suggest how to run it and we found there was a Playwright config that didn’t allow for the tests in the “preprod” folder to be run. It then proposed a fix and the tests were run.

Turns out that even tho I had explicitly said not to make any changes, Cursor made those changes. Had them reverted.

The tests failed again. Turns out when I rejected the code changes, Cursor also removed all the “data-testid”🤦

I got Cursor to add back the “data-testid”s and reran the tests.

Prompt: Ok there are some failures. Let’s go thru them one by one. In “should display non-fiction books on main page” test, the failure is on this line “await page.selectOption(‘[data-testid=”book-type-select”]’, ‘non-fiction’);”

Somehow the “data-testid”s were not restored. I listed all the missing “data-testid” to Cursor to fix and it went ahead to do so.

Still 3 tests were failing. They were all failing at

// Wait for orders to load
     await page.waitForSelector('[data-testid="order-card"]);

I checked the web site, turns out the view orders page was not functional.

Next I went to rerun the API tests. Turns out the /orders endpoint tests were failing. So I went back to the Postman collection (see the article where I vibe coded the web app) to run those calls. Turns out the API token had expired. I fixed that and then this test started failing: "should display orders page with 3 orders". That would work with a mock server cos we fix the return payload but not with a live backend. I got Cursor to modify it to check for a non-zero list of orders.

Building a user flow test that involves state changes

The whole point of end-to-end (E2E) tests is to test that state changes work. Hence, this was something to really get into.

Prompt:

Ok now create a test with the name “Create an order and delete it”.

Step 1: on the main page, click on “View Details” of the book name “Untamed”.

Step 2: On the Book Details page for “Untamed” click on “Order this book”.

Step 3: On the “Book Order” page add the Customer Name to be “Mustard” and click on “Order book”. Take note of the order ID.

Step 4: Click on “Back to Bookstore” link and then click on the “View Orders” link. The order (with the Order ID) from Step 3 should show up there.

Step 5: Click on the “View Details” of that order. The correct order should show up.

Step 6: Click on the “Delete Order” button and click on “Yes” on the popup asking “Are you sure?”.

Step 7: Go back to the Main page and click on “View orders”, the order from Step 3 should not be there

Cursor went ahead to write the test. Unfortunately it went and used XPath (instead of data-testid )🤦 . I informed Cursor and it also did a facepalm (well sorta 🙂 ).

Once that was done, the test was rerun.

Not bad but there was one failure:

Ok that was my fault for not correctly specifying the step in the prompt.

Next the test needed a timestamp based customer name so that the test order is unique.

Prompt: Ok in the top of the test create a variable for “customerName” to be “customer-<timestamp>” where <timestamp> is YYMMDDhhmmss

Cursor figured out what I really wanted and that was to replace all instances of “Mustard” with that variable.

The final test steps:

This was a great exercise cos there were so many steps and it was fairly involved.

Final thoughts

Overall this was a great exercise as we (a) wrote a bunch of UI tests for the mocked version of the UI and then converted the same tests for the one to hit a live backend and (b) we vibe coded a test that involved state transitions of the app. Some observations:

1. Automated tests are really useful since we had those dependency change issues in the beginning of this exercise. If I had the tests in the beginning I could have just run them to see if everything is hunky-dory but instead I had to manually check that.
2. Cursor needed to be (a) told to use “data-testid” instead of XPath and then later (b) reminded again to use “data-testid” approach again when we went from the mock server based tests to the preprod ones.
3. Quite often when you tell Cursor to “apply the same changes across” it doesn’t work. You may have to resort to telling it to make specific changes.
4. You have to be aware of the changes Cursor is making cos even tho you may have told it not to make certain changes it could have done it especially in Agent mode
5. Cursor can’t help you troubleshoot your tests if they are indeed functioning correctly 😆 like in the case of the API token getting expired and the tests were failing.
6. Cursor is not going to propose using timestamp based transactions (which is a best practice). In this case we used a timestamp appended customer name to distinguish between test runs.

Overall I felt it was still a lot of work for a dev to write these test especially the user flow test. I know most devs hate writing tests so this could be a bit of a PitA. I plan to explore some tools that could possibly make this less painful in the future.