I Vibe Coded a Mock Server and API Tests — Here’s What Happened

Part 1 of building an AI-assisted pipeline for a real-world web app

Author’s note: this article was written with the assistance of ChatGPT. Also, as the field of AI assisted coding and debugging is moving very fast, this article could be outdated in 6–9 months.

Introduction

This article is a follow-up to “I Vibe Coded a Web App — and Was Genuinely Impressed”. In this new series, I’m taking things further: building a full (or almost full) pipeline for a web application using AI tools like Cursor and ChatGPT.

In this first installment, I cover:

• Creating a mock server for the Simple Books API
• Writing automated API sanity tests with Playwright

Code repo: simple-books-store-pipeline

Building the Mock Server

Why This Matters

When building a frontend, you can’t always rely on a live backend. It might be unavailable, buggy, or incomplete/not ready. A mock server provides:

• A reliable stand-in backend for development
• Controlled data for frontend testing
• A way to decouple frontend and backend work
• Even after the backend is developed, the mock server should be what the frontend uses to test against first to validate basic assumptions about the API contract

The input to this could be a OpenAPI spec, Swagger YAML or Postman collection.

Prompt:

Do you know how to build a mock server based on a given API spec?

Cursor recommended Prism by Stoplight. It’s a CLI tool that can generate a mock server from OpenAPI/Swagger specs. I just gave Cursor the go ahead to set up the necessary files for Prism.

The Process

1. Used our Swagger YAML to spin up Prism
2. Found it was returning null responses like:[{"id":0,"name":"string","type":"string","available":true}]
3. Asked: How can I configure Prism to return custom responses?
4. Cursor created a folder for canned responses, but Prism didn’t load them as expected. Checked that with Cursor and it figured out the issue.
5. Solution: Embed the mock responses directly into the Swagger YAML
6. Then it seemed Prism was returning random data (with the correct format). Again I checked with Cursor and turns out I also had to disable Prism’s dynamic mode to get consistent results. Once set, the mock server behaved as expected.

Code + instructions: /api/mock-server

Writing API Sanity Tests

Why This Matters

APIs change, they evolve over time. API providers do periodically release new versions and fix bugs which could break the API contract. As best practice, the API provider should do some sort of versioning (eg. /v1/, /<datestamp>/ or semantic versioning) but sometimes they inadvertently make breaking changes or a breaking change was announced and developers miss the notification. You don’t want to find out the hard way after deploying your stuff. Having a suite of quick, automated tests ensures your app isn’t built on shifting sands.

Prompt:

Do you know how to build API tests using Playwright?

Cursor said yes and started by generating tests in tests/api, using Playwright with TypeScript.

After running npm test :

Results of initial set of tests

What Went Wrong

• Cursor wrote tests to hit the mock server instead of the real API 🤦
• Fixed it by updating the base URL and test logic
• Found some endpoints failed due to missing bearer token (the ones in red in the screenshot above)
• Prompted Cursor to: Copy .env with API_ACCESS_TOKEN to the tests folder and then update test code to load the token from the environment. Somehow Cursor could not find the token so it asked me to paste it on the prompt console (which I did) and it proceeded to update the test code, which then led to …

Security Fix

Cursor embedded the token directly in the code 🤦— a big security no-no.

Prompt:

ok can the API_ACCESS_TOKEN be stored in an env file or somewhere that doesn’t get saved into github as I plan to have the repo made public?

Cursor helped set up .env properly and updated .gitignore to protect the secret.

Expanding Test Coverage

I then took a deeper look into the API tests. First thing I notice is that they are mostly “Happy Path” tests eg. Positive cases ie. you send to the API the correct set of fields and field types and voila! it works! It did include .toBeTruthy() calls but those are the very bare minimum. Those are not the only tests you need. You'll need to know if the API handles incorrect parameters correctly too. So I proceeded to add more tests:

Prompt:

For the ‘should get a specific book’ test, add a check that id is a non-negative integer.

Cursor added:

expect(book.id).toBeGreaterThanOrEqual(0);

Then I asked:

Prompt:

Add a test that sends a “-1” to the “/books/” api endpoint. It should return a 404 response

Cursor generated:

test('should return 404 for non-existent book ID', async ({ request }) => {
  const response = await request.get(`${BASE_URL}/books/-1`);
  expect(response.status()).toBe(404);
});

These were easy to add, but made the tests much more robust.

Final code: /tests

Final set of tests

Final Thoughts

Both tasks were fairly straightforward, but they weren’t flawless. As with the previous web app build, Cursor helped speed things up and was useful for troubleshooting. However:

• You need to check its work eg. the mock server was returning random values instead of the ones configured
• It sometimes made incorrect assumptions (like defaulting to the mock server)
• It included secrets in the source code
• Its default test coverage was minimal

This is where a human — especially a good test engineer — can really add value. Or if there is no budget to hire test engineers, the steps are quite straight forward for any developer to perform and they will get you to some level of quality. Now, some folks may ask, who should be writing the tests: dev or test engineers? I cover this in a previous article here.