Cuttlefish2: Jira bug similarity search(and RAG)

Rebuilding my original Cuttlefish project with a modern stack

Image generated by ChatGPT

Background and Introduction

Way back in 2021 I built a JIRA bug similarity engine using the word embedding generated (vectors.txt) by AWS BlazingText as part of a side project to classify bugs (see article). This resulted in the article, “Building a Deployable Jira Bug Similarity Engine using Word Embedding and Cosine Similarity” and corresponding Gitlab repo. It was a proof-of-concept that turned bug descriptions into vectors for similarity search using cosine similarity (implemented as a Flask API and Docker container).

Looking back and given the current state of the art, it is obvious it had a lot of limitations:

• Static embeddings that required AWS infrastructure to have it be periodically rebuilt.
• In-memory retrieval that didn’t scale.
• No Retrieval-Augmented Generation (RAG).
• No modern UI for exploration.

Cuttlefish2

Given the vast advances in NLP over the last couple of years, it was time to to rebuild the entire system from scratch with a modern, production-ready stack:

• Embeddings: OpenAI’s models.
• Storage & Search: Qdrant (a production-grade vector DB).
• Backend: FastAPI with RAG endpoints.
• Frontend: Next.js with Tailwind.
• Deployment: Render (API) and Vercel (UI).

All the code for the project can be found here: https://github.com/foohm71/cuttlefish2

Data Used

A description of the bug data used for both Cuttlefish projects can be found here including what was the original source, what steps needed to be done to extract it out for use. In short, we used the:

1. JIRA_OPEN_DATA_ZOOKEEPER.csv in the original Cuttlefish (~ 500 data points)
2. JIRA_OPEN_DATA_LARGESET.csv (~ 42k data points).

Because of the advancements of vector DBs, retrieval can be done in O(log N) time now which makes it much more practical (not to mention easy to index from scratch and add new entries).

In the next sections I will give more details on what were some of the steps I took to vibe code (on Cursor) the following: a) setting up the context of what I was trying to build b) setting up the vector DB c) building the FastAPI (Python) API endpoint d) building the frontend e) deploying both the API and frontend on Render and Vercel respectively f) Vibe checking (learned this at AIMakerspace) the Bug Similarity Engine and RAG g) Lessons learned from this whole exercise and h) One More Thing (channeling Steve Jobs)

Author’s note: due to the need to be concise (and frankly not bore the socks out of you the reader) a lot of the back and forth prompts between myself and Cursor have been omitted. This back-and-forth (honestly) comes with the territory of vibe coding.

Setting up the context of the project

First step: get Cursor to “understand” what I am trying to achieve with the project.

Prompt:

Please review <the Cuttlefish article> and the attached Readme <the Cuttlefish README file> to understand what we are trying to build today

Cursor reviewed it and went ahead to suggest some approaches.

Setting Up the Vector Database

Next step: choosing QDrant for scalable, cloud-based vector search.

Prompt:

I would like to set up a QDrant instance to perform the similarity search using Cosine similarity

Cursor then went ahead to butter me up (“That is an excellent suggestion”) 🤷‍♂ and then listed down a series of steps to set up the instance on http://cloud.qdrant.io

Next I needed to upload the Jira bug data but first, Cursor needed to understand the data:

Prompt:

Review JIRA_OPEN_DATA_LARGESET.csv . I want to have it upsert into qdrant.

Cursor proceeded to review the data and immediately was able to figure out that the title +description fields would be perfect for vector indexing. I proceeded to ask what would happen to the other columns and Cursor replied they would be stored as meta data. 🙌

Prompt:

Write 3 scripts: one to read in and upsert a CSV file in the same format as JIRA_OPEN_DATA_LARGESET.csv , a sanity test script to check that the instance was running and that we could perform a retrieval operation, a script to nuke the instance.

Cursor generated:

• upload_jira_csv_to_qdrant.py for upsert
• sanity-test.py: Connectivity + sample search.
• nuke_qdrant.py: Wipe collections safely.

However once I started upserting the 41k entries it became apparent network issues, QDrant barfing due to too many tokens etc could break the upsert process so the upload script evolved to support batching, retries, and token limits.

Vibe Coding the FastAPI Backend

Next up was the API layer. I asked Cursor for options and it suggested Flask and FastAPI but the latter was the obvious choice for speed and async support.

Prompt:

Build a FastAPI backend with two endpoints (a) /similar: POST, takes query + openai_api_key, returns top 5 matches (b) /rag: POST, same input, retrieves top 5 matches and uses OpenAI’s chat/completions API for a contextual answer.

Cursor went ahead to build it that. You can view the README file in the api folder of the repo for the details of how to run it.

To verify, I asked:

Prompt:

Create a test.sh script to test both endpoints.

Running test.sh made sure both /similar and /rag worked as expected.

To assist in the next phase I got Cursor to add:

• CORS middleware support for frontend integration.
• Logging for debugging.

Vibe Building the Next.js Frontend

For the UI, I wanted to build a Next.js app to play around with the API.

Prompt:

I want to build a NextJS web app to access the FastAPI API and display the results. It should have the following features: 1) OpenAI API key input (and persist in localStorage). 2) An inputfield for the query. “Similarity” and “RAG” buttons to call the API endpoints similar and rag respectively. 3) display the results as tables with the following fields: key, title, description.

The inital UI looked like crap and I discovered that Tailwind CSS was not installing correctly. So asking Cursor what the alternatives are I tried Chakra UI there were a lot of issues so I went back to Tailwind and figured out it was a version issue (I had to downgrade it to an older version). No thanks to Cursor! 😠

The final UI:

The README file in the frontend folder has the instructions on how to get it running. It should connect up to the local FastAPI backend by default.

(Oh and Cursor didn’t add that cuttlefish image by default — I had to coax it to get it done. Yup no sense of aesthetics whatsover 😞. )

Deployment

I had initially tried to deploy both the API and Frontend on Vercel but kept hitting issues (according to Cursor Vercel is not very suitable for FastAPI type of microservices — apologies but I forgot to keep a log of the issues I faced so I can’t remember exactly what the issue was) so I asked Cursor for alternatives and it suggested a few including Render which I chose cos it was free 😁

Next was deploying to Vercel which is pretty straightforward and Cursor can guide you on how to deploy on both platforms so I won’t include any information here.

Vibe Checking Cuttlefish2

So I took the app for a few spins to get a sense of how it was doing. In my previous article I used the following query:

Quota is not correctly rehydrated on snapshot reload traverseNode in DataTree will never actually traverse the limit nodes properly.

But that was using the ZOOKEEPER data set so I used a different set of queries:

Bug similarity

The query was

What are the critical Null Pointer Exception (NPE) bugs?

Looks pretty good.

RAG

For this we need a more RAG type of question:

What is the current state of Null Pointer Exception (NPE) bugs?

Also looks legit.

Lessons Learned

From this little adventure 🙄 several lessons were gleaned

• Automated test scripts help a ton.
  test.sh and test-render.sh let me validate quickly after changes.
• Strict TypeScript matters.
  Vercel’s Next.js enforces strict linting. any types broke builds. (this was why I decided to try Chakra UI)
• CORS and environment variables need care.
  Hardcoded URLs and missing headers caused repeated issues.
• Framework version mismatches are sneaky.
  Chakra UI and Next.js had breaking changes that required rollbacks.
• Vector DBs make scaling real.
  Moving from in-memory to Qdrant unlocked the ability to index tens of thousands of bugs entries.
• Prompt-driven development is iterative.
  This project was built through cycles of prompt → generate → test → refine.
• Deployments have quirks.
  Render and Vercel each have their own requirements for start commands and environment configs.

One More Thing …

So this is final nugget I wanted to share. To create this article I first used this prompt to create the Medium.md file that you can find in the repo.

Prompt:

ok create a Markdown file called Medium.md. Use the main README.md file as reference material for the Introduction but elaborate on the context more from @https://medium.com/data-science/building-a-deployable-jira-bug-similarity-engine-using-word-embedding-and-cosine-similarity-1c78eeb23a8d . I want the following sections in the article:

(a) Background — this will talk about what the first Cuttlefish project was about and some key ideas and what Cuttlefish2 is about (this can be found in large part in the main README of this workspace)

(b) Setting up the Vector database — talk about some of the steps to create a simple instance on qdrant, vibe coding (ie. prompt and generation) the scripts to upload the data from JIRA_OPEN_DATA_LARGESET.csv, sanity test and nuke scripts

(c ) vibe coding the API using FastAPI including the test.sh script to validate that the 2 endpoints (/similar and /rag) are working

(d) vibe coding the Frontend using NextJS and some of the challenges faced and how they were addressed

(e) Deployment — API on Render and Frontend on Vercel

(f) Vibe Checking (Trials) — leave this section blank. I will fill it up later.

(g) Lessons learned — review the problems faced from the log of this session to extract out the key lessons for this section

Next I went on ChatGPT and used this file as input to refine this article and generate the cuttlefish image. I wish there was better integration between ChatGPT and Medium as it’s really annoying having to copy/paste and deal with the formatting issues but I’ll live.

Happy Vibing!

Author’s Note: I write about hands-on experimentation with AI, testing, and engineering systems from the perspective of someone who’s built and scaled them in production. If you’re an engineering leader dealing with similar challenges at scale, I occasionally work with teams in an advisory or fractional capacity. More here.