Building a RAG Chatbot for Your Laravel Site with pgvector

Most teams adding an AI question-answering feature reach straight for a dedicated vector database running alongside the PostgreSQL they already operate. That means another service to deploy, a sync pipeline to keep documents and vectors consistent, another set of credentials, and another dashboard to watch. For a knowledge base of a few tens of thousands of entries, that is a lot of infrastructure for what amounts to a nearest-neighbor query.

pgvector is a PostgreSQL extension that adds a vector column type and similarity search operators directly to Postgres. Documents and their embeddings live in the same table, inside the same transaction. There is no sync problem because there is nothing to sync. For the workloads most teams actually ship, that is enough.

This article walks through the practical steps of building a RAG chatbot inside an existing Laravel project.

When You Don't Need RAG Yet

Before writing any code, measure your corpus. If everything fits under roughly 200,000 tokens (about 500 pages), you can put the whole knowledge base into the prompt and skip RAG entirely. With prompt caching, that approach is both fast and cheap. RAG earns its complexity only once the corpus outgrows the context window.

1. Setting Up pgvector and Designing the Embedding Table

Enable the extension once per database, then design the chunk table. In Laravel, reach for DB::statement() because the Schema Builder has no notion of the vector type.

// database/migrations/2026_01_01_000000_create_document_chunks_table.php
public function up(): void
{
    DB::statement('CREATE EXTENSION IF NOT EXISTS vector');

    Schema::create('document_chunks', function (Blueprint $table) {
        $table->id();
        $table->foreignId('document_id')->constrained()->cascadeOnDelete();
        $table->integer('chunk_index');
        $table->text('content');
        $table->text('contextualized_content')->nullable();
        $table->string('content_hash', 64)->index();
        $table->jsonb('metadata')->nullable();
        $table->timestamps();
    });

    // 1536 matches text-embedding-3-small. Adjust to your model.
    DB::statement('ALTER TABLE document_chunks ADD COLUMN embedding vector(1536)');
}

A few design decisions pay off later:

• Dimensions are baked into the schema. Switching embedding models means a new column and a migration. Store the model name in metadata so you know which chunks need regenerating.
• content_hash lets the ingest job skip chunks whose text has not changed. This is the single biggest token saver you will add.
• The vector type supports indexes up to 2,000 dimensions. For larger models, use halfvec (up to 4,000 dimensions), which also shrinks your working set.

2. The Ingest Pipeline: Parsing, Chunking, Embedding

Ingest has four stages: load the document, split it into chunks, turn each chunk into a vector, and store it. Split on natural boundaries (headings, paragraphs) at a few hundred tokens per chunk with a little overlap.

The classic failure mode of chunking is lost context. A fragment reading "Revenue grew 3% over the previous quarter" names neither the company nor the period, which makes it hard to retrieve and hard to use. Anthropic's recommended fix is Contextual Retrieval: prepend a sentence or two of chunk-specific context before embedding. Their experiments cut the top-20 retrieval failure rate by 35%, and by 49% when combined with contextual BM25. That 50–100 token context is generated by a cheap model, and with prompt caching it costs roughly one dollar per million document tokens.

// app/Services/Rag/IngestService.php
public function ingest(Document $document): void
{
    $chunks = $this->chunker->split($document->body, maxTokens: 400, overlap: 50);

    foreach ($chunks as $index => $chunk) {
        $hash = hash('sha256', $chunk);

        // Skip unchanged chunks - this is where the token savings are.
        if (DocumentChunk::where('content_hash', $hash)->exists()) {
            continue;
        }

        $context   = $this->contextualizer->describe($document->body, $chunk);
        $forEmbed  = $context . "\n\n" . $chunk;
        $embedding = $this->embedder->embed($forEmbed); // array<float>

        DocumentChunk::create([
            'document_id'            => $document->id,
            'chunk_index'            => $index,
            'content'                => $chunk,
            'contextualized_content' => $forEmbed,
            'content_hash'           => $hash,
            'metadata'               => ['model' => 'text-embedding-3-small'],
            'embedding'              => '[' . implode(',', $embedding) . ']',
        ]);
    }
}

Run this through a queued job rather than an HTTP request. Ingesting hundreds of documents takes minutes to hours, and embedding APIs have rate limits you will need to retry with backoff.

3. Similarity Queries and Choosing an Index

pgvector ships several distance operators. Three matter for text embeddings:

The search query itself is plain SQL:

$vector = '[' . implode(',', $queryEmbedding) . ']';

$sql = <<<'SQL'
SELECT id, document_id, content,
       1 - (embedding <=> ?::vector) AS similarity
FROM document_chunks
WHERE embedding IS NOT NULL
ORDER BY embedding <=> ?::vector
LIMIT 20
SQL;

$rows = DB::select($sql, [$vector, $vector]);

Two details matter: the ORDER BY must be the raw distance operator in ascending order (not an expression like 1 - (...) DESC), and there must be a LIMIT. Without both, the planner will ignore your index.

By default pgvector performs exact search with perfect recall. Approximate indexes only become necessary as the table grows.

Choose HNSW for almost any documentation chatbot: the table keeps growing, and the index can be created before any data exists. Choose IVFFlat when build time and memory are genuine constraints and you can wait for the table to fill up first.

-- HNSW, built concurrently so production writes are not blocked
CREATE INDEX CONCURRENTLY document_chunks_embedding_hnsw
ON document_chunks USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);

-- IVFFlat: start at lists = rows/1000 (up to 1M rows)
-- and probes = sqrt(lists) at query time
CREATE INDEX ON document_chunks USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 100);

One trap catches nearly everyone: with approximate indexes, WHERE filtering is applied after the index scan. If you filter by tenant_id and only 10% of rows match, the default ef_search of 40 leaves about four rows on average. Enable iterative scans (SET hnsw.iterative_scan = strict_order;) or use partial indexes per tenant.

4. Assembling the Prompt and Handling "I Don't Know"

Once chunks come back, they go into the prompt. Two rules separate a chatbot people trust from one that invents answers.

First, enforce a similarity threshold. If the top score falls below your cutoff (start around 0.3–0.4 for cosine, then calibrate on your own data), do not call the LLM at all — return a plain "not covered in the docs" response. This saves tokens and prevents fabrication in the same move.

Second, say so explicitly in the system prompt.

$context = collect($rows)
    ->map(fn ($r, $i) => "[{$i}] {$r->content}")
    ->implode("\n\n");

$system = <<<'PROMPT'
You are a documentation assistant. Answer ONLY from the context below.
Cite the source number [n] for every claim you make.
If the context does not contain the answer, say plainly that the documentation
does not cover it and suggest contacting the support team.
Do not guess and do not use knowledge outside the provided context.
PROMPT;

Anthropic found that passing the top 20 chunks outperformed 10 or 5, though too much context can also distract the model — test it against your own use case. If you need more accuracy, add a reranking stage: pull a wide candidate set from pgvector, then filter it with a reranker model. That combination reduced retrieval failures by 67% in their tests.

For queries containing error codes or exact identifiers, embeddings often miss. Lean on Postgres full-text search and fuse both result sets with Reciprocal Rank Fusion. All of it still lives in one database.

5. Production Checklist

Re-index when content changes. Wire a Laravel model observer so document updates dispatch a re-ingest job. Let content_hash ensure only genuinely changed chunks get re-embedded. HNSW indexes vacuum slowly, so run REINDEX INDEX CONCURRENTLY first.

Cache in layers. Cache query embeddings (many questions repeat verbatim), cache retrieval results, and use prompt caching on the LLM side for your static system prompt.

Cap token spend. Rate limit per user, bound input length, truncate assembled context at a maximum token count, and log per-request token usage to its own table so costs stay auditable.

Calibrate latency expectations. Vector search takes 5–50 ms, the embedding API call 100–300 ms, and LLM generation 500 ms to 3 seconds. Optimizing the search from 10 ms to 2 ms is invisible to users. Stream the response instead.

Monitor recall. Periodically compare approximate results against exact search (SET LOCAL enable_indexscan = off; inside a transaction) to confirm the index is still healthy.

Wrapping Up

Adding RAG to a Laravel project does not require a new architecture. One Postgres extension, one chunk table, one queued ingest job, and one query endpoint are enough for a solid documentation chatbot. Dedicated vector databases start to matter at billions of vectors or extreme write throughput. For everything else, the database you already run today is probably enough.

References

• pgvector — Open-source vector similarity search for Postgres
• You probably don't need a vector database — Encore Blog
• Introducing Contextual Retrieval — Anthropic Engineering