Kavin AI

Problem

A résumé is a static list. Recruiters, founders and other developers want to ask specific questions: what did you actually build, why did you choose that stack, what would you do differently. I wanted a portfolio that answers those questions in my own voice while staying honest about what is verified and what is not.

Approach

Separate the two things a personal AI needs. Facts come from a reviewed knowledge base through retrieval, so they can change without retraining and every claim can be cited. Voice comes from a personality profile plus retrieval of approved examples of my own replies; those examples shape tone only and are never used as facts. I tried LoRA fine-tuning for the voice in two local pilots; neither adapter was adopted, and I stopped that work in October 2026 (details below). The site presents it as the AI version of me, it says so whenever someone asks, and it never makes commitments on my behalf. I used Claude and ChatGPT heavily while building it.

Architecture

1. Browser

   Next.js 16

   Chat UI with sources and follow-up chips

2. POST /api/chatzod validation, rate limits, session cookie

3. Web

   Next.js route handler

   SSE proxy: streams the service's answer back

4. SSE proxy

5. Service

   FastAPI · Python 3.12

   1. safety in
   2. →retrieve
   3. →prompt
   4. →generate
   5. →safety out
   6. →telemetry

   • Knowledge

     PostgreSQL + pgvector

     knowledge_chunks: embedding + tsvector

   • Models · Ollama

     qwen3.5:4b

     chat · qwen3-embedding:0.6b for 1024-d embeddings

The web app never talks to the model directly. It validates the request, applies per-IP and per-session rate limits, and streams the service's Server-Sent Events (meta, token, done, error) back to the browser.

Technical decisions

• Qwen3.5-4B through Ollama. Open weights (Apache-2.0, per the model's Ollama manifest) that serve on a 24 GB laptop. Thinking mode is disabled for latency.
• Hybrid retrieval. Each query runs a pgvector cosine search and a Postgres full-text search in parallel (top 20 each), fused with reciprocal rank fusion, capped at three chunks per document, top six returned with citation numbers.
• Chunking. Documents are split heading-aware into chunks of about 350 tokens with 60 tokens of overlap, each prefixed with its title and section so chunks are self-describing.
• One schema everywhere. The same SQL migrations run on an embedded PostgreSQL 16 locally and are intended for managed Postgres in production, so retrieval code has a single implementation.
• Approval is separate from verification. Every record carries public_ok (did I authorise it) and verification (verified, self-reported, or not yet reviewed). Only approved records are indexed, and self-reported facts are labelled so the model can qualify them.
• Inference stays off the web host. Model serving is a separate service (today Ollama on the development laptop; a GPU host in the deployment plan); the website only proxies streams.

Challenges

• Qwen3.5's hybrid Gated DeltaNet layers had no backward kernel in MLX in October 2026, so local LoRA training was slow and memory-bound: on the 24 GB laptop the pilots could only fit training sequences of about 640 to 760 tokens with 8 adapted layers.
• Relative retrieval scores are not confidence: a question about a non-existent project can still score high on generic terms, so hallucination resistance has to come from the prompt rules and be measured, not assumed.
• Keeping private material out: the résumé text is phone-redacted on ingestion, the output filter removes any unapproved email or phone number, and excluded projects are deleted from content, the index and the training corpus rather than merely hidden.

Results

The source code lives in a private repository.

What works today: the service reports healthy with the chat and embedding models loaded and the approved index built; the chat pipeline runs input checks, hybrid retrieval, prompt assembly, streamed generation and an output filter that strips citations, email addresses and phone numbers that are not approved; the system prompt carries one exact unknown-answer sentence for questions the sources do not cover (a GPA, an internship I never had); and prompt-injection attempts are detected and the prompt rules reinforced, with tests covering both. As of 13 October 2026 the public evaluation set has 239 questions across twelve categories, with a further private set for restricted names. The Python service has over 240 automated tests. Latency and quality numbers are not published yet.

Lessons

Grounding and voice are different problems with different tools. Honesty rules need their own tests, because a fluent model will happily cite sources it never received unless the filter removes them. The fine-tuning pilots added a second lesson: with about two dozen examples, validation loss improved for about one epoch and then the adapter started memorising, and an old answer about the project could come back as if it were current.

Dataset

No public training dataset exists. The two LoRA pilots trained on small private sets of my own replies (21 and 26 training examples), which stay outside the repository; that work has stopped. My original personality examples remain private; approved ones are retrieved as style examples only, never used as facts. The approved public writing corpus consists of my own README prose for the approved projects.

Models and methods

The model that answers in this chat is Qwen3.5-4B (the qwen3.5:4b build served through Ollama), with qwen3-embedding:0.6b producing the 1024-dimensional embeddings used for retrieval. No fine-tuned model is used. The voice comes from a personality profile plus retrieved examples of my own replies (style only, never used as facts). In the two LoRA pilots, adapters were trained locally with MLX, fused into the base and imported into Ollama under separate names for comparison; neither was adopted, and fine-tuning stopped in October 2026.

Training setup

Stopped in October 2026 after two local pilots. The originally planned configuration was a bf16 base, batch size 1, sequence length 1024, gradient checkpointing and LoRA rank 8 to 16 on the last 8 to 16 layers. The pilots that actually ran had to fit the 24 GB laptop: a 4-bit base copy, LoRA rank 8 on all linear projections of the last 8 layers, batch size 1, gradient checkpointing, a maximum sequence of 768 tokens (pilot 1) and 640 tokens (pilot 2), and learning rates of 1e-4 and 5e-5. In both, validation loss improved for one epoch and then rose.

Evaluation

The evaluation runner scores prompt configurations of the same Qwen3.5-4B model that answers in this chat: the bare model, the model with the personality prompt, and the model with the personality prompt plus retrieval (the configuration running here, which also retrieves approved examples of my own replies for style). The two LoRA pilot adapters were compared against the same baseline in October 2026 and were not adopted. Metrics: factual coverage, hallucination resistance, citation correctness, retrieval recall, style similarity rated by me, first-person consistency, AI disclosure, privacy and contact-policy compliance, and latency.

Metrics

No evaluation numbers have been published yet.

Limitations

A 4B model can paraphrase loosely and over-explain. The assistant knows only what the approved documents say, so recent work is missing until I add it. Fine-tuning was tried and stopped, so the voice comes from the personality profile and retrieved examples of my own replies, not from a fine-tuned model.