Retrieval-augmented generation (RAG) has demonstrated improved factual accuracy overpurely parametric large language models (LLMs) in general knowledge-intensive tasks,yet its application to domain-specific administrative guidance, a setting characterisedby precise legal terminology, high error consequences, and structured form requirements,remains empirically understudied. This thesis addresses that gap through the design,implementation, and controlled evaluation of a RAG-based assistant for Swedish estateinventory (bouppteckning) form-filling guidance.

The system combines Cohere’s multilingual embedding model with a Pinecone vectordatabaseandtheLlama3.370Blanguagemodel, groundingresponsesinofficialSkatteverketdocumentation. An evaluation dataset of 21 questions across three categories; terminology,procedural, and edge cases was constructed from authoritative source material. Twoconditions were compared: a RAG condition where retrieved document passages wereinjected into the generation context, and a no-RAG baseline relying solely on parametricknowledge.

Results from the primary evaluation (Llama 3.3 70B, 500-token chunks, top-k = 4) indicatethat RAG improves mean ROUGE-L from 0.3299 to 0.6058, a relative gain of 83.5%, withthe largest improvement on terminology questions (+141%). Source-faithfulness, assessedvia an LLM-as-judge procedure, improved from 9.5% to 42.9%. A separate retrievalconfiguration ablation (Llama 3.1 8B, local cosine similarity) across nine combinationsof chunk size and top-k parameter identified a Pareto-optimal configuration at 150-tokenchunks with top-k = 2, achieving ROUGE-L 0.6024 at a mean latency of 4,886 milliseconds;these two scores are from different experiments and should not be compared directly. Erroranalysis reveals that the principal failure modes are knowledge base coverage gaps on legaledge cases and judge model self-evaluation bias.

The contribution is threefold: an implemented and evaluated RAG prototype for adminis-trative form guidance; empirical evidence quantifying retrieval augmentation benefits andretrieval configuration trade-offs in a specialised Swedish-language domain; and a reusableevaluation methodology applicable to analogous administrative guidance tasks.