The framework's author spent years mapping 12 coping styles in a 107,000-word manuscript. I'm building the platform that applies that framework to real text: self-hosted, grounded in the source, and measured against expert judgment on every change. The interesting part isn't the model. It's the evaluation harness that says whether a change made the system better, or just made it feel better.
I built the platform end to end: the Docker Compose stack (n8n, FastAPI, Supabase with pgvector, Langfuse, native Ollama), the retrieval architecture over the manuscript, the LoRA fine-tuning pipeline, and the companion app that puts a classification in front of a reader. Before any of that was allowed to matter, I built the evaluation harness. Every prompt and model change gets a precision/recall score against practitioner-diagnosed cases. Nothing ships on impression.
The framework defines 12 coping styles that overlap by design. The same word can point at three different styles depending on what surrounds it, and real statements mix up to seven at once. There is no public dataset for this. Ground truth exists in one place: the author's own diagnoses of real statements.
The text is also sensitive. People describing how they cope is not data to post to an API. Inference, embeddings, the database, and the traces all had to run on a single local machine.
The manuscript is chunked semantically with LlamaIndex, embedded locally at 4,096 dimensions, and stored in pgvector. A FastAPI service does exactly one thing: retrieve. Generation lives in n8n workflows, where a prompt, a model, or a retrieval strategy can be swapped without touching the service. Langfuse traces every call.
That split pays for itself during experiments. Change one variable, run the harness, read the score. The system is built to be measured.
One run, end to end
Ollama runs natively on the host; everything else runs in containers on the same machine. Nothing in this diagram has an internet connection.
Before optimizing anything, I built the measurement: an automated test runner that scores each configuration on precision and recall against practitioner-diagnosed statements, writes a JSON and markdown report, and feeds a running knowledge base of observed facts, confirmed theories, and open hypotheses. Experiments live in dated folders. A conclusion has to cite its runs.
Current best: 6.6 to 6.8 out of 10. That's the honest number. The target is 9.9, and every step toward it is written down as a hypothesis with a predicted score before it's attempted. When the prediction misses, the theory changes, not the report.
The fine-tuning pipeline is built and parked: scripts that extract diagnosed sessions, generate question-answer pairs from the manuscript, and convert both into LoRA training format for LLaMA-Factory. It waits for 200+ examples, because below that the evaluation predicts marginal gains and the effort belongs in data collection.
Next in line: retrieval that pulls the most similar diagnosed examples into each prompt (predicted 8.5–9.0), and a two-tier pass that reads how a statement is told, not only which words it uses. Each step has a number attached before it runs.
It would be easy to demo this system on its best cases and call it done. Clear statements score 10/10 on every model. The harness keeps everyone honest, including me: it shows the 2/10 cases too, and those cases set the roadmap. A system that knows exactly how good it is can get better on purpose. One that doesn't can only get lucky.
The platform is a research and self-reflection tool grounded in the author's framework. It is not a diagnostic instrument, and it doesn't pretend to be one.