LLM fine-tuning services · CPT · SFT · Datasets · Evaluation

Your data.
Your domain.
Your model.

We fine-tune open-weight language models on your company's data — manuals, tickets, contracts, processes, terminology — and prove the gain on a frozen benchmark before anything ships. Custom LLM training as a service, from dataset to deployed model. Trained in the EU. You get the weights. And when a better open base appears, we re-tune on your data and your eval within days, so the model never goes stale.

20+ open models on Hugging Face Trained in the EU You own the weights · we keep them current
trainer · qwen3.6-35b-a3b · slovenian
$ bash scripts/run_cpt.sh # QLoRA r256 step 2405/2405 loss 1.42 → 1.34 ✓ adapter saved $ bash scripts/run_sft_autoresume.sh # chat + MT · 2 ep $ python eval/eval_llm_4bit.py --model outputs/merged arc_easy 0.71 → 0.78 boolq 0.83 → 0.89 avg (7) 0.623 → 0.654 +3.1 pts · 6/7 tasks won $ bash scripts/merge_lora.sh && quantize Q4_K_M ✓ GGUF ready
held-out perplexity · CPT 6.996 → 3.438
6.996 3.438 0 1.78 B tok
base vs tuned Slovenian-LLM-Eval
arc_easy0.710.78+7
boolq0.830.89+6
hellaswag0.590.62+3
sl→en BLEU30.935.0+4.1
ASR WER46.822.3−52 %
20+Open models published on Hugging Face, each with a card, provenance and numbers
1.78BTokens in our largest continued pre-training run (full-parameter, 4B model)
−52 %Speech-recognition word error after fine-tuning on 962 h of audio
140 kDomain training records engineered, QC-gated and licence-tracked
The gap

General models have never seen your business.

Frontier APIs are excellent at the average case. Your tickets, manuals, contracts and processes are not the average case — and the last 20 % of quality is where the value is.

01 — KNOWLEDGE

It has not read your documents

Your product manuals, support history, procedures and terminology exist only inside your company. No off-the-shelf model has seen them — so it guesses, confidently.

02 — RELIABILITY

Prompts get you to 80 %

Exact output formats, house terminology, tool calls that work every time, answers in your language — the last 20 % is trained in, not prompted in.

03 — SOVEREIGNTY & COST

Per-token APIs don't scale, or comply

For special-category and confidential data, a foreign API is a sub-processor your DPO has to justify. A small model trained on your data runs on your own servers at a flat cost — and the weights are yours.

04 — DEPRECIATION

A model trained once is a model going stale

A new open base ships every quarter. A model tuned once on last year's base loses to next year's untouched one. The asset is not the weights — it is the dataset, the frozen eval and the ability to re-tune in a day.

What matters Public API Prompting + RAG Fine-tuned with us
Data stays in the EU, under your control✕~✓
Your terminology and output formats, every time~~✓
Reliable tool calls into your systems~~✓
Facts that change dailyretrieval belongs here — we combine both✕✓✓
Flat cost at scale, runs offline✕~✓
You own the model✕✕✓
Improves as base models improvewith the retainer — scored on your own eval, not the vendor's✓~✓
What we do

LLM fine-tuning services: from raw data to a model you own.

Six capabilities, one pipeline. Take any of them alone or the whole chain end-to-end. Every one has been exercised on a model you can download.

01

Continued pre-training (CPT)

Teach a base model a whole language or a domain corpus — billions of tokens of cleaned, deduplicated, licence-tracked text.

  • Full-parameter training for dense models; high-rank QLoRA for 30B+ mixture-of-experts
  • Perplexity gates on held-out text after every checkpoint
  • Retention mixes so the model keeps what the base did well
  • Packed sequences, crash-resilient multi-day runs
1.78 B tokens → held-out perplexity −51 % · Sokol 4B
02

Supervised fine-tuning (SFT)

Shape behaviour: chat, translation, tool calling, reasoning, identity — in the model's own template, the one your serving stack renders.

  • Native function calling with multi-turn call → result → answer loops
  • Reasoning distillation from a stronger teacher, judge-filtered
  • Identity, tone and policy tuning
  • Sequential re-tunes that build on earlier adapters
+3.1 pts Slovenian-LLM-Eval · 6/7 tasks won · +4.1 BLEU sl→en
03

Dataset engineering

Training-ready corpora with provenance — not scraped piles. Half of every program is here.

  • One schema, quality scoring, exact + embedding dedup, deterministic splits
  • Licence gating: commercial mix and research mix from the same sources
  • Teacher distillation with judge filtering; synthetic dialogues from verified blocks
  • Domain translation with terminology freezing — codes, names, IDs survive intact
  • Exports for CPT, SFT and preference training (DPO/GRPO) + frozen hold-outs
140,063 QC-gated medical records · 35-source cybersecurity registry · 49,772 distilled traces
04

Training runs as a service

Bring data, get back a model. Or bring nothing and we build the data too.

  • GPU capacity we control — own and vetted EU partner clouds — or your cloud / air-gapped hardware
  • Auto-resume wrappers and checkpoint discipline
  • Merged bf16 weights, LoRA adapters and GGUF quantizations delivered together
  • Model card with full provenance and the numbers behind every claim
20 models and 4 datasets published on Hugging Face
05

Evaluation flywheel

"Better" is measured, not asserted. The benchmark is defined before the data is touched.

  • Frozen, versioned benchmark sets — never edited, only extended
  • Objective track (accuracy, BLEU/chrF, WER/CER) + multi-judge track
  • Regression-flagging scoreboard across every model version
  • Go/no-go on numbers before anything ships
every model in Results was scored base-vs-tuned on the same frozen set
06

Deployment & speech

From a 4B model on a laptop to a 35B mixture-of-experts on a workstation — with vision, tools and voice.

  • GGUF Q4–Q8 for llama.cpp / LM Studio; vLLM and transformers recipes
  • Image input and tool use working in production, not just in the demo
  • Streaming speech recognition and text-to-speech fine-tuning
  • Optional hosting or on-site install that runs unplugged
Slovenian streaming ASR — WER 46.8 % → 22.3 %, runs on CPU
How it works

Five steps, one number at the end.

A first measured model lands within weeks, not quarters. Every phase ends with something you can check.

  1. Scope

    Define the job and the eval

    The users, the task, the frozen eval set. An audit of your data and its licences. A written go/no-go criterion.

  2. Data

    Build the corpus

    Clean, translate, distil and mix. Retention slices protect what the base already does well.

  3. Train

    CPT → SFT → re-tunes

    Each checkpoint gated by a quick eval so bad runs die early and good ones get the budget.

  4. Evaluate

    Base vs tuned vs alternatives

    Head-to-head on the frozen benchmark, including the API or RAG setup you were considering.

  5. Ship

    Weights, GGUF, card, recipe

    Hand-over workshop. Hosting or an on-site install if you want it. No dependency on us remains.

Results

Everything below is public.

Every model and dataset sits on Hugging Face with a card, its provenance and its numbers; research models (Sraka, Vran) are gated, access on request. Slovenian is our reference implementation — the same pipeline runs on your domain.

Qwen3.6-35B-A3B · base vs CPT+SFT (Slovenian)

Slovenian-LLM-Eval · acc_norm · n=100/task · 4-bit eval
TaskBaseTunedΔbase ░ tuned ▓
arc_easy0.710.78+0.07
arc_challenge0.480.51+0.03
hellaswag0.590.62+0.03
piqa0.650.64−0.01
openbookqa0.480.51+0.03
winogrande0.620.63+0.01
boolq0.830.89+0.06
average0.6230.654+0.031
CPTSFTtool-callingvisionGGUF

Sraka 27B — our strongest Slovenian model

A 27B general model for Slovenian: 208 million tokens of Slovenian continued pre-training, then chat, translation and tool-calling fine-tuning. Better than its base on all seven Slovenian benchmark tasks, and much better at knowing when not to call a tool. Takes images, runs locally as GGUF. Experimental release, non-commercial licence, access on request.

0.871Slovenian-LLM-Eval (base 0.844)
7/7tasks won
0.956correct no-call (base 0.847)
CPTSFTtool-callingvisionGGUF

Qwen3.6-35B-A3B Slovenian

A 35B mixture-of-experts (3B active) taught Slovenian through continued pre-training, then chat and translation SFT, then a tool-calling re-tune that kept the language gains. Vision and native function calls included.

+3.1pts acc_norm
6/7tasks won
+4.1BLEU sl→en
full-parameter CPTSFTreasoning distillationvisionon-device

Sokol 4B — on-device assistant

A dense 4B model pre-trained on 1.78 billion Slovenian tokens, then fine-tuned on ~250k examples of chat, translation, tool calling and teacher-distilled reasoning. One repo, full weights plus Q4–Q8 GGUF and a vision projector. Runs on a laptop and also reads images.

−51 %perplexity
+0.047acc_norm avg
6/7tasks won
dataset engineeringdomain MTeval suiteresearch

Vran 27B — biomedical research model

We built the corpus first: 140,063 Slovenian medical instruction records, machine-translated from a provenance-tracked English gold set with a purpose-built medical translation model and a four-layer QC gate, plus a dedicated eval suite. Then we trained the model. Research use.

140 kQC-gated records
27 Bparameters
3datasets released
ASRfull fine-tunestreamingCPU deploy

Slovenian streaming speech recognition

Full fine-tune of NVIDIA's Nemotron 0.6B streaming recogniser on 962 hours of Slovenian speech, scaled up in versions with no catastrophic forgetting. Word-error rate halved on studio speech and cut by two thirds on multi-domain audio. Deployed on CPU as GGUF.

22.3 %WER studio (from 46.8)
23.4 %WER multi-domain (from 67.7)
962 htraining audio
TTSLoRAvoice stack

Slovenian text-to-speech voices

LoRA fine-tunes of two open speech generators on Slovenian studio speech, closing the loop of a fully local speech-in → speech-out assistant with camera vision and tool use — served from one command in the browser.

2open TTS models
100 %runs locally
datasetsCPT corpusSFTbenchmark

Published datasets

A cleaned continued-pre-training corpus, a 120,000-pair bidirectional EN↔SL translation SFT set, the medical instruction corpus and its evaluation suite — each with a card that states sources, licences and how it was built.

Public model or tuned?

A public model is the starting point. Your task is the difference.

Public Slovene bases such as GaMS, LLMs4EU and the CLARIN.SI catalogue are good news for us: a cheaper starting point. On general tasks we do not claim to be better. On your task, with your documents, on your server, the measured difference is the only thing that counts, and that is exactly what the pilot measures.

Same harness, same examples: our domain model vs the public GaMS3-12B-Instruct

Slovenian-LLM-Eval · n=500/task · acc_norm · August 2026
ProtocolVran 27B (ours, biomedical)GaMS3-12B-Instruct (public)Δ
Continuation likelihood (0-shot, log-lik)0.6700.663tie
Question answering through the chat template0.8520.773+7.9 pts

What it means: a model trained for medical tasks loses nothing on general Slovene against the public model (the first row is inside noise) and leads when asked questions, partly through format familiarity. The advantage we sell is not this table but the base-vs-tuned ladder on your set you get at the end of a pilot. Method, code and raw results are public at hf.co/texdata.

Use cases

What companies fine-tune with us: use cases for custom language models.

Typical starting points. Most programs combine two or three.

Customer support & service

Assistants trained on your ticket history and manuals: consistent answers, your tone, escalation when unsure.

Documents & extraction

Contracts, invoices, forms, reports: structured extraction, classification and summaries in your schema, in your language.

Internal knowledge

Procedures, standards, ISO documentation, engineering notes — a colleague that has actually read them, offline.

Agents & automation

Reliable tool calling into your ERP, CRM or ticketing API. Trained on your function signatures, tested on pass rate.

Speech & voice

Call transcription with your vocabulary, voice bots and dictation — recognition and synthesis that run on your own servers.

Language adaptation

Slovenian or any under-served language: continued pre-training and instruction tuning so the model is fluent before it is expert.

Ownership

Your data stays in the EU. The model leaves with you.

We train on GPU capacity we control — our own and vetted partner clouds inside the EU. Either way your data is handled under NDA and never leaves European jurisdiction.

What you receive

  • ModelMerged weights + LoRA adapters
  • DeploymentGGUF / vLLM build + serving recipe
  • DataEvery dataset we built, with provenance
  • EvaluationFrozen benchmark + base-vs-tuned scoreboard
  • DocumentationModel card, data card, training configs
  • Hand-overWorkshop; optional hosting or on-site install

The rules

  • Your data is handled under NDA, on infrastructure we control, inside the EU. GDPR is the default, not an add-on.
  • Air-gapped training for the most sensitive data: we run with the network cable unplugged and hand over the drive.
  • Open-weight bases only — Qwen, Gemma, Llama, Mistral, Nemotron — with the licence of every base and every data source tracked.
  • You receive the weights, the adapters, the datasets, the eval set and the provenance record. No dependency on us remains unless you want one. The one most teams choose to keep is the retrain cadence, not the weights.
  • Everything is versioned — data, configs, checkpoints, scores — so a result can be reproduced a year later.

Where the model runs

Sensitivity picks the rung. Ordinary personal data is fine on an EU cloud under a processing agreement. Special-category data — health, biometric, banking secrecy, classified — usually needs the model on hardware you can walk up to. Everything we ship runs on every rung.

  1. LaptopOn-device

    4B models on a laptop or phone. Nothing leaves the machine. Field work, offline, no infrastructure.

  2. WorkstationSingle GPU

    27–35B models on one workstation GPU. Team of a few dozen, one box, no ops team.

  3. Your rackOn-premises server

    Your data centre, your firewall, vLLM serving recipe included. Standard for hospitals, banks, public sector.

  4. EU cloudHosted by us, in the EU

    Capacity we control, EU-owned providers, no US sub-processors. For teams without hardware who still need residency.

  5. Air-gappedNever networked

    Trained with the cable unplugged, delivered on a drive, run on an isolated network. Defence, critical infrastructure, trade secrets.

The asset

The weights are the output. The pipeline is the asset.

Anyone can copy a weights file. Nobody can copy what produced it. Four things make a fine-tuned model worth more next year than it is today — and none of them is the model.

01 — DATA

Dataset pipeline

Schema, dedup, quality scoring, licence gating, terminology freezing. Built for your data once, reused on every retrain.

02 — EVAL

Frozen evaluation

Versioned benchmark sets and a regression scoreboard. Without them nobody can tell whether a new model is better or just different.

03 — YOUR DATA

Proprietary input

The one ingredient no outsider can reproduce. That is why we tune per customer instead of selling generic vertical models.

04 — CADENCE

Retrain in a day

New base drops, we re-tune on your data and re-score on your eval within days. A model bought once cannot do that.

Engagement

Start small, measure, then commit.

Three ways to work with us. Most teams start with the pilot and end on the retainer. If you do not yet know whether you need training at all, a one-day evaluation on your documents comes before the pilot.

End-to-end

Full program

CPT → SFT → evaluation → deployment for a production model, including the dataset engineering and the serving recipe.

  • Corpus build, translation or distillation as needed
  • Multi-stage training with checkpoint gates
  • GGUF / vLLM delivery, model card, hand-over workshop
Discuss a program
Where programs end up

Flywheel retainer

A new open base every quarter, new data every week. We re-tune, re-score and ship monthly. The scoreboard is the contract. Every version of the weights is yours.

  • Base-model upgrade within days of release, scored on your frozen eval
  • Regression alerts before users notice
  • Cascade option: your small model takes the bulk, a frontier fallback catches the rest
Talk about a retainer

GPU time is also available on its own for teams that bring their own pipeline.

FAQ

Straight answers.

Who owns the model?

You do. Weights, adapters, the datasets we build for you, the eval set and the provenance record are delivered to you. We keep nothing you don't want us to keep.

How much does LLM fine-tuning cost?

A pilot is a fixed price, agreed before we touch your data. A full program is quoted after a one-week scoping phase; the range depends on training stages, data readiness and deployment. Ongoing retraining is a monthly retainer. The full price list is public: see the pricing page linked from the Engagement section.

Should we fine-tune a small language model (SLM) instead of using a large API?

For a narrow, well-specified task - support answers, document extraction, classification, tool calling - a fine-tuned 4B to 12B model usually matches or beats a frontier API on that task, at a fraction of the cost, and runs on your own hardware. For open-ended general assistance the large model still wins. The pilot measures which side of that line your use case is on.

Won't my model be obsolete when the next base model comes out?

Only if nobody re-tunes it. Because the dataset and the frozen eval already exist, moving to a new base is days of work, not months. That is what the retainer is for. Without it, you still own everything needed to do it yourself.

Why pay when GaMS and other Slovene models are free?

Because they are a starting point, not a product. A public model does not know your documents, your terminology or your procedures; it does not run on your side with a written evaluation, and nobody re-trains it when your data changes. Our process starts on exactly such bases and returns a model measured on your set, running on your server. Where the difference lies on general tasks we show publicly in the Public or tuned section.

Can we just buy one of your published models?

They are open — take them. But a generic model is a starting point, not an advantage. The gain comes from your data, your eval and keeping the model current. We use our published models as accelerators for exactly that.

Can a small model really replace our API spend?

On a narrow, well-specified task, usually yes, and at a fraction of the cost. Below a certain volume the API is still cheaper than running your own GPU. The pilot tells you which side of that line you are on — and a cascade (small model first, frontier fallback on low confidence) often wins on both cost and quality.

Fine-tuning or retrieval (RAG)?

Usually both. Retrieval is right for facts that change daily; fine-tuning is right for behaviour — formats, terminology, tool use, tone, and making a small model do what only a large one could. The pilot tests which one actually moves your metric before you commit.

Which base models do you work with?

Any open-weight model whose licence fits your use: Qwen, Gemma, Llama, Mistral for text; Nemotron and Parakeet for speech recognition; open TTS models for speech synthesis. We track the licence of every base and every data source, and we will tell you if a combination is not commercially usable.

Which languages?

Any. Most industry programs run in the customer's language on top of a strong multilingual base; Slovenian is our proven case end to end. For under-served languages the same recipe — continued pre-training, fine-tuning, frozen evaluation — applies, and we assess honestly how much usable text exists before promising anything.

How much data do I need?

For fine-tuning, a few thousand high-quality examples already move the needle. For continued pre-training, hundreds of millions of tokens. What you don't have, we can build, translate or distil — that is half of what we do.

Do I have to run the model locally for sensitive data?

Not by law — for ordinary personal data, an EU-hosted provider under a processing agreement is compliant. For special-category data (health, biometric), banking secrecy, classified material or trade secrets, a foreign API is a sub-processor your DPO has to justify, and most choose not to. A model on your own hardware is the only setup where "the data never left" can be verified rather than promised. See the deployment ladder above for the rung that fits.

Can the model run offline?

Yes. Our 4B models run on a laptop; 27–35B models on a single workstation GPU. Everything we ship works with the network cable unplugged.

Does this help with the EU AI Act?

With the documentation side, yes: every model ships with a data card, provenance record, training configs and evaluation results — the artefacts transparency and record-keeping obligations ask for. We are not lawyers; we give your compliance team what it needs.

How long does it take?

A measured pilot model within weeks. A full program depends on data readiness; you get a dated schedule after the scoping week, and every phase ends with a number you can check.

Working in Croatian, Serbian, a Baltic or another under-served language? Language models for under-served languages →

Send us fifty examples.

Show us fifty real prompts and answers from your domain. Within a week we'll tell you what a model trained on your data could do — and exactly what it would take.

MediaAtlas d.o.o. · Sevnica, SloveniaNDA by defaultEU jurisdiction