Same Engineering Habits, New AI Words

More than twenty years into my career, I reinvented separation of concerns and didn't recognize it.

I had given my AI agents job titles: Java developer, front-end developer, designer, architect, tester, DevOps engineer. The output got better, so I kept handing out hats. Then I read the research, which says a job title on its own doesn't make a model any more accurate.¹ The titles weren't doing the work. Each agent had a smaller job and saw only the files that job needed. That's separation of concerns, a habit I've had since 2004, under a new name.

It wasn't the first time a pattern got ahead of its name for me. Early in my career I wrote a factory in a J2ME app before I knew the factory pattern existed, and later found my own code in a book.

I wrote the pattern first and learned its name later.

A year and a half into building with AI, I keep finding the same thing. The tools are new, and so are the names: context engineering, skills, agent hooks. Underneath most of them is a fundamental I already knew. And this isn't only about code. Some of my biggest lessons came from pointing agents at documents and research, and the fundamentals held there too.

The way I learn them hasn't changed either. In 2004 I read other people's code, asked whoever sat at the next desk, tried things and kept what worked. I only sometimes found out what it was called. That's how the patterns got their names in the first place: the Gang of Four wrote that their book held nothing new, only designs that already worked in more than one real system.² It's the same loop today: try something, read what others found, notice what keeps working, give it a name, reuse it.

The loop I learned on in 2004 is the one I still use.

The fundamentals underneath

At first the model wrote code faster than I could, so I let it. Then I read what it wrote more closely, and added guardrails. Those are CI checks by another name: automated gates every change has to pass, whoever wrote it.

On bigger projects, what the model could see decided how good its work was. That has a name now, context engineering: giving the model everything relevant, so the task is solvable at all.³ It's what I've always done when handing work to a new teammate: the background, the right documents, the constraints. With agents, I had to build that briefing as software. The project's instructions file (CLAUDE.md, AGENTS.md) is its first page, the README a new teammate reads on day one. I built retrieval over Markdown, using libraries to convert documents into text, and the hard part was making sure everything relevant actually arrived. The conversion libraries did little with images, and some of what mattered was only in the diagrams and charts. First I had to clear the clutter: meeting transcripts repeat each participant's profile photo, and emails end with the same signature. In one document library, 25,082 image references came down to 991 unique images once ingestion hashed them and skipped the repeats, and most of the 413 small ones it then dropped were avatars. I had an LLM describe the remaining 578 in plain words, and search could find what was in them.

My first knowledge graph made search worse. Its schema had effectively been designed by regular expressions: any email address became a person, and any phrase after the word "about" became a topic. The next one was small, and every part of it was there because a question needed it. With that schema in place I often didn't need retrieval at all. That's data modeling, and I had to learn it again. Now I write down the questions before I design a schema.

Around then I started turning anything I did twice into a command, a saved prompt I run by name. That's a macro, and I've always scripted anything I do twice. Producing a chapter of the reference I'm writing used to be a sequence of steps I kept in my head, and now and then I skipped one. Now it's one command that runs every step.

Commands grew into skills: a folder of instructions, and sometimes scripts, that an agent picks up when a task calls for it. Once I had more than a handful, copying them from project to project was the problem Maven and Artifactory solved for Java libraries, and the fix was the same. My skills live in a shared repository, and each project pulls the ones it needs. Agent plugins can now even declare dependencies on each other, which I wrote about as a pom.xml moment. Skills also load lazily. The agent starts with only each skill's name and a one-line description, and reads the full instructions when a task needs them.⁴ The skill docs call this progressive disclosure, a term borrowed from interface design. To an engineer, it's lazy loading.

Hooks were the most familiar idea of the lot. Write "never push to main" in a prompt and the model will usually listen. Put it in a hook and the question is settled before the command runs, by my code rather than the model's judgment. Hooks work in the other direction too: one can refuse to let the model stop until the tests pass. I've written pre-commit hooks for years, and these work the same way, except the change they're checking came from an agent.

Then agents learned to message each other, and I stopped being the switchboard. I automated the tedious setup (worktrees across several repos, each on its own ports) and handed the scheduling to a chief-of-staff agent. It runs sessions in parallel, lets them talk when they need to, and lines them up when one is waiting on another.

Two ideas from other people changed how I think about long runs. Boris Cherny's advice is to give the agent a goal, with /goal or /loop, and it will keep working for hours.⁵ A goal is a definition of done, written where the agent can check it. Andrej Karpathy's autoresearch is a loop where the agent changes something, measures it, keeps the change if the score improved, and goes again.⁶ He built it to improve model training, but it's hill climbing, and it fits anything you can put a number on.

There's one idea I haven't tried. People keep saying that if you encourage the model, tell it the work is good and that it can be bold, it takes on more of the job by itself. I can't tell yet whether that's real or folklore. I'll try it and report back.

New words, mostly for habits engineers already had.

The noise

The loop only works if the reading step works, and that has gotten harder. My LinkedIn and X feeds are full of sponsored posts and threads written to be clicked. "This guy built a whole SaaS in a weekend." "This paper will revolutionize everything." "Just dropped." Most of them are a screenshot and a promise, and I've learned to scroll past.

Every technology I've worked with arrived with someone promising it would change everything. The difference now is the volume and the speed. What I look for is someone who tried the thing on real work and tells you what broke.

Most of the feed is shouting. Keep what was tried on real work.

These are the people whose writing I look forward to, and I'm grateful they take the time to share it:

• Simon Willison, who writes up what he tried, nearly every day, failures included.
• Boris Cherny and Thariq Shihipar, who build Claude Code and share how they use it.
• Anthropic's engineering blog, especially Building effective agents and Effective context engineering for AI agents.
• Birgitta Böckeler's memos on Martin Fowler's site, short and grounded in real teams.
• Hamel Husain, on evals: how to tell whether a change actually helped.
• Latent Space, for long conversations with the people building the tools.

What's different now

Not everything has an old name. In most of the pairs above, the decision stays with me or my code: the hook blocks the push, the CI check fails the build, the schema decides what can be asked. When the model decides which tool to call next, or what to try after a step fails, I don't have an old name for that. That part is new, and I'm learning it the same way I learned everything else.

The other change is what I carry between projects. For years, a new Java service started at Spring Initializr: pick the dependencies, download a zip, build from there. Now that the model writes the code, I rarely start from a zip. I start from a short list I set up before the first prompt: guardrails, one small job per agent, a schema designed from the questions, a goal before any long run, a command for anything I do twice, skills pulled from a shared repository, and a hook for anything that must never happen. The model writes the code fresh each time. The list is what I take from one project to the next.

then

now

The zip used to travel between projects. Now the list does.

A list doesn't choose what to build, though. Knowing what's worth building was the hard part in 2004, and it still is. I wrote about that separately.

If you're new to this and it feels as if everyone else got a manual, they didn't. They're trying things and reading each other, the same as you. Some of them write it down, and I'm thankful they do. If there's someone you look forward to reading, I'd love to hear who.

---

1. Zheng et al., "When 'A Helpful Assistant' Is Not Really Helpful", Findings of EMNLP 2024 (162 personas, no gain on factual questions); Basil et al., "Playing Pretend: Expert Personas Don't Improve Factual Accuracy", 2025. ↩

2. Gamma, Helm, Johnson and Vlissides, Design Patterns (Addison-Wesley, 1994), Introduction: "None of the design patterns in this book describes new or unproven designs. We have included only designs that have been applied more than once in different systems." ↩

3. Tobi Lütke proposed the term in June 2025 and Andrej Karpathy endorsed it; Simon Willison collected both. ↩

4. Barry Zhang, Keith Lazuka and Mahesh Murag, "Equipping agents for the real world with Agent Skills", Anthropic, October 2025. ↩

5. Claude Code's /goal keeps working until a completion condition is met; under the hood it is a Stop hook. Boris Cherny, "five tips for running Opus autonomously for hours", June 2026. ↩

6. Andrej Karpathy, autoresearch, March 2026: the agent edits a training script, trains for five minutes, keeps the change if the result improved, and repeats. ↩

Lessons from fine-tuning a SLM

Field report · four training runs · June 2026

Simplified Technical English didn't do it. Zinsser didn't do it. Google's style guide didn't do it. So I spent three weeks and $113 training a model to sound like the engineers I like reading. A one-line prompt to Claude beat it.

4training runs

$113total spend

11logged failures

1usable model

Openingthe bet

Forty dollars of rented H100 time, and my model finally spoke. It opened with a byline, for an article nobody had written, then repeated its own title and dropped a stray HTML comment into the middle of a paragraph.

That was run one of four.

The plan had a clean shape. A frontier model knows things, so let it carry the facts. A small model, tuned on good human technical writing, would carry the prose. Knowledge from the big one, voice from the little one.

I ran the idea past Claude before I spent a dollar. It told me, politely, that I would lose: a well-prompted frontier model would out-write a fine-tuned small one, because it is already the better writer and prompting costs nothing next to training. It was a good argument. I built the thing anyway, because I did not want to be told.

It lost. And I would spend the $113 again, which is what this post is about.

Onethe problem

The rulebooks I tried first

The technical prose these models produce is slop. Padded, hedged, structurally identical from one answer to the next, every point wearing a bold label. Once you have noticed the shape you cannot stop noticing it, and I was fighting it in every draft.

So I did not start by renting a GPU. I started with the rulebooks, because people have been codifying good technical writing for fifty years and I assumed somebody had already written the answer down.

ASD-STE100, Simplified Technical English, is the strictest: roughly nine hundred approved words, one meaning per word, short sentences. It exists so a technician reading a maintenance procedure in their third language, at two in the morning, on an aircraft, cannot misread it. At that job it is superb. It also strips the voice out on purpose, which is the entire point of it and exactly my problem.

Google's developer documentation style guide makes a large team's output consistent, which is hard and worth doing. Consistent and alive are different things.

Zinsser comes closest. But when you codify On Writing Well into something a model can follow, the half that survives is the subtractive half. Cut the adverb. Cut the qualifier. Those are rules. "Build a paragraph that carries an argument" is a craft, and it does not fit in a rubric.

Hold on to that. I am about to make the same mistake, at much greater expense.

Each of those helped, and none got me where I wanted. The output came back shorter, tighter, more correct and no more alive, because "don't sound like a machine" is a far weaker instruction than "sound like this person." You cannot write down what a good voice is. You can point at a few hundred examples of one. So point.

Twothe pipeline

Eight stages, and how each one lies to you

Tuning a twelve-billion-parameter model is a sequence, and almost nothing in the sequence fails loudly. The model rarely crashes. It hands you something slightly wrong and lets you discover it much later, usually after you have paid for the run that produced it. Every failure below happened to me.

The eight stages, and where each one lies

Left: what the stage does. Right: how it goes wrong while reporting success.

Every setting is a hypothesis, and you learn which ones were wrong all at once, at the end.

Threethe corpus

Your model is a mirror of your corpus

Run one finished cleanly and produced garbage. It wrote author bylines, repeated its own title, left HTML comments sitting in the prose. I had scraped a few hundred sources and then taught the model, carefully and at some expense, to imitate the scrape.

The audit was humbling. My data preparation code was prepending a markdown byline to 89% of the training documents. Nearly a fifth of my instruction examples ran longer than the context window, so the trainer cut their tails off and took the stop token with them. That is why the model never knew when to stop talking.

The subtler failure showed up after I had fixed all of that. The corpus passed every cleanliness check I could write and was still wrong, because four-fifths of it by word count was dry specification text. I had assembled it for coverage when the model's entire job was register. Rewriting the weighting took readable human voice from 16.5% of the corpus to 59.7%, and made it smaller doing it: 11,546 packed blocks down to 7,034.

Then someone asked me a plain question mid-run: will it stop answering with links? I went and looked instead of guessing, and 84% of my instruction answers contained inline markdown links, six apiece at the median. I killed the run there, two dollars in, stripped the URLs and started over. Letting it finish would have cost $9 and produced something I would have thrown away. One plain question was worth more than every automated check I had written.

Fourthe failure

The run that went perfectly and produced a wreck

The second attempt trained to completion, published its result and reported success. Final evaluation loss of 7.37, worse than where an untrained model starts. Training dropped the way it should, bottomed out around 1.48 by step 30, then blew up and stayed up for the remaining seven hundred steps. A learning rate one notch too high. Nothing errored. It spent the money and handed me the wreck with a clean exit code.

Three separate safety nets should have caught this. All three were looking somewhere else.

Where the guards were looking

Run v2b, lr 2e-4. Loss values are from the run's history; guard positions are from the config.

The smoke test ran forty steps and saw only the healthy opening descent. The quality gate that generates real text crashed on a tokenizer quirk, and my own rule said to treat a crashed check as infrastructure and carry on. The trainer's keep-the-best-checkpoint setting had nothing good to keep.

A smoke test that ends before your failure mode starts only reassures you. So I stopped and rebuilt the harness before spending another dollar: a kill switch inside the training loop, milestones with the billing switched off between them, and uploads verified by re-reading the destination rather than trusting an exit code.

Fivethe bill

What it cost, run by run

Three of the four runs produced nothing I kept. That ratio is roughly what learning something looks like.

Four runs, one usable model

Rented H100 time at $3.29/hour, June 2026.

v1poisoned corpus

$40

v2 first tryuploads failed silently

$21

v2 second trydiverged, stopped

$19

v3trained clean, published

$33.50

The $21 run still stings: it trained correctly, then every upload failed quietly because the command was written to ignore its own errors, and the machine destroyed itself on schedule with the only copy on it. An H100 also sat idle for 1.6 hours while I made one decision, which cost $5.30 for nothing.

Sixthe verdict

The test that actually mattered

The third run trained clean and measurably cut the machine-writing tics: 0.91 on my LLM-speak metric against a stock Gemma's 1.33, winning thirteen of twenty held-out briefs. I had built a voice tool rather than a smarter model, which is what I set out to build.

Then I gave the same notes and the same plain instruction to three writers: Claude Sonnet, a stock Gemma 4 26B, and mine. Then I read all three.

Claude Sonnet

Frontier · plain prompt · 972 words

"When you stuff a complex task into a single prompt, you're asking the model to do too many things at once. It will drop instructions, mix up concerns that should stay separate, and produce output that's difficult to debug."

Every word inside a paragraph. Not one bullet in the whole chapter.

Gemma 4 26B

Stock · plain prompt · 874 words

"…you gain three massive advantages:

* Reliability: Each call is simpler…
* Observability: You can inspect and test…
* Granular Control: You aren't locked in."

Opens with good paragraphs, then reaches for a bold label every other line. More than half its words live inside bullets.

My fine-tune

Tuned for voice · 422 words

"Your LLM prompt is too big and unwieldy for the task."

"Your prompt has a lot of conflicting instructions."

"Your prompt's output is hard to parse."

No filler, no hedging, and no paragraphs either. I asked for a chapter and got a list of sentences standing on their own.

Average length of a paragraph, in words

Measured on the three chapter files. Headings excluded.

Sonnet

44.9

Gemma 26B

40.1

My fine-tune

10.8

At eleven words, those are sentences with line breaks between them. My model had learned to strip the padding out of a sentence and had never learned to build anything out of sentences. Sonnet put 943 words in paragraphs and zero in bullets; Gemma split 361 against 487.

So the model I trained specifically for voice lost on voice, to a model I simply asked nicely.

Seventhe reason

Why it lost

I had taught it the easy half of writing well. It dropped the hedges, the throat-clearing, the "it's important to note." Those are words, and words are cheap to count, so words are what my scoring rewarded.

Structure is the hard half, and structure is the deeper tell. Bulleting everything is what makes machine writing feel like machine writing, more than any individual word does, and my corpus was still full of documents that bullet everything. It taught the structure right back in while I was congratulating myself about vocabulary. I optimized the symptom I could count and missed the one I could not. Which is the same mistake the rulebooks make, run again at $113 a go. Subtraction fits inside a rule. Construction needs judgment.

A frontier model, asked nicely, wrote better prose than three weeks of training did.

Eightwhat to take

What I owe the people who do this for a living

Every mistake here is an ordinary, known thing to someone who trains models for a living. A practitioner would have started the learning rate lower and clipped the gradients without thinking about it. A clean dataset can still be the wrong dataset, which is the first thing anybody who has built one will tell you. I paid $113 to learn a page of things a good team already knows in their hands.

And every choice in the recipe is a hypothesis. I made reasonable guesses and ran four configurations. The people who are good at this run hundreds, changing one variable at a time, and most of those runs teach them nothing except that a door is closed. Patient, largely unrewarded, mostly negative-result work. That is the actual job.

Almost none of what I used did I pay for. The loaders, the trainers, the quantiser, the hub the model sits on: built and given away by people who did not have to. So were the essays my corpus is made of, a few hundred engineers who wrote carefully in public for no particular reward. Whatever voice my model has, it is theirs. I borrowed it.

What $113 bought me

The eight rules I would hand to anyone starting out, in the order they cost me money.

The last row is the only one in the black. Going to look cost two dollars and saved nine.

If you want this voice today, prompt a frontier model. One line of instruction got me cleaner prose than three weeks of training did. If you want a fine-tune that genuinely wins on voice, I think it is findable and I did not find it: a corpus weighted hard toward long-form essayists, an explicit penalty against list structure, and a judge that scores the shape of the prose instead of which words show up in it. Whether that works, I don't know.

Claude told me I would lose, and Claude was right. Being told cost me nothing and taught me nothing. Losing cost me $113 and three weeks, and I now understand with my hands, rather than in the abstract, why training is hard and why the people who are good at it are worth listening to closely. The models will keep getting better at telling us things. We will keep needing to go and find out.

The model is public if you want to poke at it, and the full list of what went wrong is much longer than this post. If you have run this experiment and found the corpus that wins on voice, I would like to hear about it.

The tuned model, adapter and merged weights: huggingface.co/abdwiv/gemma4-12b-tech-writer
Base model Gemma 4 12B · two-pass LoRA · trained on rented H100 · June 2026 · every figure here measured from the run logs and the output files.