Viet-Anh Nguyen

A handbook for ML engineers whose primary implementation engine is an AI agent. Practice for the era when the bottleneck is deciding and verifying, not typing.

The interesting changes are second-order. Most posts focus on the first-order wins; the leverage is in what happens when the cost of trying an idea collapses.

Why AI-augmented ML is different

Honest accounting of an AI-augmented week shows "writing code" rarely tops the list anymore. Three things take its place.

The new bottlenecks

To use the agent well, you need an honest model of what it is. Most failures come from treating it as something else.

Claude Code mental model

The highest-leverage category of files in any AI-assisted ML repo is the set agents read to figure out how to behave. CLAUDE.md is the most visible. It is one of several. They form a layered system; teams that treat…

Agent context files

The short list. Print it. Pin it next to your monitor. Apply on every task.

Operating principles

The loop has the same shape. The economics are different.

Classic vs AI-augmented loop

The loop survives 100× speedup only if each step is structured. Here is the structure we use.

Hypothesis → experiment → measure

You generate results 5–10× faster. If those results are not reproducible, you generate garbage 5–10× faster. Internalize this chapter before the others.

Reproducibility is non-negotiable

The mechanics of building an experiment harness that an agent can drive itself — a bounded action space, machine-readable results, and a hard budget.

Designing research loops

The two parts of an auto-research loop where teams quietly fool themselves. Both warrant their own discussion.

Reward & stop conditions

Three concrete shapes of auto-research that work today, and one that does not. Use them as templates, not gospel — adapt to your own constraints.

Case studies

We default to Claude Code as the primary implementation agent. The reasoning is in Foundations; this chapter is about configuration that actually pays off.

Claude Code as the engine

Pick one. Use it on every run. The worst tracker used consistently beats the best tracker used sometimes.

Experiment tracking

Code without versioning is unsupportable; data and weights are no different. This chapter is about the practical minimum.

Data & model versioning

A repo layout that is friendly to humans and agents. Both audiences benefit from the same property: predictability.

Repo conventions

A pragmatic survey, not a hype list. For each category we name what exists, what we'd actually pick, and what to skip.

SOTA tool roundup (2026)

Tabular, sklearn, XGBoost, LightGBM, classical NLP. This is where AI-augmented workflow shines brightest, because:

Traditional ML with AI

PyTorch, JAX, fine-tuning, distributed training. The economics are different from traditional ML: each experiment costs real money, runs take hours not seconds, and a wrong configuration can burn a day of GPU time.

Deep learning with AI

Building agent systems with the help of agents. The framing is recursive but the practice is concrete: you are designing software whose primary primitive is "LLM call + tool use," and your IDE is also an agent.

Agentic AI with AI

The classical ML failure mode, amplified. Agents refactor data pipelines five times in an afternoon and lose the split each time.

Data leakage & dataset mixing

The most common AI-written bug by volume. Easy to write. Easy to miss in review. Painful to find later.

Hardcoded paths & magic constants

The most damaging failure mode in AI-augmented ML. The agent produces a number that looks like a measurement but came from nowhere. The number lands in a model card, a slide, a blog post, a tweet. Then someone tries to…

Fabricated benchmarks

A test that always passes is worse than no test. It looks like coverage and provides none. Agents produce these by accident with surprising regularity. The patterns below are the common ones; learn to spot them on sight.

Silent test failures

The agent's training data is a snapshot. The libraries you use have moved since. Result: confident code calling functions that no longer exist, with arguments that were renamed, against APIs that were deprecated two…

Hallucinated APIs & versions

The consolidated list. Use it before merging any AI-generated PR that touches data, training, eval, or anything customer-facing. Two minutes. Saves hours.

Guardrails checklist

The shortest path to maintainable AI-augmented code: write the doc first, generate the code from the doc, keep them in sync.

Doc-driven development

The core rule, restated: every number in user-facing material comes from a measurement script in the repo, or a cited public source with a working URL. Nothing else.

Numbers must be measured

When you ship code, you ship the doc that describes it. Same commit. Always.

The same-commit rule

Model cards and dataset cards are the primary external interface for an ML artifact. The agent can draft them well; only you can verify them.

Model & dataset cards

Markdown is the lingua franca for engineers. DOCX, PPTX, and XLSX are the lingua franca for everyone else — clients, leadership, finance, sales, regulators. Treating them as second-class cuts you off from collaboration.

Client deliverables (DOCX / PPTX / XLSX)

A 30-minute path from empty directory to "agent can ship features here." Skip nothing the first time you do it; everything below has earned its place.

Recipe: spinning up a new project

The first model in any new project is the baseline. Not the headline model. The baseline. Without it, no future result has a number to beat.

Recipe: adding a baseline

A benchmark is a runnable measurement script that produces a number you can publish. This recipe walks through writing and running one for inference performance — the most common case.

Recipe: running a benchmark

The 30-minute version. Produces a card that survives external review without embarrassing you.

Lever	What changes	Where to focus your time
Cost to try an idea	Crashes by 10×	Generating better ideas, filtering them well
Cost to ship bad code	Crashes by 10×	Guardrails, verification loops
Verification	Becomes the bottleneck	Cheap, fast, automated checks
Taste	Still 100% human	Encode it in CLAUDE.md, scripts, reviews

Why AI-augmented ML is different

First-order (true but boring)

Second-order (where the leverage is)

1. The cost of trying an idea collapses

2. The cost of bad code also collapses

3. Verification becomes the binding constraint

4. The agent has no taste

What does not change

Summary