SOTA tool roundup (2026)

A pragmatic survey, not a hype list. For each category we name what exists, what we'd actually pick, and what to skip.

Methodology note. Tool landscapes shift quarterly. This is a snapshot for 2026; treat it as "starting set," not "evergreen reference." When the capability you need is new, go to the source repo's README — not a blog post.

Coding agents

Tool	Notes	Pick when
Claude Code	CLI-first; markdown config (CLAUDE.md, slash commands, subagents); strong tool use.	Default.
Cursor	IDE-first; tight inline edits; Tab completion is best-in-class.	You live in an IDE and want inline UX.
Aider	CLI; diff-based edits; transparent.	You want a minimal, scriptable edit/apply loop.
Windsurf	IDE; agent-style task runner.	Cursor alternative; team preference.
Codex CLI	OpenAI's CLI agent.	When the workflow targets OpenAI models specifically.
Continue	Open-source IDE plug-in; bring-your-own-model.	Self-host, OSS-purist.

Our default: Claude Code, with patterns documented here. The patterns transfer.

Local model serving

Tool	Notes	Pick when
vLLM	Fast, mature, paged-attention; the production default for transformer LLMs.	Self-hosted serving, throughput matters.
SGLang	Strong programmable scheduler; great for structured outputs and agent loops.	Multi-step prompts, structured generation.
TGI (HF)	Stable; tight HF integration.	You're already in the HF ecosystem.
llama.cpp / ollama	CPU/GPU local; quantized; user-friendly.	Laptops, edge, prototypes, demos.
MLX (Apple)	Mac-native; fast on M-series.	macOS local dev.

Experiment tracking

Covered in its own chapter. Short answer: MLflow if you self-host, W&B if you don't, Aim if you want OSS without MLflow's quirks, plain JSONL for auto-research harnesses.

Hyperparameter / sweep

Tool	Notes
Optuna	Mature, scriptable, integrates with most trackers. Default.
Ax / BoTorch	Stronger Bayesian primitives; heavier.
Ray Tune	Scales to clusters; pulls in Ray.
W&B Sweeps	Convenient if you're already on W&B.

Data versioning

Covered in its own chapter. Short: HF Hub for public, DVC or S3-versioning for private, manifest+hash as the universal floor.

Annotation

Tool	Notes
AnyLabeling	Local-first, cross-platform, supports SAM-style assist. Disclosure: maintained by the author of this handbook.
Label Studio	Mature; large team workflows; SaaS or self-host.
CVAT	Computer-vision-heavy; web; self-hosted.
Roboflow	SaaS; computer vision pipelines; quick start.

Eval / benchmarks

Tool	Notes
lm-evaluation-harness	Standard for LLM benchmarks; pin a version.
Inspect AI	Anthropic-built; structured eval framework with strong logging.
OpenAI Evals	Reference; strong for paired comparisons.
LangSmith / Langfuse	LLM tracing + eval; SaaS or self-host.
Phoenix (Arize)	OSS LLM observability; local-friendly.
Promptfoo	YAML-driven prompt evals; CI-friendly.

Vector / retrieval

Tool	Notes
pgvector	PostgreSQL extension; one less moving part if you already have Postgres. Default for most teams.
Qdrant	Standalone vector DB; strong filtering; OSS.
LanceDB	Embedded; Parquet-friendly; great for analytics workflows.
Weaviate, Milvus	Mature; heavier; choose for scale or specific features.
FAISS	Library, not a server. Use inside your own service when scale fits a single box.

Agent frameworks

Tool	Notes
Pydantic AI	Typed; small surface; works with any model. Our default for new code.
LangGraph	Stateful, graph-based; good for branching workflows.
LlamaIndex	Strong RAG primitives; can feel heavy.
Instructor / Outlines	Structured output enforcement — pair with whichever framework.
DSPy	Compile-style approach; novel but framework-shaped.
AutoGen, CrewAI	Multi-agent orchestration; useful when the abstraction matches your problem.

For most ML-engineering work, don't reach for a framework first. A 200-line agent.py with explicit tool calls and a hand-written loop will outperform a framework setup for the first three iterations. Adopt a framework when you've felt the pain it solves, not before.

Cloud compute / training

Tool	Notes
Modal	Python-native, serverless GPUs; great for one-off training and batch jobs.
Lightning AI Studios	IDE + GPU sessions; good for interactive work.
Skypilot	Multi-cloud; spot-instance friendly.
RunPod / Lambda / Vast	Raw GPU rentals; cheapest per hour but you manage everything.
HF Inference Endpoints	Easiest path from HF model to deployed API.

Documentation sites

Tool	Notes
MkDocs Material	Python; mermaid native; what this site uses. Default.
Docusaurus	React-based; richer plugins; heavier.
VitePress	Fast; Vue-based; great DX.
mdBook	Rust-simple; less flexible.
Quarto	Strong for executable docs (notebooks → HTML).

What to skip (mostly)

Heavyweight ML platforms with proprietary configs. They lock you in and the abstractions rot when fundamentals shift.
Tools whose value is mostly their UI. UIs change. If the underlying artifact is a runs/<id>/ directory you can grep, you have the durable thing.
"All-in-one" agent platforms that wrap an LLM call in a SaaS UI. Build the loop with primitives; you'll move faster.

What to watch

Open-source long-running agent harnesses (continuous "compute → LLM → act") are improving fast. Worth re-evaluating quarterly.
Code-running sandboxes (E2B, Daytona) for safer agent tool use.
Better eval-as-code standards. Today's eval tooling is fragmented; expect consolidation.

The tools shift. The patterns in this handbook should not.