# Ask, or It Will Guess: Question-Asking Is the Core Skill of Agentic Work

**Published:** August 8, 2026
**Tags:** AI Agents, Agentic Workflow, Software Engineering, LLM, Prompt Engineering, Productivity

**Summary:** Models almost never ask you what you meant: across ambiguous questions, answer rates run above 95% and clarification barely registers. Repairing that in conversation makes it worse: underspecified multi-turn runs measure a 39% average performance drop, driven by a 112% jump in unreliability, and the effect shows up from two turns onward. The leverage is no longer in writing better prompts. It is in forcing the questions out before the first answer attempt, and in asking the questions afterward that prove you understood what you shipped.


---

The hard part of working with an agent is no longer getting it to produce something. It is that it produces something no matter what you hand it.

Give an agent a half-specified task and you do not get a question back. You get a confident, well-structured, fully-tested implementation of the task it decided you meant. It looks like work. Sometimes it is. But the failure I hit most often now is not a bug. It is a beautifully built solution to the problem sitting next to mine. Nothing in the output signals the fork. It happened silently, in the first thirty seconds, in a decision the agent made on my behalf and never mentioned.

Two years ago the scarce skill was writing the code. Then it was reviewing the code. I wrote about [that shift](/blog/2026-07-05-the-bottleneck-moved-to-review) in detail. But review catches a wrong implementation of the right problem. It rarely catches a right implementation of the wrong problem, because there is nothing wrong with the diff. The only place that failure is cheap to catch is upstream, in the questions nobody asked.

## The model will not ask. It will assume.

This is measured behavior, not a vibe. Su and Cardie evaluated ten models across three families on AmbigQA questions in three settings: plain question answering, explicit ambiguity judgment, and a behavioral pass where a judge model labels each response as a direct answer, a refusal, or a clarifying question. The result: **"Across all models and conditions, the dominant behavior is to give a direct answer: answer rates are typically above 95%, and 80–95% of responses are judged as _only answered_"**, meaning an answer with no refusal and no clarifying question. ([Su and Cardie, 2026](https://arxiv.org/abs/2605.25284))

The part that should bother you is that the models are not confused. When the same models are explicitly asked to judge whether a question is ambiguous, they often get it right. They know. They just do not act on it. The authors call it "knowing but not showing." Of the families tested, the Claude models asked for clarification most often, and "most often" topped out at about 5% on ambiguous questions without retrieved context.

One caveat, because it cuts against my argument: their model set is a generation behind (GPT-4.1 and older, Claude 3.5, Qwen2.5), and I have not seen the experiment rerun on current frontier models, so treat the exact rates as dated. Nothing below depends on whether the real number is 5% or 15%.

Then there is the finding that changes how you should think about your coding agent specifically: **retrieved context makes clarification rarer.** In their words, "providing context slightly increases the share of pure answers and further suppresses the already tiny rates of clarification." That was measured on open-domain QA, not on code, but the mechanism transfers uncomfortably well. An agent that just read forty files of your repository is holding a lot of context that looks like grounding. It will act more prepared to answer, not less, and the ambiguity that lives in your head rather than in the files is exactly the kind context cannot resolve.

It is also not simply a matter of the model being bad at reasoning. QuestBench isolates the narrowest possible version of the skill: underspecified reasoning tasks that become solvable by asking at most one question, where the model only has to pick the right question from a list of options. Current models handle the grade-school math variants well and "achieve only 40-50% accuracy on Logic-Q and Planning-Q." The conclusion is the one to sit with: **"models struggle to identify the right question even when they can solve the fully specified version."** ([Li, Kim, and Wang, NeurIPS 2025 Datasets & Benchmarks](https://arxiv.org/abs/2503.22674))

Solving is not the bottleneck. Knowing what is missing is. QuestBench's whole point is that the two abilities come apart, so getting better at one does not hand you the other.

## Chatting your way to a spec does not work

The intuitive fix is to treat it as a conversation: start loose, refine over turns, let the shape emerge. That is how you would work with a human collaborator, and it is what the chat interface invites. It is also the single worst thing you can do, and the size of the effect is startling.

Laban and colleagues sharded 600 instructions across six tasks and simulated them against 15 LLMs in three conversation types, for more than 200,000 simulated conversations. The headline comparison is the one that matters here: the same instruction delivered fully specified in a single turn, versus "sharded" across multiple turns the way a real user actually reveals requirements. Every top open- and closed-weight model they tested degraded, with **an average performance drop of 39% across six generation tasks**. ([Laban et al., 2025](https://arxiv.org/abs/2505.06120))

The decomposition is where it gets useful. Aptitude (how well the model does in its best runs) fell by only 16%. Unreliability, the spread between its best and worst runs on the same instruction, **increased by 112%**, with performance varying 50 percentage points on average between the best and worst run of a fixed instruction. The model does not get dumber across turns. It gets erratic.

And the threshold for triggering this is far lower than you would hope. Their gradual sharding experiment expanded instructions from 2 to 8 shards while holding task complexity fixed (run on two models, GPT-4o and GPT-4o-mini, so a narrower result than the headline number), and found the collapse begins immediately: "any conversation that involves underspecification and occurs in two or more turns leads to models getting lost in conversation." Their conclusion for users is blunt: **"providing all the information at once (1-shard) is the only effective method to improve reliability."**

The mechanism is the part that should change your workflow. Models commit early: they propose a full answer before they have the information to justify it, anchor on it, and absorb later conflicting turns into an increasingly bloated response instead of correcting. On the two tasks where answer timing varied enough to analyze (code and math), conversations where the first answer attempt landed in the first 20% of turns scored **30.9**, against **64.4** when the model held off until the last 20%. That split is observational, not a controlled intervention, so read it as a strong association. The authors' own summary: "premature answer attempts detract LLM performance."

So there are two failure modes stacked on each other. The model will not ask. And if you try to repair the gap conversationally, you land in the regime where it is least reliable.

```mermaid
flowchart TD
    A["Underspecified request"] --> B{"Does the model ask?"}
    B -->|"~5% at best"| C["Questions surfaced<br/>before any answer attempt"]
    B -->|"Overwhelmingly"| D["Silent assumption<br/>+ early answer attempt"]
    D --> E["Later turns conflict<br/>with the assumption"]
    E --> F["Lost in conversation<br/>unreliability +112%"]
    C --> G["One consolidated spec"]
    G --> H["Single-turn, fully specified run"]
    classDef focal fill:#f3bd92,stroke:#8c4000,stroke-width:2.5px,color:#4a2200
    class C focal
```

The orange node is the whole intervention. Everything downstream of it is determined by whether you got there.

## Make the agent interview you

The fix is not to ask better questions of the agent. It is to make the agent ask questions of you, before it commits to anything, and then to collapse the answers into a single specification you run fresh.

This is not a clever hack. It is in Anthropic's own Claude Code documentation as a first-class pattern, and the prompt they publish is close to what I use:

```text
I want to build [brief description]. Interview me in detail using the AskUserQuestion tool.

Ask about technical implementation, UI/UX, edge cases, concerns, and tradeoffs.
Don't ask obvious questions, dig into the hard parts I might not have considered.

Keep interviewing until we've covered everything, then write a complete spec to SPEC.md.
```

The critical instruction is the last clause, and it is the one people drop. The interview output goes to a file, and **you execute it in a fresh session**. That is not tidiness. It is exactly the single-turn, fully-specified condition Laban et al. found to be the only reliable one. The interview gets to be multi-turn because nothing is being built during it. The build gets one consolidated shot.

The docs put the tradeoff plainly: "Time spent making the spec precise pays off more than time spent watching the implementation." ([Claude Code best practices](https://code.claude.com/docs/en/best-practices))

The measured version of this exists too. ClarifyGPT wraps a code-generation model in exactly this loop: detect ambiguity, generate targeted clarifying questions, refine the requirement, then generate. With real human answers to its questions, GPT-4's Pass@1 on MBPP-sanitized went from **70.96% to 80.80%**, and from 51.52% to 60.19% on MBPP-ET. (Their four-benchmark averages use simulated user answers rather than human ones, which is a weaker claim; the human-feedback numbers are the ones I would quote.) ([Mu et al., FSE 2024](https://arxiv.org/abs/2310.10996))

Nearly ten points of Pass@1 on a benchmark where the model already scored 71, from asking questions. No fine-tuning, no better model, no larger context window.

### Detect ambiguity mechanically instead of guessing at it

The most transferable idea in that paper is not the clarification loop. It is how they decide a requirement needs one. They call it a code consistency check: sample _n_ independent solutions from the same requirement, execute them all against generated test inputs, and compare outputs. Identical outputs mean the model read the requirement the same way every time. Divergent outputs mean it did not, which is a machine-checkable definition of "this requirement is ambiguous." They then cluster by output and feed one solution per cluster into question generation, so the questions target the actual fork instead of generic uncertainty.

You can run a crude version of this today without any framework:

```bash
# Three independent plans for the same task, in separate contexts.
for i in 1 2 3; do
  claude -p "Read the repo and write an implementation plan for: $TASK.
             Output the plan only, no code." > "plan-$i.md"
done

# Where they disagree is where your requirement is ambiguous.
diff plan-1.md plan-2.md
diff plan-1.md plan-3.md
```

Do not read the plans looking for the best one. Read the diffs. Every place three runs made different choices from the same words is a place your specification did not constrain the outcome, and a place the agent would have picked one silently if you had run it once. The diffs are your question list, generated for you.

## The four questions before it writes a line

For anything beyond a one-sentence change, I make the agent answer these before it touches a file. This is my Gate 0 for specification, the way sandboxing is my Gate 0 for execution:

```text
Before writing any code, answer these four:

1. Restate the task as you understand it, in your own words. Do not quote me back to me.
2. List every assumption you are making that I did not state. Be exhaustive and specific.
3. Give me the two most plausible alternative readings of this request, and why you rejected each.
4. Name the single thing that, if I have it wrong, makes this entire plan wrong.

Stop after answering. Do not start implementing.
```

Each one earns its place for a different reason.

**Question 1** catches the fork immediately. A restatement in the model's own words either matches your intent or it does not, and the mismatch is visible in one paragraph rather than in 400 lines of diff. Insisting on "your own words" matters: a model that quotes your prompt back has demonstrated nothing.

**Question 2** is the load-bearing one. Assumptions are where the divergence lives, and the model will make them regardless. This does not give it a new capability. It changes what it does with one it already has. Su and Cardie's result is that models often identify ambiguity when explicitly asked to judge it, and do not surface it when asked to answer. So ask it to judge.

**Question 3** forces the alternatives above the waterline. Left alone, a model picks an interpretation and never mentions the branch existed. This is the manual version of the consistency check: you are asking one run to enumerate what three runs would have disagreed about.

**Question 4** is a value-of-information question, and it is the one I would keep if I could keep only one. It is the same thing QuestBench formalizes: of everything unknown, which one you actually need. It also produces the shortest useful answer. If the agent says "I am assuming this table is append-only," you know in five seconds whether the next hour is wasted.

Then, and only then: approve the plan and let it run. If the answers reveal real gaps, do not patch them in follow-up turns. Fold them into the spec and start fresh.

## The other direction: ask so that you understand

Everything above is about the agent understanding you. The harder half is you understanding what you just shipped, because the natural state after reviewing a good agent diff is a confident feeling of comprehension that has never been tested.

There is a name for that failure. Rozenblit and Keil documented the illusion of explanatory depth: ask people to rate how well they understand how something works, then ask them to produce a step-by-step causal explanation, then ask them to re-rate. The ratings drop, sharply, and the illusion is far stronger for explanatory knowledge than for facts or procedures. ([Rozenblit and Keil, _Cognitive Science_ 26(5), 2002](https://cogdevlab.yale.edu/sites/default/files/files/rozenblit%20%26%20keil%20%202002.pdf))

Reading an agent's diff is the perfect substrate for that illusion. The code is coherent, the naming is good, the tests pass, and following it feels like understanding it. You never had to produce the explanation, so you never found out that you could not — until it breaks at 2am eight weeks later and nobody on the team, including the person who approved it, can say why it was built that way.

The counter is the same three-step procedure, aimed at yourself:

```text
I just merged [change]. Here is my explanation of how it works and why it was
built this way: [your explanation, written from memory, without the diff open].

Find every place my explanation is wrong, incomplete, or where I am asserting
something the code does not actually do. Do not be generous.
```

Write the explanation first, with the diff closed. The friction is the point: it is the retrieval step, and retrieval is what converts reading into knowing. I went into [why this works](/blog/2026-04-30-ai-education-paradox) at more length elsewhere; the short version is that under AI assistance the feeling of understanding and the fact of understanding come apart badly, and the only honest test is production.

Two more questions belong in this set, and I ask them on almost every non-trivial change:

- **"What would make this wrong?"** — asked of the design, not the code. It surfaces the failure conditions the agent optimized past instead of solving.
- **"What did you consider and not do?"** — the rejected alternatives are where the architectural reasoning lives, and they are the first thing lost when a machine writes the commit message.

## The same skill, everywhere else

None of this is specific to code. It is specific to systems that produce fluent output regardless of input quality, which is now most of how people use AI. The clearest evidence is that vendors have had to build the questioning step themselves. Deep research in ChatGPT runs a clarification step before it starts researching. The API version, in OpenAI's own words, "skips this clarification step," and their cookbook is refreshingly direct about the consequence: the model **"expects fully-formed prompts up front and will not ask for additional context or fill in missing information; it simply starts researching based on the input it receives."** The recommended fix is to put a lightweight model in front as a prompt rewriter to "expand or specify user queries before passing them to the research model." ([OpenAI Cookbook](https://developers.openai.com/cookbook/examples/deep_research_api/introduction_to_deep_research_api))

Read that as an architectural admission. The industry's answer to underspecification is to bolt a questioning stage onto the front of the pipeline. If a vendor needs that stage to get usable output, so do you, and when the product does not provide it, you are the stage.

The general-purpose version of the four questions, for research briefs, hiring decisions, financial analysis, medical questions, a legal document you are trying to understand:

```text
Before you answer: ask me the five questions whose answers would most change
your recommendation. Rank them by how much the answer would change it.
Ask one at a time and wait for my response.
```

Ranked by how much the answer changes the recommendation, not "what else should I know," which produces a survey. This is a value-of-information request, and it is the difference between five questions and five useful questions.

Then, once you have an answer you like, one more:

```text
What evidence would make this recommendation wrong?
And what did you leave out because I did not ask?
```

The second half catches something specific to how these systems behave. The model optimizes to answer the question you asked. It will not volunteer the adjacent consideration that reframes the whole problem, not because it is hiding it, but because nothing rewards volunteering it. The tax on a narrow question is a narrow answer that looks complete.

This is where the organizational version bites. DORA's 2025 report, surveying nearly 5,000 technology professionals, put AI adoption at 90% and found that AI acts as "a mirror and a multiplier," amplifying whatever the organization already is. ([Google, September 2025](https://blog.google/innovation-and-ai/technology/developers-tools/dora-report-2025/)) A team that was vague about requirements before AI is now vague at far higher throughput, with plausible artifacts to show for it. The multiplier does not distinguish between clarity and confusion. It scales what you feed it.

## Where more questions make things worse

The honest counterweight, because "ask more questions" degrades into ritual fast.

**Do not drip-feed.** This is the mistake that looks most like following the advice. Asking questions across many turns while the agent is building is the exact sharded condition that produced the 39% drop. Interview freely, build once. If you are three corrections deep, the fix is not a fourth correction. Laban et al.'s own recommendation is to ask the model to "consolidate everything I've told you so far" and carry that into a fresh conversation.

**Do not interview for a typo.** The Claude Code docs are explicit about the overhead: "If you could describe the diff in one sentence, skip the plan." Specification work is proportional to the cost of being wrong. A rename does not need four questions. A schema migration does.

**Do not ask the agent things it can find out.** "Which module owns session refresh?" is not a clarifying question. It is a retrieval task you should have delegated. Clarification is for intent, constraints, and priorities: the things that exist only in your head. Spending the interview on facts the agent could look up wastes the one channel carrying information it cannot get anywhere else.

**Cap the interview.** An agent instructed to find gaps will find gaps, indefinitely, because that is what you asked for. The same caution the Claude Code docs give about adversarial reviewers (that chasing every finding leads to over-engineering) applies to interviews. Ask for the top five ranked by impact, not for everything.

## The bigger claim

Prompt engineering was never really the skill. It was a temporary workaround for models that needed particular phrasing to behave, and the models absorbed it. What did not get absorbed — and shows no sign of being absorbed — is knowing what you actually want and noticing when you have not said it.

The measured picture is consistent across every source here: models solve well-specified problems well, fail to identify what is missing from underspecified ones, almost never ask, and ask less rather than more once retrieved context is in the window. My bet is that this division holds for a while. The machine took the answering; the asking stayed on your side of the table. So the leverage moved again: from writing to reviewing, and from reviewing to specifying. A review can only catch a wrong answer, and the expensive failure in agentic work is a right answer to a question you never meant to ask.

Take the last non-trivial thing your agent shipped. Do not open the diff. Write down, from memory, what it does and why it was built that way. Then open the diff and count how many of your sentences were wrong. That number is how much of your last release you actually own.