Bản tiếng Việt: Context Engineering cho agent: cái gì vào context window, và khi nào. Part 3 of the series. Previous: Agent Harness. Next: Tools, Permissions and Guardrails.

The run that taught me this cost about sixty dollars. A coding agent was asked to fix a flaky integration test. It read the test file, then the module under test, then three dependencies, then the test file again because by step twelve the first read had scrolled out of its attention even though it was technically still in the window. By step twenty it had read the same four files three times each, the context held 400,000 tokens, every call took forty seconds, and the final patch was wrong. Nothing in that run was a model failure. Everything in it was a context failure.

Context engineering is the discipline of deciding what is in the model's window at each step of the loop. The term got popular in mid-2025 when Andrej Karpathy called it the better name for what people were calling prompt engineering, and Anthropic wrote the reference post on it that September. I like the name because it says what the job is. You are not writing a prompt. You are curating a small, expensive working memory for a worker that forgets everything between calls.

The window is a budget, and the loop compounds it#

Everything the model sees on a call is in the window: the system prompt, the tool definitions, every prior turn, every tool result, anything you retrieved. You pay for all of it on every call, and in an agent loop every call includes everything from the previous calls. The cost is not linear in the number of steps. It is roughly quadratic.

Suppose the fixed part (system prompt plus tool definitions) is 8,000 tokens and each step adds 12,000 tokens of tool results and reasoning. Step n sends 8,000 + 12,000 × n. Over thirty steps that is 30 × 8,000 plus 12,000 × (1 + 2 + … + 30), which is 240,000 plus 5,580,000: about 5.8 million input tokens for a single task. Prompt caching takes a large bite out of the price, but not out of the latency, and not out of the quality problem, which is that models get measurably worse at using information in a crowded window long before they hit the hard limit. The industry has a name for that too: context rot.

So the goal is not "fit in the window". The goal is to keep each step's window as small as it can be while still containing what that step needs. Five techniques do most of the work.

1. Retrieve just in time; do not pre-load#

The RAG reflex is to run a search up front and stuff the top twenty chunks into the prompt. For a one-shot question that is fine. For an agent it is usually wrong: you are guessing what the model will need before it has started, and paying for the guesses on every subsequent step whether they were right or not.

The agentic alternative is to give the model a map and tools, and let it pull what it needs when it needs it. The map is small: a directory listing, a table of contents, the CLAUDE.md pointers from part 2. The tools are narrow: search(query, limit), read(path, start_line, end_line), sql(query) with a row cap. The model decides what to open, and only the things it opened enter the context.

In our document pipeline this flipped a retrieval design on its head. Instead of embedding every contract and retrieving chunks, the agent gets a list_sections(doc) tool that returns headings with page numbers, and a read_section(doc, heading) tool. It opens two or three sections per question instead of receiving twenty chunks, and accuracy went up because the sections are coherent units rather than 512-token slices. The embeddings index still exists; it backs the search tool for the cases where the model does not know where to look.

The hybrid is the common case: a small pre-loaded map plus tools for everything else. What to avoid is the pure pre-load, where retrieval happens once, blindly, before the loop.

2. Tool results are the thing you control#

Part 1 said the tool result string is the first lever of agent quality. In context terms it is also the biggest line item: in most traces I have looked at, tool results are 70 to 90 percent of the tokens. Three habits keep them in check.

Cap every result, and say what was cut. A read that returns 300 lines and "… 2,140 more lines; call again with start_line=300" beats one that returns 2,440 lines. The model can ask for more; it cannot un-read what you forced on it.

Return references for large artifacts. If a tool produces something big (a build log, a dataset sample, a rendered page), write it to disk and return the path plus a short head. The model can open the parts it needs with read. This also makes the trace useful: the artifact is on disk, not inlined into a megabyte of JSON.

Replace old tool results with stubs once they have been used. After the model has acted on a file read, the full text rarely matters again. Swap it for a one-line stub ("read src/extract.py, 8,124 chars, see trace step 12"). If the model needs it, it reads it again, which costs one call instead of taxing every call. This is the single most effective trick in the list and almost nobody does it, because frameworks treat the message history as immutable.

Here is a context manager that does the last two mechanically. It tracks an estimate of the window size, stubs out tool results older than the last few turns, and compacts when the estimate crosses a threshold.

# context.py — keep an agent's window small without losing what matters
import json
import os
from dataclasses import dataclass, field

import anthropic

MODEL = os.environ.get("ANTHROPIC_MODEL", "claude-sonnet-5-5")
client = anthropic.Anthropic()


def estimate_tokens(obj) -> int:
    """Cheap estimate: ~4 chars per token for English, ~2.5 for Vietnamese.
    Use client.messages.count_tokens() before expensive decisions."""
    text = obj if isinstance(obj, str) else json.dumps(obj, ensure_ascii=False, default=str)
    return len(text) // 3


COMPACT_PROMPT = """Summarise the conversation so far for an engineer who will continue the task.
Keep: the task, decisions made and why, files/resources touched with their paths, facts discovered,
open questions, and the exact next step. Drop: raw tool output, dead ends already resolved.
Use short headed sections. Under 600 words."""


@dataclass
class Context:
    system: str
    tools: list
    messages: list = field(default_factory=list)
    keep_recent_turns: int = 6          # tool results in the last N turns stay verbatim
    compact_at: int = 120_000           # estimated tokens that trigger compaction
    result_cap: int = 8_000             # characters per tool result
    _stubbed: set = field(default_factory=set)   # tool_use_ids already replaced by stubs

    def add_user(self, content):
        self.messages.append({"role": "user", "content": content})

    def add_assistant(self, content):
        self.messages.append({"role": "assistant", "content": content})

    def add_tool_results(self, results: list[dict]):
        for r in results:
            text = r["content"]
            if len(text) > self.result_cap:
                head = text[: self.result_cap]
                r["content"] = f"{head}\n... [{len(text) - self.result_cap} chars omitted; ask for a narrower range]"
        self.messages.append({"role": "user", "content": results})
        self._stub_old_results()
        if estimate_tokens(self.messages) > self.compact_at:
            self.compact()

    def _stub_old_results(self):
        """Replace tool results older than keep_recent_turns with one-line stubs."""
        user_turns = [i for i, m in enumerate(self.messages) if m["role"] == "user"]
        for i in user_turns[: -self.keep_recent_turns]:
            content = self.messages[i]["content"]
            if not isinstance(content, list):
                continue
            for block in content:
                if block.get("type") == "tool_result" and block["tool_use_id"] not in self._stubbed:
                    size = len(block["content"])
                    block["content"] = f"[tool result, {size} chars, elided; call the tool again if needed]"
                    self._stubbed.add(block["tool_use_id"])

    def compact(self):
        """Summarise everything except the last two turns into a single note."""
        head, tail = self.messages[:-2], self.messages[-2:]
        summary = client.messages.create(
            model=MODEL, max_tokens=2000,
            system="You compress agent transcripts without losing decisions or state.",
            messages=[*head, {"role": "user", "content": COMPACT_PROMPT}],
        )
        note = "".join(b.text for b in summary.content if b.type == "text")
        self.messages = [
            {"role": "user", "content": f"Context so far (compacted):\n\n{note}"},
            {"role": "assistant", "content": "Understood. Continuing from that state."},
            *tail,
        ]

    def call(self):
        return client.messages.create(
            model=MODEL, max_tokens=4096, system=self.system, tools=self.tools, messages=self.messages
        )

Note what compaction keeps: decisions, paths, facts, next step. That list is the difference between a compaction that works and one that quietly deletes the reason the agent chose approach B over approach A, so it tries A again.

A caution on caching: prompt caching keys on an exact prefix. Stubbing old results and compacting both change the prefix and invalidate the cache for that call. Do them in big, rare steps rather than a little on every turn, and keep the system prompt and tool definitions at the very front so that part always hits.

3. Compact deliberately, not when you run out#

Every harness compacts eventually; the question is whether it happens on your terms. Automatic compaction at 95 percent of the window, which is what most tools do by default, means the summary is written at the moment the model has the least room to write it well, and it summarises the whole run including the parts that no longer matter.

The better pattern is to compact at a boundary: a feature finished, a sub-task closed, a plan agreed. At a boundary, the agent knows what mattered. The features.json and progress-file pattern from part 2 is compaction by design: the state that matters is written to disk continuously, so the transcript can be thrown away at any session end and the next session rebuilds its context from files. If you take only one idea from this section, take that one: put durable state in files the agent maintains, and let the transcript be disposable.

4. Memory files: the agent writes its own context#

A memory file is a file the agent is told to read at the start and update as it goes. The simplest version is a NOTES.md with three sections: what I have learned about this codebase, what I have tried that did not work, and what is left. Claude Code's CLAUDE.md plays this role across sessions for humans and agents alike; the progress log plays it within a task.

The rule that makes memory files work: instructions must say when to write, not just that writing is allowed. "After every tool call that reveals something surprising, append one line to NOTES.md" produces a useful file. "You may keep notes" produces nothing. Models are good at following triggers and bad at volunteering.

A second rule, learned the hard way: memory files are context too. A 2,000-line NOTES.md is just another way to blow the budget. Give the agent a rule to prune it ("keep under 200 lines; replace detail with pointers to files") or have a nightly job summarise it.

5. Sub-agents are context isolation, not parallelism#

Multi-agent systems get sold as teamwork. The reason they actually help is narrower and more useful: a sub-agent has its own window. When the main agent needs to search fifty files for a pattern, it can either read them itself and carry 150,000 tokens of their contents for the rest of the run, or hand the question to a sub-agent that reads them in its own window and returns a 400-token answer. The main agent's context grows by 400 tokens, not 150,000.

That is the whole trick, and it is why the right unit of delegation is "a question with a small answer": find where X is configured, list every caller of Y, check whether the docs mention Z. It is also why sub-agents should not share the parent's transcript; the point is that they do not.

# research_tool.py — a sub-agent exposed to the main agent as a single tool
from context import Context, client, MODEL

SEARCH_TOOLS = [
    {"name": "grep", "description": "Search files for a regex. Returns up to 50 matches with path:line.",
     "input_schema": {"type": "object", "properties": {"pattern": {"type": "string"}}, "required": ["pattern"]}},
    {"name": "read", "description": "Read lines [start, end) of a file.",
     "input_schema": {"type": "object", "properties": {"path": {"type": "string"}, "start": {"type": "integer"},
                                                       "end": {"type": "integer"}}, "required": ["path"]}},
]


def research(question: str, execute, max_steps: int = 12) -> str:
    """Answer a narrow question about the codebase in an isolated window. Returns a short answer."""
    ctx = Context(
        system="You answer one question about a codebase using grep and read. "
               "Be exhaustive in your search and terse in your answer: under 300 words, cite path:line.",
        tools=SEARCH_TOOLS, keep_recent_turns=4, compact_at=60_000,
    )
    ctx.add_user(question)
    for _ in range(max_steps):
        response = ctx.call()
        ctx.add_assistant(response.content)
        calls = [b for b in response.content if b.type == "tool_use"]
        if not calls:
            return "".join(b.text for b in response.content if b.type == "text")
        ctx.add_tool_results([
            {"type": "tool_result", "tool_use_id": c.id, "content": execute(c.name, c.input)} for c in calls
        ])
    return "Research budget exhausted; partial findings: " + "".join(
        b.text for b in response.content if b.type == "text"
    )


# In the main agent's tool list:
RESEARCH_TOOL = {
    "name": "research",
    "description": "Delegate a narrow codebase question to a research agent. Use for anything that needs "
                   "reading more than two files. Returns a short answer with citations, not raw files.",
    "input_schema": {"type": "object", "properties": {"question": {"type": "string"}}, "required": ["question"]},
}

The description of that tool is doing real work: it tells the main agent when to delegate (more than two files) and what it will get back (a short answer, not files). Without that, the main agent reads the files itself, because reading is always available and delegation is a judgement call it will not make unprompted.

The system prompt, briefly#

After all that, the system prompt is the smallest part of context engineering, but it still matters. Anthropic's post describes the target as the right altitude: specific enough to guide behaviour, not so rigid that it becomes a brittle list of rules the model pattern-matches against. The failure modes are on both ends. A ten-line prompt leaves the model to guess your conventions. A thousand-line prompt trains it to ignore most of them, and you cannot tell which.

What has worked: short sections with headings (role, environment, how to work, when to stop), one or two concrete examples of a good final answer, and pointers to files for everything else. Put the stable parts first for caching. Put the task last. If you find yourself adding a rule to the prompt after every incident, the rule probably belongs in a hook or a tool error message instead, where it is enforced rather than suggested.

A note on Vietnamese#

Most of my agents read Vietnamese documents, and the budget arithmetic is worse than the English numbers above suggest. Vietnamese tokenises at roughly 1.5 to 2 times the tokens per word of English on current tokenisers, so a 30-page contract that would be 25,000 tokens in English is often 40,000 to 50,000 in Vietnamese. Everything in this post applies with the thresholds lowered, and the section-reading tools from technique 1 matter more, not less. Measure with count_tokens on your own documents before you set any limit; the per-character estimate in the code above is a placeholder.

What to measure#

You cannot engineer what you do not measure, and the trace from part 1 gives you everything you need: tokens per step, tokens per task, cache hit rate, the share of tokens that are tool results, and how many times the same file was read in one run. That last number was the one that embarrassed us. It is now a dashboard metric, and it is usually one.

References#

• Anthropic, Effective context engineering for AI agents (September 2025)
• Anthropic, Effective harnesses for long-running agents (November 2025)
• Anthropic, Prompt caching
• Chroma, Context Rot: How increasing input tokens impacts LLM performance (2025)