Here is the content with Markdown formatting preserved:

# Codex’s autonomous coding sessions vs. typical LLM‑driven coding

## 1 Overview and context

Most generative coding tools act as **conversational assistants**: the user describes a change, the model emits a snippet, the human runs it locally, then the user iterates with additional chat turns until the code works.  These tools rely on the language model’s internal reasoning, sometimes enhanced by reinforcement‑learning methods like **Group Relative Policy Optimization (GRPO)**.  GRPO trains the model to produce **chain‑of‑thought outputs** by sampling multiple responses and optimizing them via RL; it improves internal reasoning but does not give the model access to an execution environment.  Tools based purely on the model (e.g., ChatGPT or other LLMs with GRPO) therefore require multiple interactive rounds because the model cannot **run tests**, modify files or persist state across requests.

OpenAI’s **Codex** takes a different approach.  It is **not just a model**; it is a **cloud‑based software‑engineering agent** with its own container, terminal and filesystem.  When a user assigns a task through ChatGPT or the `codex` CLI, the agent spins up an isolated environment preloaded with the user’s repository.  Within this environment the agent, powered by the RL‑tuned `codex‑1` model, can read and edit files, execute shell commands, run test suites, and commit changes.  The agent returns a complete **pull request (PR)** rather than a single code snippet.  Tasks typically run autonomously for **1–30 minutes** and can run in **parallel**.  This asynchronous, delegated workflow is why Codex can handle long‑running coding tasks that would normally require multiple chat turns.

## 2 Architecture of Codex

### 2.1 Agent–model separation

* **Model component (codex‑1)** – `codex‑1` is a variant of OpenAI’s `o3` model fine‑tuned using reinforcement learning on real software engineering tasks.  The training emphasised professional coding practices—matching code style, writing proper commit messages, and producing passing tests.  Reinforcement learning (similar to GRPO) is used to align the model to human preferences, but the model does not directly execute commands.
* **Agent orchestrator** – The Codex agent wraps the model in a **controller** that manages state and tool calls.  It maintains a context about the repository, uses the model to decide the next action, issues shell commands (e.g., `grep`, `sed`, `git`), interprets outputs, and persists state across steps.  Because the agent is separate from the model, it can handle long sessions, pause/resume tasks and assemble multiple model inferences into a coherent workflow.
* **Sandboxed environment** – Each task runs in its own **container**.  The environment is configured via an **environment configuration screen** that allows the user to specify the container image, environment variables, secrets and a startup script.  After the startup script finishes, **internet access is disabled**, so the agent cannot reach external websites or leak secrets.  This container is identical between training and production to avoid “works on my machine” issues.
* **Logging and evidence** – The agent records all actions and provides a log view of step‑summaries and executed commands.  When the task finishes, Codex commits the changes and returns citations of test outputs, enabling users to inspect each step.

### 2.2 Execution loop

A Codex task proceeds through the following stages:

1. **Context gathering** – The agent reads `AGENTS.md`, `README.md` and other documentation in the repository to understand instructions.  It can use simple tools like `grep` to search for relevant code; this brute‑force text search was observed during a Codex run.  The `AGENTS.md` file acts as a “prompt” telling Codex how to navigate the codebase and how to run tests.
2. **Planning and editing** – Using the model’s reasoning, the agent plans modifications and edits code.  It opens files, inserts imports, modifies functions, creates tests and runs linters.  Because the environment is stateful, these changes persist across steps.
3. **Testing and verification** – Codex runs test harnesses (`pytest`, `yarn test`, etc.) inside the container to verify its changes.  If tests fail, it revises the code.  Codex also runs linters and formatters like Prettier or Ruff as defined in `AGENTS.md`.  If the environment lacks dependencies, the agent may fail—emphasising the importance of a correct dev environment.
4. **Commit and PR creation** – When tests pass or the agent runs out of time, it commits the changes, writes a PR description and returns a summary.  The agent includes citations to logs so the user can trace how decisions were made.

### 2.3 Long‑running behaviour

Because the agent controls the environment, it can run for tens of minutes without human intervention.  Alexander Embiricos of OpenAI explained that Codex is “a coding agent that has its **own container and its own terminal**, fully in the cloud,” and you “give it a task, and it comes back to you with a PR in this **one‑shot style**”.  This design allows asynchronous delegation—developers submit tasks and review results later, rather than engaging in a synchronous chat loop.  The Sequoia interview also notes that **long‑running sessions require new interaction patterns**: agents often start by generating a plan for user approval and must have graceful failure modes because tasks sometimes exceed allotted time.

## 3 How Codex differs from typical LLM coding workflows

| Aspect                                 | Codex (agentic)                                                                                                                                                                            | Typical LLM coding (chat/GRPO‑only)                                                                                                                                                    |
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Stateful environment**               | Each task runs in its own container with a filesystem and shell; the agent can read/edit files and run tests.                                                                              | No persistent environment; the model returns code snippets and relies on the user to run them.                                                                                         |
| **Tool usage**                         | Agent issues commands (grep, git, test runners).  Tool outputs inform subsequent reasoning.                                                                                                | Tools (if available) are often hidden behind a single API call (e.g., Python sandbox) and require manual invocation.                                                                   |
| **Autonomy and duration**              | Tasks run autonomously for 1–30 min and can be parallelised.  The agent plans, iterates and produces a PR.                                                                                 | Single model calls return immediate responses; longer tasks require multiple user‑prompted turns.                                                                                      |
| **User interaction pattern**           | Delegation: the user specifies a task once and reviews the result.  Logs provide transparency.                                                                                             | Pair‑programming: the user and model engage in a back‑and‑forth conversation, adjusting code manually.                                                                                 |
| **Training and alignment**             | Reinforcement learning on real codebases and test suites to align with professional style and PR preferences.  The model learns to run tests and produce mergeable code.                   | Models with GRPO or other reasoning algorithms learn to produce detailed chain‑of‑thought but do not learn how to interact with a filesystem; they depend on chat prompts for context. |
| **Reliability and environment design** | Requires careful environment configuration—container images, secrets, and startup scripts must match the project.  Training and production environments are identical to avoid mismatches. | No environment to configure; reliability depends on the user’s own setup and manual testing.                                                                                           |
| **Session resumption**                 | Early versions lacked session resumption and background jobs; stepping away could cause lost context.                                                                                      | Chat sessions persist across turns; the user can continue later, but the model does not resume execution; context length limits still apply.                                           |

## 4 Relation to GRPO and other reasoning techniques

**Group Relative Policy Optimization (GRPO)** is a reinforcement‑learning algorithm that improves an LLM’s ability to reason through problems.  It eliminates the need for a value function by computing baselines from groups of sampled answers.  GRPO fine‑tuning encourages models to generate **self‑corrective chain‑of‑thought** and has been used to enhance reasoning on math problems.  For example, a tutorial on building a reasoning LLM with GRPO describes how the model is prompted to “think step by step,” generates multiple candidate responses, assigns deterministic rewards, and updates the model using GRPO to improve reasoning ability.  These techniques improve **internal reasoning**, but they do not give the model access to an execution environment or persistent state.

Codex uses **agentic RL training** akin to GRPO but goes further by integrating tool use and environment interaction.  The `codex‑1` model learned to run tests, adhere to coding styles and generate professional PRs.  However, Codex still separates **thinking** (performed by the model) from **acting** (handled by the agent controller).  The model’s outputs are used as **plans** and tool invocations, while the agent executes commands and reads back results.  This separation allows long‑running sessions: if the model misinterprets a state, the agent can adjust the plan by feeding new observations back into the model, instead of requiring the user to re‑prompt.

## 5 Broader implications and limitations

* The **remote dev environment** is both a strength and a limitation.  It enables delegated work, but tasks can fail if the environment lacks dependencies or if the sandbox lacks network access.  Configuring container images and startup scripts is critical for reproducibility.
* **Simple search strategies** – Observations of Codex’s logs show that it often uses crude text search (`grep`) to locate relevant files.  This simplicity may explain why Codex sometimes misses reusable code or fails to discover hidden tests.
* **User experience gaps** – An external evaluation comparing Codex with other agents noted that Codex’s sandbox lacked session resumption and background jobs; stepping away meant losing context.  Competing agents like Claude Code offer features like background tasks and checkpoint reverts.  Codex is cheaper but may require tuning and improved UX to match its competitors.
* **Planning and failure modes** – Long‑running tasks require planning phases and graceful failure handling.  Codex sometimes gives up when tasks are too complex, echoing human‑like limitations; effective use therefore involves breaking tasks into manageable chunks or letting Codex make multiple attempts.

## 6 Summary

Codex represents a shift from chat‑based code suggestion to **agentic software engineering**.  By delegating work to a model‑powered agent running in an isolated environment, it can autonomously read code, run tests, make changes and produce pull requests.  Reinforcement‑learning alignment ensures the generated code matches professional standards.  The separation between the **thinking model** and the **acting agent** allows long‑running, asynchronous sessions—a design pattern that differs fundamentally from typical LLM tools that rely solely on chain‑of‑thought reasoning and user‑driven iteration.
