Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 26 additions & 75 deletions BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,89 +74,40 @@ Left over:

## Medium priority

### 13. Resuming a transcript that ends mid-turn sends an invalid request
Items 13, 15, 16, 17 and 18 landed on `feature/medium-priority` (PR pending). Item 14 was checked live
(below). Left over:

**Status:** gap. `Agent.converse` / `run_conversation` send `transcript.messages` as they
are and never call `check_invariants()`. `from_jsonl` deliberately loads mid-turn logs, so
this is a supported state.
### 13. Resuming a transcript that ends mid-turn: the paused case

**Concrete triggers:**
- The process dies while tools run, and the saved transcript ends with an assistant message
whose tool calls have no results. Resuming sends unanswered calls, every provider rejects
them with a 400, and that `BadRequestError` is not retried.
- A turn paused (`StopReason.PAUSED`) and then hit a provider error. The new `_Run` starts
with `continuing=False`, so the continuation becomes a second consecutive assistant
message instead of extending the first.
Unanswered tool calls are now run before the first model call, and `check_invariants()` runs
once up front. Still open: a turn that paused (`StopReason.PAUSED`) and then hit a provider
error. The new `_Run` starts with `continuing=False`, so the continuation becomes a second
consecutive assistant message instead of extending the first. `Message` would have to record
that the turn is open, for example in `Origin`.

**What implementing it takes:** a preflight in `drive()`. Pending tool calls should be
either executed before the first model call (a natural resume) or rejected with a
`TranscriptError` that says how to fix it. That is an API decision. For the paused case,
`Message` would have to record that the turn is open, for example in `Origin`.
### 14. Responses reasoning replay without `include` (checked live, 2026-10-18)

### 14. Verify Responses reasoning replay without `include: ["reasoning.encrypted_content"]`
Confirmed for `o4-mini`: a tool-using turn with `store: false` and no `include` returned a
`reasoning` item with a non-empty `encrypted_content`, and replaying the transcript with it was
accepted. The `_openai_responses.py` docstring is right for that model; no `include` is needed.
Not covered: `gpt-5.5` (the `.env` model) produced no reasoning item at its default effort
(0 reasoning tokens), so it did not exercise replay. Recheck if a request option for reasoning
effort is added. Item 21's fixture for a Responses reasoning item is still unrecorded.

**Status:** needs a live check. The docstring in `_openai_responses.py` says the encrypted
reasoning content "`store: false` returns by default". OpenAI's API reference documents it
as an opt-in `include` value for stateless use. If it is opt-in, reasoning items are
replayed without their encrypted content. Reasoning models then lose their chain of
thought across tool calls, or the request is rejected.
### 17. Model-aware provider `Quirks`: what is left

**What implementing it takes:** run one tool-using turn on a reasoning model live and check
`output[*].encrypted_content`. If it is missing, add `"reasoning.encrypted_content"` to
`include` whenever the model may reason, and save the captured body as a fixture (item 21).
`providers/openai.py` now picks `Quirks` by model-id prefix (`MODEL_QUIRKS`, `quirks_for_model`),
and `build_openai_client(quirks=...)` overrides it. The table holds only the o-series
(`o1`, `o3`, `o4`). Not done: other OpenAI families that reject parameters (add rows as they
are confirmed live), and OpenRouter and Modal, which stay provider-wide because their models
vary too much for one table.

### 15. No way to mark a tool as unsafe to run in parallel
### 18. Tracing setup warnings: what is left

**Status:** not implemented. `run_tools` always runs a turn with two or more tool calls in
parallel, on a fixed `ThreadPoolExecutor(max_workers=8)` (`tooling.py`). There is no
`ToolSpec` flag for a handler that isn't thread-safe: a shared DB cursor, a non-atomic
shared counter, an SDK client not documented as thread-safe.

**Concrete trigger:** two tools that both mutate the same plain `list` or
`sqlite3.Connection` without a lock, called by the model in one turn. Corruption or
exceptions show up only intermittently, when a model happens to call them together.

**What implementing it takes:** a `ToolSpec.parallel_safe: bool = True` (or similar) that
`run_tools` checks. Any turn that contains an unsafe call runs sequentially.

### 16. No per-provider timeout override through `Settings` / `build_client()`

**Status:** not implemented. Timeouts are hard-coded per adapter (`anthropic.py`'s
`DEFAULT_TIMEOUT_SECONDS`, the `TIMEOUT_SECONDS` dicts in `openrouter.py`, `modal.py` and
`openai.py`). Someone who needs a longer read timeout must call `build_<provider>_client()`
directly and give up `Settings.from_env()`. Item 2's `http_client` does not cover the
Anthropic path, and its timeout heuristic is fragile (item 23).

**What implementing it takes:** an optional `timeout_seconds` (or a richer shape) on
`Settings`, passed through each `_build_*` helper in `config.py` to the provider builder's
`timeout=`.

### 17. Model-aware provider `Quirks`

**Status:** not implemented. `Quirks` is chosen once per *provider*, not per *model*.
`providers/openai.py` sends `parallel_tool_calls` on every request. OpenAI's reasoning
models reject that parameter and its non-reasoning chat models accept it, and the only way
around it is two hand-built clients.

**What implementing it takes:** a second selection axis, probably a small `model -> Quirks`
override table (regex or prefix match) that `build_openai_client(model=...)` checks before
falling back to the provider default. It needs real design work: what to match on, what
happens when nothing matches, and how Modal's provider-wide quirks interact.

### 18. Tracing setup fails silently (`configure_tracing()` twice; `trace=True` without it)

**Status:** not implemented. Two symptoms with one cause:

- OpenTelemetry's global `trace.set_tracer_provider()` takes effect only once per process.
A second `configure_tracing()` returns a provider that looks live but receives no spans.
The real first provider may then never be flushed.
- `Agent(trace=True)` without `configure_tracing()` throws every span away, and nothing
tells the user.

**What implementing it takes:** detect reliably whether the global provider is still the
default proxy or no-op one. Then warn (or raise) on a second install, and log one warning
the first time `OTelObserver.conversation()` runs with no provider installed. The
detection has to be tested across the OpenTelemetry SDK versions the package supports.
A second `configure_tracing()` and a traced run with no provider now log a warning. It warns
rather than raises, and the check uses `ProxyTracerProvider` / `NoOpTracerProvider` from the
OpenTelemetry API. It is untested against older SDK versions than the one in the dev
environment.

---

Expand Down
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,23 @@ while `0.x`, a minor bump may break the public API (`agentloop.__all__`).
- `OTelObserver(redact=..., allow_unredacted=...)`, with the event trace's rules.
- `ConversationStatus.TOKEN_LIMIT` and `LoopBudgetExceeded(kind="max_tokens")`.

- `ToolSpec.parallel_safe` (and `@tool(parallel_safe=...)`): `False` makes a turn that calls
the tool run all its calls sequentially, in call order.
- `Settings.timeout_seconds` / `LLM_TIMEOUT_SECONDS`, and `timeout_seconds=` on
`build_openai_client`, `build_openrouter_client` and `build_modal_client`: replaces the read
timeout of the OpenAI-compatible providers (Anthropic: the SDK's overall timeout). An explicit
timeout now wins over a supplied `http_client`'s, so the "is it 5 seconds?" guess only applies
when none was asked for.
- `build_openai_client(quirks=...)` and `providers.openai.MODEL_QUIRKS` / `quirks_for_model`: the
o-series reasoning models no longer receive `parallel_tool_calls`, which they reject.

### Changed
- Resuming a transcript that ends in unanswered tool calls (the process died while tools ran)
runs those calls before the first model call instead of sending an invalid request. Any other
invalid transcript raises `TranscriptError` before a model call, not a provider 400.
- `configure_tracing()` logs a warning when a provider is already installed (it cannot replace
it, so the one it returns receives no spans), and a traced conversation logs one warning when
no provider is installed at all.
- `OTelObserver(capture_content=True)` now needs `redact` or `allow_unredacted=True`, else it
raises `ConfigurationError`. Captured text is redacted, then clipped; a redactor that raises
drops the attribute. Exceptions are recorded as an `exception` event with a redacted message
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,7 @@ and the rest of the package.
|---|---|---|
| `LLM_PROVIDER` | `anthropic` | `anthropic`, `openai`, `openrouter`, or `modal` |
| `LLM_MAX_ATTEMPTS` | `3` | Total attempts, including the first request |
| `LLM_TIMEOUT_SECONDS` | provider default | Read timeout for one model call (Anthropic: the SDK's overall timeout) |
| `ANTHROPIC_API_KEY` | unset | Anthropic credential |
| `ANTHROPIC_MODEL` | `claude-opus-5` | Anthropic model |
| `OPENAI_API_KEY` | unset | OpenAI credential |
Expand Down
1 change: 1 addition & 0 deletions docs/api/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ A frozen Pydantic model. Unknown fields are rejected.
| `openai_model` | `str` | `"gpt-5.1"` |
| `openai_api` | `"responses" | "completions"` | `"responses"` |
| `max_attempts` | `int >= 1` | `3` |
| `timeout_seconds` | `float > 0 \| None` | `None` (provider default) |

### `Settings.model`

Expand Down
4 changes: 4 additions & 0 deletions docs/api/loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,10 @@ A provider that pauses a turn (`StopReason.PAUSED`, for example Anthropic's `pau
a hosted tool runs) is continued without sending a user message: the continuation joins the
same assistant message, and each continuation counts as an iteration.

If `transcript` ends in tool calls with no results (a log cut off while tools ran), the loop
runs them before the first model call and then validates the transcript with
`check_invariants()`, so a bad one raises `TranscriptError` instead of a provider 400.

The deadline is checked between turns. A provider call, its retries, or a started tool batch
may finish after it. `None` disables the deadline. Exhaustion raises `LoopBudgetExceeded`;
provider errors propagate unchanged.
Expand Down
1 change: 1 addition & 0 deletions docs/api/tooling.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ A frozen Pydantic model containing the model-visible definition and handler.
| `description` | non-empty `str` | Model-visible instructions |
| `parameters` | `dict[str, Any]` | Bare handwritten JSON Schema |
| `handler` | `Handler` | Callable whose first argument is the deps bundle |
| `parallel_safe` | `bool` | `True`. Set `False` for a handler that is not thread-safe: a turn that calls it runs all of its tool calls one after another, in call order |
| `timeout_seconds` | `float \| None` | Per-call limit; `None` waits forever. On expiry the model gets an error result and the handler thread is abandoned (Python cannot kill it), so the handler should still set its own network timeouts |

```python
Expand Down
1 change: 1 addition & 0 deletions docs/getting-started/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ finally:
|---|---|---|
| `LLM_PROVIDER` | `anthropic` | `anthropic`, `openai`, `openrouter`, or `modal` |
| `LLM_MAX_ATTEMPTS` | `3` | Total attempts, including the first request |
| `LLM_TIMEOUT_SECONDS` | provider default | Read timeout for one model call (Anthropic: the SDK's overall timeout) |
| `ANTHROPIC_API_KEY` | unset | Anthropic credential |
| `ANTHROPIC_MODEL` | `claude-opus-5` | Anthropic model |
| `OPENAI_API_KEY` | unset | OpenAI credential |
Expand Down
4 changes: 4 additions & 0 deletions docs/guides/tracing.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ Tracing creates one top-level span for the conversation, one child span for each
request, and one span for each local tool call. A span is one timed operation in an
OpenTelemetry trace. Retries appear as events on the related model-request span.

Tracing needs a provider installed first. A traced conversation with none logs one warning
and discards its spans; calling `configure_tracing()` a second time logs a warning too, because
OpenTelemetry keeps the first provider and the returned one receives nothing.

Prompts, tool arguments, tool results, and provider error messages are not captured by
default. Construct an
`OTelObserver(..., capture_content=True, redact=scrub)` (a redactor is required, as for the event
Expand Down
3 changes: 3 additions & 0 deletions src/agentloop/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ def tool(
name: str | None = None,
description: str | None = None,
timeout_seconds: float | None = None,
parallel_safe: bool = True,
) -> Callable[[Callable[..., Any]], ToolSpec]:
"""Turn a plain function into a ToolSpec.

Expand All @@ -88,6 +89,7 @@ def tool(

`timeout_seconds` bounds one call: past it the model is told the tool timed out and
the loop moves on. The handler's thread is abandoned, not killed (see `ToolSpec`).
`parallel_safe=False` makes any turn that calls this tool run its calls sequentially.
"""

def decorate(fn: Callable[..., Any]) -> ToolSpec:
Expand All @@ -104,6 +106,7 @@ def decorate(fn: Callable[..., Any]) -> ToolSpec:
parameters=parameters,
handler=_make_handler(fn),
timeout_seconds=timeout_seconds,
parallel_safe=parallel_safe,
)

return decorate
Expand Down
11 changes: 11 additions & 0 deletions src/agentloop/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,11 @@ class Settings(BaseModel):
# every provider -- the Anthropic SDK's own retries are switched off.
max_attempts: int = Field(default=3, ge=1)

# Seconds to wait on one model call. None keeps each provider's own default. For the
# OpenAI-compatible providers it replaces the READ timeout (connect, write and pool
# keep theirs); for Anthropic it is the SDK's overall request timeout.
timeout_seconds: float | None = Field(default=None, gt=0)

@field_validator(
"anthropic_api_key",
"openrouter_api_key",
Expand Down Expand Up @@ -121,6 +126,7 @@ def from_env(cls, env: Mapping[str, str]) -> Settings:
# The raw string: pydantic coerces it, and rejects "three" with a
# ValidationError that names the field, where int() would not.
max_attempts=env.get("LLM_MAX_ATTEMPTS", "3"), # type: ignore[arg-type]
timeout_seconds=env.get("LLM_TIMEOUT_SECONDS") or None, # type: ignore[arg-type]
)


Expand Down Expand Up @@ -163,7 +169,9 @@ def _build_anthropic(settings: Settings, http_client: Any) -> LLMClient:
from agentloop.providers.anthropic import AnthropicClient # lazy: keeps the extra optional

key = settings.anthropic_api_key
timeout = {} if settings.timeout_seconds is None else {"timeout": settings.timeout_seconds}
return AnthropicClient(
**timeout,
# None lets the SDK resolve credentials itself.
api_key=key.get_secret_value() if key else None,
model=settings.anthropic_model,
Expand All @@ -180,6 +188,7 @@ def _build_openrouter(settings: Settings, http_client: Any) -> LLMClient:
api_key=settings.openrouter_api_key.get_secret_value(),
model=settings.openrouter_model,
http_client=http_client,
timeout_seconds=settings.timeout_seconds,
)


Expand All @@ -203,6 +212,7 @@ def _build_modal(settings: Settings, http_client: Any) -> LLMClient:
proxy_token_id=token_id.get_secret_value() if token_id else "",
proxy_token_secret=token_secret.get_secret_value() if token_secret else "",
http_client=http_client,
timeout_seconds=settings.timeout_seconds,
)


Expand All @@ -216,4 +226,5 @@ def _build_openai(settings: Settings, http_client: Any) -> LLMClient:
model=settings.openai_model,
api=settings.openai_api,
http_client=http_client,
timeout_seconds=settings.timeout_seconds,
)
18 changes: 18 additions & 0 deletions src/agentloop/loop.py
Original file line number Diff line number Diff line change
Expand Up @@ -361,9 +361,27 @@ def _add_turn(
)
)

def _resume_pending_calls(self) -> None:
"""Answer tool calls a loaded transcript left unanswered, before any model call.

`from_jsonl` deliberately loads a log cut off mid-turn (the process died while
tools ran). Sending that as it stands is a 400 on every provider, which is not
retried. Running the calls is the natural resume; anything else wrong with the
transcript is raised here, with the fix, rather than as an opaque provider error.
"""
pending = self.transcript.pending_tool_calls()
if pending:
logger.warning("resuming: running %d tool call(s) left unanswered", len(pending))
started = time.monotonic()
results = run_tools(pending, self.registry, self.deps, self.watcher)
self.tool_seconds += time.monotonic() - started
self.transcript.add_tool_results(results)
self.transcript.check_invariants()

def drive(self) -> ConversationStatus:
"""Loop until the model stops asking for tools; raise if it will not."""
transcript, watcher = self.transcript, self.watcher
self._resume_pending_calls()

for _ in range(self.max_iterations):
elapsed = time.monotonic() - self.started
Expand Down
18 changes: 16 additions & 2 deletions src/agentloop/providers/_openai_compat.py
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,18 @@ def httpx_timeout(**kwargs: float) -> httpx.Timeout:
return _httpx().Timeout(**kwargs) # type: ignore[no-any-return]


def timeout_with_read(defaults: Mapping[str, float], read_seconds: float | None) -> httpx.Timeout:
"""`defaults` as an `httpx.Timeout`, with the read timeout replaced when one is given.

Read is the one that matters for a model: the gap between bytes of a response that
is still being generated. Connect, write and pool keep the provider's own values.
"""
values = dict(defaults)
if read_seconds is not None:
values["read"] = read_seconds
return httpx_timeout(**values)


class Quirks(BaseModel):
"""Behavioural differences between OpenAI-compatible servers.

Expand Down Expand Up @@ -558,12 +570,14 @@ def __init__(
timeout: httpx.Timeout,
api: Api = "completions",
http_client: httpx.Client | None = None,
timeout_explicit: bool = False,
) -> None:
"""`http_client` is the escape hatch for trust, proxy and transport: build an
`httpx.Client` with the `verify`, `proxy` or `transport` you need and pass it in.
It stays yours to close. The URL and auth headers are sent per request, so the
client needs no `base_url` or default headers; `timeout` is applied per request
too, unless the client was given a timeout of its own.
too, unless the client was given a timeout of its own -- or `timeout_explicit` says
the caller asked for this `timeout`, which then wins over the client's.
"""
self.name = name
self.model = model
Expand All @@ -588,7 +602,7 @@ def __init__(
self._http = http_client
# httpx's own default means "the caller did not choose one".
chose_own = http_client.timeout != self._httpx.Timeout(5.0)
self._timeout = None if chose_own else timeout
self._timeout = None if chose_own and not timeout_explicit else timeout

@property
def api(self) -> Api:
Expand Down
6 changes: 4 additions & 2 deletions src/agentloop/providers/modal.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
if TYPE_CHECKING: # pragma: no cover - type checkers only, never imported at runtime
import httpx

from agentloop.providers._openai_compat import OpenAICompatClient, Quirks, httpx_timeout
from agentloop.providers._openai_compat import OpenAICompatClient, Quirks, timeout_with_read

DEFAULT_MODEL = "Qwen/Qwen3.6-35B-A3B-FP8"
# Cold starts spin up a container and load weights: minutes, not seconds.
Expand All @@ -27,6 +27,7 @@ def build_modal_client(
proxy_token_id: str = "",
proxy_token_secret: str = "",
http_client: httpx.Client | None = None,
timeout_seconds: float | None = None,
) -> OpenAICompatClient:
"""Proxy tokens are both-or-neither; empty means an --unauthenticated endpoint."""
headers: dict[str, str] = {}
Expand All @@ -38,6 +39,7 @@ def build_modal_client(
base_url=f"{endpoint_url.rstrip('/')}/v1",
headers=headers,
quirks=QUIRKS,
timeout=httpx_timeout(**TIMEOUT_SECONDS),
timeout=timeout_with_read(TIMEOUT_SECONDS, timeout_seconds),
timeout_explicit=timeout_seconds is not None,
http_client=http_client,
)
Loading
Loading