Document coding agents and give them a sandbox image
Agents work on this repository in the agent-envs sandbox: a container whose only way out is an egress proxy with an allowlist, which also adds the Gitea bot token. docs/contributing/agents.md covers setting it up for Politrack (1Password item, .env, commit identity), what it can reach and its limits; AGENTS.md tells agents how to recognise the sandbox and behave in it. backend/tests/Dockerfile.debian gains a `sandbox` target: the agent-envs agent image plus the backend's dependencies from Debian packages, the same list as the PyPI-free test image (still the default target). The sandbox runs `pytest` without PyPI, which the owner's PC can't reach. A new invariant in AGENTS.md keeps that list in step with pyproject.toml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
456f9816f4
commit
0e21730b9f
5 files changed
+117
-7
No files matched your search
@@ -19,6 +19,7 @@ next-intl, German by default, English at `/en`. Deployed with Docker Compose on
|
|||||||
| UI page or new claim type in the UI | [docs/contributing/frontend.md](docs/contributing/frontend.md) |
|
| UI page or new claim type in the UI | [docs/contributing/frontend.md](docs/contributing/frontend.md) |
|
||||||
| How the UI should look and read | [docs/design/README.md](docs/design/README.md), [design-system.md](docs/design/design-system.md) |
|
| How the UI should look and read | [docs/design/README.md](docs/design/README.md), [design-system.md](docs/design/design-system.md) |
|
||||||
| Tests and fixtures | [docs/contributing/testing.md](docs/contributing/testing.md) |
|
| Tests and fixtures | [docs/contributing/testing.md](docs/contributing/testing.md) |
|
||||||
|
| Agent sandbox: setup, what it can reach | [docs/contributing/agents.md](docs/contributing/agents.md) |
|
||||||
| Why it is built this way | [docs/architecture/02-comparison-and-decisions.md](docs/architecture/02-comparison-and-decisions.md) |
|
| Why it is built this way | [docs/architecture/02-comparison-and-decisions.md](docs/architecture/02-comparison-and-decisions.md) |
|
||||||
| Tables, claim types, identifier schemes | [docs/data-model.md](docs/data-model.md) |
|
| Tables, claim types, identifier schemes | [docs/data-model.md](docs/data-model.md) |
|
||||||
| What each source allows | [docs/sources.md](docs/sources.md), [docs/legal.md](docs/legal.md) |
|
| What each source allows | [docs/sources.md](docs/sources.md), [docs/legal.md](docs/legal.md) |
|
||||||
@@ -57,6 +58,10 @@ next-intl, German by default, English at `/en`. Deployed with Docker Compose on
|
|||||||
- **API types:** after changing a response model, regenerate the frontend types:
|
- **API types:** after changing a response model, regenerate the frontend types:
|
||||||
`cd backend && politrack openapi --ts ../web/src/lib/api-types.ts`.
|
`cd backend && politrack openapi --ts ../web/src/lib/api-types.ts`.
|
||||||
- **UI text:** every new key goes into both `web/messages/de.json` and `en.json`.
|
- **UI text:** every new key goes into both `web/messages/de.json` and `en.json`.
|
||||||
|
- **Backend dependencies:** a dependency added to or changed in `backend/pyproject.toml` also goes
|
||||||
|
into the Debian package list in `backend/tests/Dockerfile.debian` (usually `python3-<name>`). That
|
||||||
|
list builds both the PyPI-free test image and the agent sandbox image; without the entry, tests
|
||||||
|
there fail on the import. If Debian doesn't package it, add it to the exceptions in the file's header.
|
||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
|
|
||||||
@@ -68,6 +73,22 @@ politrack reparse <adapter> # re-parse archived respon
|
|||||||
politrack enrich | status | review | migrate # enrichers, recent runs, legal review, schema
|
politrack enrich | status | review | migrate # enrichers, recent runs, legal review, schema
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## In the agent-envs sandbox
|
||||||
|
|
||||||
|
You are in the sandbox if the repository is at `/workspace` and `HTTPS_PROXY` is `http://egress:8080`
|
||||||
|
([details](docs/contributing/agents.md)). There:
|
||||||
|
|
||||||
|
- **Network:** only allowlisted hosts. HTTP 403 "Blocked by egress policy" means the host isn't
|
||||||
|
allowed: tell the owner which host you need instead of working around it.
|
||||||
|
- **Git:** pushing and Gitea API calls over HTTPS work without credentials; the proxy adds them.
|
||||||
|
Don't set up tokens or credential helpers. The commit identity comes from the repository's config.
|
||||||
|
- **Tests:** the sandbox image has the backend's dependencies from Debian packages, so
|
||||||
|
`cd backend && python3 -m pytest -q` works without `pip install`. It comes from
|
||||||
|
`backend/tests/Dockerfile.debian` (see "Backend dependencies" above).
|
||||||
|
- **No Docker, no database.** Leave adapter runs against the database and the compose stack to the
|
||||||
|
owner. The notes below on Git's `safe.directory` don't apply in the sandbox; the PyPI and npm block
|
||||||
|
does when the session runs on the owner's PC.
|
||||||
|
|
||||||
## The owner's development machine
|
## The owner's development machine
|
||||||
|
|
||||||
These notes apply to the owner's Windows PC (Docker Desktop, Git Bash, PowerShell):
|
These notes apply to the owner's Windows PC (Docker Desktop, Git Bash, PowerShell):
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ Thanks for helping make Austrian politics more transparent. Three principles gui
|
|||||||
| add an API endpoint or MCP tool | [docs/contributing/api.md](docs/contributing/api.md) |
|
| add an API endpoint or MCP tool | [docs/contributing/api.md](docs/contributing/api.md) |
|
||||||
| add a page or show a new claim type in the UI | [docs/contributing/frontend.md](docs/contributing/frontend.md) |
|
| add a page or show a new claim type in the UI | [docs/contributing/frontend.md](docs/contributing/frontend.md) |
|
||||||
| run or write tests | [docs/contributing/testing.md](docs/contributing/testing.md) |
|
| run or write tests | [docs/contributing/testing.md](docs/contributing/testing.md) |
|
||||||
|
| work on this repository with a coding agent | [docs/contributing/agents.md](docs/contributing/agents.md) |
|
||||||
| understand the design | [docs/architecture/01-design-from-scratch.md](docs/architecture/01-design-from-scratch.md), [docs/data-model.md](docs/data-model.md) |
|
| understand the design | [docs/architecture/01-design-from-scratch.md](docs/architecture/01-design-from-scratch.md), [docs/data-model.md](docs/data-model.md) |
|
||||||
| add a social account or party channel | `backend/config/social_accounts.yaml` (comments explain the fields) |
|
| add a social account or party channel | `backend/config/social_accounts.yaml` (comments explain the fields) |
|
||||||
| add a news feed | `backend/config/feeds.yaml` (+ a `SourceSpec` in `sources/news.py`) |
|
| add a news feed | `backend/config/feeds.yaml` (+ a `SourceSpec` in `sources/news.py`) |
|
||||||
|
|||||||
@@ -1,11 +1,24 @@
|
|||||||
# Test image with the Python dependencies from Debian packages, for networks where PyPI is unreachable.
|
# The Python dependencies from Debian packages, for networks where PyPI is unreachable. Lacks
|
||||||
# Lacks procrastinate and mcp (not packaged in Debian), so it runs the test suite and adapters, not the
|
# procrastinate and mcp (not packaged in Debian), so it runs the test suite and adapters, not the
|
||||||
# workers or the MCP server. Usage: docs/contributing/testing.md ("Without PyPI access").
|
# workers or the MCP server; ruff isn't packaged either.
|
||||||
FROM debian:trixie-slim
|
#
|
||||||
RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends \
|
# Two images share this package list; keep it in step with backend/pyproject.toml:
|
||||||
|
# - the test image, the default target: docs/contributing/testing.md ("Without PyPI access");
|
||||||
|
# - the agent sandbox image, target `sandbox` with BASE set to the agent-envs agent image, which
|
||||||
|
# agent-envs builds from this checkout: docs/contributing/agents.md.
|
||||||
|
ARG BASE=debian:trixie-slim
|
||||||
|
FROM ${BASE} AS deps
|
||||||
|
# eatmydata skips dpkg's disk syncs, which crawl on some Docker Desktop disks.
|
||||||
|
RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends eatmydata >/dev/null \
|
||||||
|
&& eatmydata apt-get install -y -qq --no-install-recommends \
|
||||||
python3 python3-psycopg python3-psycopg-pool python3-pydantic-settings python3-fastapi python3-httpx \
|
python3 python3-psycopg python3-psycopg-pool python3-pydantic-settings python3-fastapi python3-httpx \
|
||||||
python3-protego python3-feedparser python3-bs4 python3-slugify python3-rapidfuzz python3-typer \
|
python3-protego python3-feedparser python3-bs4 python3-slugify python3-rapidfuzz python3-typer \
|
||||||
python3-yaml python3-uvicorn python3-pytest python3-openpyxl ca-certificates >/dev/null \
|
python3-yaml python3-uvicorn python3-pytest python3-openpyxl ca-certificates >/dev/null \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Keeps the agent image's user, entrypoint and /workspace as they are.
|
||||||
|
FROM deps AS sandbox
|
||||||
|
|
||||||
|
FROM deps AS test
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
ENV PYTHONPATH=/app PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1
|
ENV PYTHONPATH=/app PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
# Coding agents
|
||||||
|
|
||||||
|
Coding agents (Claude Code) work on this repository in a sandbox from
|
||||||
|
[agent-envs](https://git.disi.dev/AICode/agent-envs): a Linux container whose only way out is an
|
||||||
|
egress proxy with an allowlist. The proxy also adds the Gitea token to requests, so the agent never
|
||||||
|
holds it. This page covers setting that up for Politrack; [AGENTS.md](../../AGENTS.md) is what the
|
||||||
|
agent itself reads.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
1. Prepare the computer once, as described in agent-envs'
|
||||||
|
[docs/setup.md](https://git.disi.dev/AICode/agent-envs/src/branch/main/docs/setup.md): Docker, the
|
||||||
|
registry login, the 1Password CLI and, for VS Code, the Dev Containers settings.
|
||||||
|
2. In 1Password's `Agents` vault, create an API Credential item `gitea-politics-tracking` with the
|
||||||
|
`username` and an access token (`credential`) of a Gitea bot user. Make that user a collaborator
|
||||||
|
on `AIPlayground/politics-tracking` only, with write access. Protect `main` so the agent works
|
||||||
|
through pull requests.
|
||||||
|
3. In agent-envs, `cp projects/politics-tracking/.env.example projects/politics-tracking/.env` and set
|
||||||
|
`WORKSPACE` to this checkout.
|
||||||
|
4. Set the commit identity in this checkout, once. The sandbox uses the same `.git`, so its commits
|
||||||
|
get it too:
|
||||||
|
```bash
|
||||||
|
git config user.name "Simon Diesenreiter"
|
||||||
|
git config user.email simon@disi.dev
|
||||||
|
```
|
||||||
|
5. Start a session from anywhere: `agent politics-tracking` (Claude in the terminal) or
|
||||||
|
`agent politics-tracking code` (VS Code attached to the sandbox).
|
||||||
|
|
||||||
|
## What the sandbox can reach
|
||||||
|
|
||||||
|
| What | Hosts | For |
|
||||||
|
|---|---|---|
|
||||||
|
| Claude Code | `api.anthropic.com`, `claude.ai`, `platform.claude.com`, `console.anthropic.com` | the agent itself |
|
||||||
|
| Gitea | `git.disi.dev` (DELETE requests blocked) | push, pull requests through the API, as the bot user |
|
||||||
|
| Packages | `pypi.org`, `files.pythonhosted.org`, `registry.npmjs.org` | dependencies (see the limits below) |
|
||||||
|
| Sources | Parlament, data.gv.at, RTR, Rechnungshof, Justiz, Bundeskanzleramt, OTS, the European Parliament, data.europa.eu, Wikidata, German Wikipedia | capturing fixtures, short test runs |
|
||||||
|
| VS Code | the extension marketplace | installing the Claude Code extension in the sandbox |
|
||||||
|
|
||||||
|
The exact list is in agent-envs, `projects/politics-tracking/egress.yaml`. Media sites and social
|
||||||
|
networks are not included; add a host there when a task needs it, and only one that the source's
|
||||||
|
legal checklist ([adapters.md](adapters.md#0-before-writing-code-the-legal-checklist)) allows.
|
||||||
|
Everything else, and anything that isn't HTTP(S) (SSH, database connections), is blocked.
|
||||||
|
|
||||||
|
## The sandbox image
|
||||||
|
|
||||||
|
Politrack runs in its own sandbox image: the agent-envs agent image plus the backend's dependencies
|
||||||
|
as Debian packages. It is the `sandbox` target of
|
||||||
|
[backend/tests/Dockerfile.debian](../../backend/tests/Dockerfile.debian), the same package list as the
|
||||||
|
[PyPI-free test image](testing.md#without-pypi-access), so `cd backend && python3 -m pytest` works in
|
||||||
|
the sandbox without downloading anything. agent-envs builds it from this checkout at every start;
|
||||||
|
unchanged, that takes seconds from the cache.
|
||||||
|
|
||||||
|
Keep the list in step with `backend/pyproject.toml` (AGENTS.md says so too). Debian doesn't package
|
||||||
|
`procrastinate`, `mcp` and `ruff`, so the workers, the MCP server and linting don't run in the
|
||||||
|
sandbox.
|
||||||
|
|
||||||
|
## Limits
|
||||||
|
|
||||||
|
- **No Docker** in the sandbox: no compose stack.
|
||||||
|
- **No database:** the compose `db` on the host isn't reachable from the sandbox, so adapter runs that
|
||||||
|
write claims and [the database checks](testing.md#against-a-real-database) have to run outside it.
|
||||||
|
- **On the owner's PC**, PyPI and npm downloads fail inside the sandbox too (the same network filter;
|
||||||
|
see AGENTS.md): no `pip install` of new packages and no `npm install` for the frontend there.
|
||||||
|
- Claude's memory in the sandbox is separate from the one on the host. Rules that must always hold
|
||||||
|
belong in AGENTS.md, not in memory.
|
||||||
|
|
||||||
|
## Changing what the agent can reach or knows
|
||||||
|
|
||||||
|
All of it lives in agent-envs, not in this repository: hosts in `projects/politics-tracking/egress.yaml`,
|
||||||
|
kits (Gitea, Python, Node, VS Code) in `projects/politics-tracking/compose.yaml`, the token in
|
||||||
|
1Password. Instructions for the agent stay in this repository's AGENTS.md.
|
||||||
@@ -26,10 +26,14 @@ docker build -t politrack-test -f backend/tests/Dockerfile.debian backend/tests
|
|||||||
docker run --rm -v "$PWD/backend:/app" politrack-test python3 -m pytest -q -p no:cacheprovider
|
docker run --rm -v "$PWD/backend:/app" politrack-test python3 -m pytest -q -p no:cacheprovider
|
||||||
```
|
```
|
||||||
|
|
||||||
The same image can run adapters against the compose database: add `--network politics-tracking_default`
|
The test image can also run adapters against the compose database: add
|
||||||
and `-e POLITRACK_DATABASE_URL=postgresql://politrack:<password>@db:5432/politrack`, then
|
`--network politics-tracking_default` and
|
||||||
|
`-e POLITRACK_DATABASE_URL=postgresql://politrack:<password>@db:5432/politrack`, then
|
||||||
`python3 -m politrack.cli run <adapter> --max-requests 20`.
|
`python3 -m politrack.cli run <adapter> --max-requests 20`.
|
||||||
|
|
||||||
|
The same package list builds the agent sandbox image (target `sandbox`, see
|
||||||
|
[agents.md](agents.md#the-sandbox-image)), so keep it in step with `pyproject.toml`.
|
||||||
|
|
||||||
## Golden fixtures
|
## Golden fixtures
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|||||||
Reference in new issue
Block a user