From 0e21730b9fead7a936ff3d0b89b105c5f138a4e6 Mon Sep 17 00:00:00 2001 From: Simon Diesenreiter Date: Wed, 28 Oct 2026 23:59:01 +0100 Subject: [PATCH] Document coding agents and give them a sandbox image Agents work on this repository in the agent-envs sandbox: a container whose only way out is an egress proxy with an allowlist, which also adds the Gitea bot token. docs/contributing/agents.md covers setting it up for Politrack (1Password item, .env, commit identity), what it can reach and its limits; AGENTS.md tells agents how to recognise the sandbox and behave in it. backend/tests/Dockerfile.debian gains a `sandbox` target: the agent-envs agent image plus the backend's dependencies from Debian packages, the same list as the PyPI-free test image (still the default target). The sandbox runs `pytest` without PyPI, which the owner's PC can't reach. A new invariant in AGENTS.md keeps that list in step with pyproject.toml. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 21 ++++++++++ CONTRIBUTING.md | 1 + backend/tests/Dockerfile.debian | 23 ++++++++--- docs/contributing/agents.md | 71 +++++++++++++++++++++++++++++++++ docs/contributing/testing.md | 8 +++- 5 files changed, 117 insertions(+), 7 deletions(-) create mode 100644 docs/contributing/agents.md diff --git a/AGENTS.md b/AGENTS.md index 054974f..346c33e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -19,6 +19,7 @@ next-intl, German by default, English at `/en`. Deployed with Docker Compose on | UI page or new claim type in the UI | [docs/contributing/frontend.md](docs/contributing/frontend.md) | | How the UI should look and read | [docs/design/README.md](docs/design/README.md), [design-system.md](docs/design/design-system.md) | | Tests and fixtures | [docs/contributing/testing.md](docs/contributing/testing.md) | +| Agent sandbox: setup, what it can reach | [docs/contributing/agents.md](docs/contributing/agents.md) | | Why it is built this way | [docs/architecture/02-comparison-and-decisions.md](docs/architecture/02-comparison-and-decisions.md) | | Tables, claim types, identifier schemes | [docs/data-model.md](docs/data-model.md) | | What each source allows | [docs/sources.md](docs/sources.md), [docs/legal.md](docs/legal.md) | @@ -57,6 +58,10 @@ next-intl, German by default, English at `/en`. Deployed with Docker Compose on - **API types:** after changing a response model, regenerate the frontend types: `cd backend && politrack openapi --ts ../web/src/lib/api-types.ts`. - **UI text:** every new key goes into both `web/messages/de.json` and `en.json`. +- **Backend dependencies:** a dependency added to or changed in `backend/pyproject.toml` also goes + into the Debian package list in `backend/tests/Dockerfile.debian` (usually `python3-`). That + list builds both the PyPI-free test image and the agent sandbox image; without the entry, tests + there fail on the import. If Debian doesn't package it, add it to the exceptions in the file's header. ## Commands @@ -68,6 +73,22 @@ politrack reparse # re-parse archived respon politrack enrich | status | review | migrate # enrichers, recent runs, legal review, schema ``` +## In the agent-envs sandbox + +You are in the sandbox if the repository is at `/workspace` and `HTTPS_PROXY` is `http://egress:8080` +([details](docs/contributing/agents.md)). There: + +- **Network:** only allowlisted hosts. HTTP 403 "Blocked by egress policy" means the host isn't + allowed: tell the owner which host you need instead of working around it. +- **Git:** pushing and Gitea API calls over HTTPS work without credentials; the proxy adds them. + Don't set up tokens or credential helpers. The commit identity comes from the repository's config. +- **Tests:** the sandbox image has the backend's dependencies from Debian packages, so + `cd backend && python3 -m pytest -q` works without `pip install`. It comes from + `backend/tests/Dockerfile.debian` (see "Backend dependencies" above). +- **No Docker, no database.** Leave adapter runs against the database and the compose stack to the + owner. The notes below on Git's `safe.directory` don't apply in the sandbox; the PyPI and npm block + does when the session runs on the owner's PC. + ## The owner's development machine These notes apply to the owner's Windows PC (Docker Desktop, Git Bash, PowerShell): diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 34c1b13..16feacf 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -17,6 +17,7 @@ Thanks for helping make Austrian politics more transparent. Three principles gui | add an API endpoint or MCP tool | [docs/contributing/api.md](docs/contributing/api.md) | | add a page or show a new claim type in the UI | [docs/contributing/frontend.md](docs/contributing/frontend.md) | | run or write tests | [docs/contributing/testing.md](docs/contributing/testing.md) | +| work on this repository with a coding agent | [docs/contributing/agents.md](docs/contributing/agents.md) | | understand the design | [docs/architecture/01-design-from-scratch.md](docs/architecture/01-design-from-scratch.md), [docs/data-model.md](docs/data-model.md) | | add a social account or party channel | `backend/config/social_accounts.yaml` (comments explain the fields) | | add a news feed | `backend/config/feeds.yaml` (+ a `SourceSpec` in `sources/news.py`) | diff --git a/backend/tests/Dockerfile.debian b/backend/tests/Dockerfile.debian index ded7877..2db4f8b 100644 --- a/backend/tests/Dockerfile.debian +++ b/backend/tests/Dockerfile.debian @@ -1,11 +1,24 @@ -# Test image with the Python dependencies from Debian packages, for networks where PyPI is unreachable. -# Lacks procrastinate and mcp (not packaged in Debian), so it runs the test suite and adapters, not the -# workers or the MCP server. Usage: docs/contributing/testing.md ("Without PyPI access"). -FROM debian:trixie-slim -RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends \ +# The Python dependencies from Debian packages, for networks where PyPI is unreachable. Lacks +# procrastinate and mcp (not packaged in Debian), so it runs the test suite and adapters, not the +# workers or the MCP server; ruff isn't packaged either. +# +# Two images share this package list; keep it in step with backend/pyproject.toml: +# - the test image, the default target: docs/contributing/testing.md ("Without PyPI access"); +# - the agent sandbox image, target `sandbox` with BASE set to the agent-envs agent image, which +# agent-envs builds from this checkout: docs/contributing/agents.md. +ARG BASE=debian:trixie-slim +FROM ${BASE} AS deps +# eatmydata skips dpkg's disk syncs, which crawl on some Docker Desktop disks. +RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends eatmydata >/dev/null \ + && eatmydata apt-get install -y -qq --no-install-recommends \ python3 python3-psycopg python3-psycopg-pool python3-pydantic-settings python3-fastapi python3-httpx \ python3-protego python3-feedparser python3-bs4 python3-slugify python3-rapidfuzz python3-typer \ python3-yaml python3-uvicorn python3-pytest python3-openpyxl ca-certificates >/dev/null \ && rm -rf /var/lib/apt/lists/* + +# Keeps the agent image's user, entrypoint and /workspace as they are. +FROM deps AS sandbox + +FROM deps AS test WORKDIR /app ENV PYTHONPATH=/app PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1 diff --git a/docs/contributing/agents.md b/docs/contributing/agents.md new file mode 100644 index 0000000..324d60c --- /dev/null +++ b/docs/contributing/agents.md @@ -0,0 +1,71 @@ +# Coding agents + +Coding agents (Claude Code) work on this repository in a sandbox from +[agent-envs](https://git.disi.dev/AICode/agent-envs): a Linux container whose only way out is an +egress proxy with an allowlist. The proxy also adds the Gitea token to requests, so the agent never +holds it. This page covers setting that up for Politrack; [AGENTS.md](../../AGENTS.md) is what the +agent itself reads. + +## Setup + +1. Prepare the computer once, as described in agent-envs' + [docs/setup.md](https://git.disi.dev/AICode/agent-envs/src/branch/main/docs/setup.md): Docker, the + registry login, the 1Password CLI and, for VS Code, the Dev Containers settings. +2. In 1Password's `Agents` vault, create an API Credential item `gitea-politics-tracking` with the + `username` and an access token (`credential`) of a Gitea bot user. Make that user a collaborator + on `AIPlayground/politics-tracking` only, with write access. Protect `main` so the agent works + through pull requests. +3. In agent-envs, `cp projects/politics-tracking/.env.example projects/politics-tracking/.env` and set + `WORKSPACE` to this checkout. +4. Set the commit identity in this checkout, once. The sandbox uses the same `.git`, so its commits + get it too: + ```bash + git config user.name "Simon Diesenreiter" + git config user.email simon@disi.dev + ``` +5. Start a session from anywhere: `agent politics-tracking` (Claude in the terminal) or + `agent politics-tracking code` (VS Code attached to the sandbox). + +## What the sandbox can reach + +| What | Hosts | For | +|---|---|---| +| Claude Code | `api.anthropic.com`, `claude.ai`, `platform.claude.com`, `console.anthropic.com` | the agent itself | +| Gitea | `git.disi.dev` (DELETE requests blocked) | push, pull requests through the API, as the bot user | +| Packages | `pypi.org`, `files.pythonhosted.org`, `registry.npmjs.org` | dependencies (see the limits below) | +| Sources | Parlament, data.gv.at, RTR, Rechnungshof, Justiz, Bundeskanzleramt, OTS, the European Parliament, data.europa.eu, Wikidata, German Wikipedia | capturing fixtures, short test runs | +| VS Code | the extension marketplace | installing the Claude Code extension in the sandbox | + +The exact list is in agent-envs, `projects/politics-tracking/egress.yaml`. Media sites and social +networks are not included; add a host there when a task needs it, and only one that the source's +legal checklist ([adapters.md](adapters.md#0-before-writing-code-the-legal-checklist)) allows. +Everything else, and anything that isn't HTTP(S) (SSH, database connections), is blocked. + +## The sandbox image + +Politrack runs in its own sandbox image: the agent-envs agent image plus the backend's dependencies +as Debian packages. It is the `sandbox` target of +[backend/tests/Dockerfile.debian](../../backend/tests/Dockerfile.debian), the same package list as the +[PyPI-free test image](testing.md#without-pypi-access), so `cd backend && python3 -m pytest` works in +the sandbox without downloading anything. agent-envs builds it from this checkout at every start; +unchanged, that takes seconds from the cache. + +Keep the list in step with `backend/pyproject.toml` (AGENTS.md says so too). Debian doesn't package +`procrastinate`, `mcp` and `ruff`, so the workers, the MCP server and linting don't run in the +sandbox. + +## Limits + +- **No Docker** in the sandbox: no compose stack. +- **No database:** the compose `db` on the host isn't reachable from the sandbox, so adapter runs that + write claims and [the database checks](testing.md#against-a-real-database) have to run outside it. +- **On the owner's PC**, PyPI and npm downloads fail inside the sandbox too (the same network filter; + see AGENTS.md): no `pip install` of new packages and no `npm install` for the frontend there. +- Claude's memory in the sandbox is separate from the one on the host. Rules that must always hold + belong in AGENTS.md, not in memory. + +## Changing what the agent can reach or knows + +All of it lives in agent-envs, not in this repository: hosts in `projects/politics-tracking/egress.yaml`, +kits (Gitea, Python, Node, VS Code) in `projects/politics-tracking/compose.yaml`, the token in +1Password. Instructions for the agent stay in this repository's AGENTS.md. diff --git a/docs/contributing/testing.md b/docs/contributing/testing.md index bad84c9..b65d7e8 100644 --- a/docs/contributing/testing.md +++ b/docs/contributing/testing.md @@ -26,10 +26,14 @@ docker build -t politrack-test -f backend/tests/Dockerfile.debian backend/tests docker run --rm -v "$PWD/backend:/app" politrack-test python3 -m pytest -q -p no:cacheprovider ``` -The same image can run adapters against the compose database: add `--network politics-tracking_default` -and `-e POLITRACK_DATABASE_URL=postgresql://politrack:@db:5432/politrack`, then +The test image can also run adapters against the compose database: add +`--network politics-tracking_default` and +`-e POLITRACK_DATABASE_URL=postgresql://politrack:@db:5432/politrack`, then `python3 -m politrack.cli run --max-requests 20`. +The same package list builds the agent sandbox image (target `sandbox`, see +[agents.md](agents.md#the-sandbox-image)), so keep it in step with `pyproject.toml`. + ## Golden fixtures ```