# AGENTS.md Guidance for AI coding agents working on this repository. Humans: start with [README.md](README.md) and [CONTRIBUTING.md](CONTRIBUTING.md); everything below links there instead of repeating it. Politrack tracks Austrian politicians from EU to municipal level (roles, votes, money, statements, meetings, social media, legal proceedings). Every fact is a claim backed by evidence links. Backend: Python (FastAPI, Postgres 16 + pgvector, Procrastinate workers, MCP server). Frontend: Next.js 15 with next-intl, German by default, English at `/en`. Deployed with Docker Compose on the owner's Unraid server. ## Before you start | Task | Read | |---|---| | Pick up open work | [docs/roadmap.md](docs/roadmap.md): *Next*, *Not yet verified*, *Known limitations* | | Add a data source | [docs/contributing/adapters.md](docs/contributing/adapters.md), legal checklist in §0 first | | Derive facts from stored data | [docs/contributing/enrichers.md](docs/contributing/enrichers.md) | | API endpoint or MCP tool | [docs/contributing/api.md](docs/contributing/api.md) | | UI page or new claim type in the UI | [docs/contributing/frontend.md](docs/contributing/frontend.md) | | How the UI should look and read | [docs/design/README.md](docs/design/README.md), [design-system.md](docs/design/design-system.md) | | Tests and fixtures | [docs/contributing/testing.md](docs/contributing/testing.md) | | Agent sandbox: setup, what it can reach | [docs/contributing/agents.md](docs/contributing/agents.md) | | Why it is built this way | [docs/architecture/02-comparison-and-decisions.md](docs/architecture/02-comparison-and-decisions.md) | | Tables, claim types, identifier schemes | [docs/data-model.md](docs/data-model.md) | | What each source allows | [docs/sources.md](docs/sources.md), [docs/legal.md](docs/legal.md) | ## Ground rules 1. **Legal access only.** Collect as much as possible, but only through official APIs, open data and pages whose terms and robots.txt allow it. Never use X proxies (Nitter and the like), paywall bypasses, logged-in scraping, or WhatsApp/Signal. Telegram: public channels via the `t.me/s` web preview only. A robots.txt exemption needs the publisher's documented permission, recorded in the adapter's `SourceSpec.tos_notes`; the RTR API is the only one so far. If a source can't be accessed cleanly, say so and propose legal alternatives instead of working around it. 2. **Every claim has evidence.** No claim without a document link. 3. **Fairness.** Neutral wording. Legal matters carry the procedural stage and "Es gilt die Unschuldsvermutung". Automatically detected legal claims stay in review (`review_status = pending`). Connections are not accusations. Personal data only in the person's public role. 4. **Not public yet.** The instance is private (LAN only) as of November 2026. Posts that disappear from X are marked, not erased, but their text must go (`social.x_deletions`, docs/legal.md). 5. **Commits:** only when the owner asks, as `Simon Diesenreiter `, not the global (work) address: `git -c user.email=simon@disi.dev commit …`, unless the repo-local config sets it. ## Invariants that are easy to break - **Four stages.** Adapters fetch only through `Request` objects handled by the framework's `Fetcher` (robots, throttling, TDM, archive), never with their own HTTP client. `parse()` is pure: no database, no network. Only the `Loader` writes claims. Enrichers derive from stored data. - **Claim identity.** Claims with a facet get their fingerprint from it. Claims without a facet need an explicit `key`, or duplicates and lost updates follow. - **Parser changes:** bump `parser_version`. Otherwise unchanged responses are skipped and the fix never applies. `politrack reparse ` rebuilds from the raw archive and deletes claims the new parser no longer emits. - **Golden fixtures:** new or changed parsers come with a fixture (`politrack fixture …`). Review the `.expected.json` diff; don't blindly regenerate with `UPDATE_GOLDEN=1`. - **Schema:** `0001_schema.sql` and `0002_seed.sql` are applied in existing databases. Change the schema only with a new numbered migration, with `COMMENT`s on new tables and columns. - **API types:** after changing a response model, regenerate the frontend types: `cd backend && politrack openapi --ts ../web/src/lib/api-types.ts`. - **UI text:** every new key goes into both `web/messages/de.json` and `en.json`. - **Backend dependencies:** a dependency added to or changed in `backend/pyproject.toml` also goes into the Debian package list in `backend/tests/Dockerfile.debian` (usually `python3-`). That list builds both the PyPI-free test image and the agent sandbox image; without the entry, tests there fail on the import. If Debian doesn't package it, add it to the exceptions in the file's header. ## Commands ```bash cd backend && pytest # unit + golden tests, no DB or network needed politrack sources list # adapters, enrichers, schedules, queues politrack run --max-requests 20 -p key=val # limit requests when testing against live sources politrack reparse # re-parse archived responses politrack enrich | status | review | migrate # enrichers, recent runs, legal review, schema ``` ## In the agent-envs sandbox You are in the sandbox if the repository is at `/workspace` and `HTTPS_PROXY` is `http://egress:8080` ([details](docs/contributing/agents.md)). There: - **Network:** only allowlisted hosts. HTTP 403 "Blocked by egress policy" means the host isn't allowed: tell the owner which host you need instead of working around it. - **Git:** pushing and Gitea API calls over HTTPS work without credentials; the proxy adds them. Don't set up tokens or credential helpers. The commit identity comes from the repository's config. - **Tests:** the sandbox image has the backend's dependencies from Debian packages, so `cd backend && python3 -m pytest -q` works without `pip install`. It comes from `backend/tests/Dockerfile.debian` (see "Backend dependencies" above). - **No Docker, no database.** Leave adapter runs against the database and the compose stack to the owner. The notes below on Git's `safe.directory` don't apply in the sandbox; the PyPI and npm block does when the session runs on the owner's PC. ## The owner's development machine These notes apply to the owner's Windows PC (Docker Desktop, Git Bash, PowerShell): - **Git** reports "dubious ownership" for this folder. Don't change the global git config; prefix git commands with `git -c safe.directory=C:/Users/disi0/source/repos/politics-tracking`. - **PyPI and npm** downloads failed there in October 2026 (a network filter blocks `files.pythonhosted.org` and `registry.npmjs.org`). Until that is fixed, run the tests with the Debian test image ([testing.md, "Without PyPI access"](docs/contributing/testing.md#without-pypi-access)). The real images, `npm install` and the workers can't be built or run there. - **Local database:** `docker compose up -d db` (port 5433, passwords in the gitignored `.env`, data in `data/`). It has the schema and sources but no scraped data yet, and no job-queue schema until the real `migrate` image runs. Postgres on a Windows bind mount is slow: the first start takes minutes. - **`DATA_DIR`, not `APPDATA`:** Windows already defines `APPDATA`, so the compose file uses `DATA_DIR`.