politics-tracking/AGENTS.md

AGENTS.md

Guidance for AI coding agents working on this repository. Humans: start with README.md and CONTRIBUTING.md; everything below links there instead of repeating it.

Politrack tracks Austrian politicians from EU to municipal level (roles, votes, money, statements, meetings, social media, legal proceedings). Every fact is a claim backed by evidence links. Backend: Python (FastAPI, Postgres 16 + pgvector, Procrastinate workers, MCP server). Frontend: Next.js 15 with next-intl, German by default, English at /en. Deployed with Docker Compose on the owner's Unraid server.

Before you start

Task	Read
Pick up open work	docs/roadmap.md: Next, Not yet verified, Known limitations
Add a data source	docs/contributing/adapters.md, legal checklist in §0 first
Derive facts from stored data	docs/contributing/enrichers.md
API endpoint or MCP tool	docs/contributing/api.md
UI page or new claim type in the UI	docs/contributing/frontend.md
How the UI should look and read	docs/design/README.md, design-system.md
Tests and fixtures	docs/contributing/testing.md
Agent sandbox: setup, what it can reach	docs/contributing/agents.md
Why it is built this way	docs/architecture/02-comparison-and-decisions.md
Tables, claim types, identifier schemes	docs/data-model.md
What each source allows	docs/sources.md, docs/legal.md

Ground rules

1. Legal access only. Collect as much as possible, but only through official APIs, open data and pages whose terms and robots.txt allow it. Never use X proxies (Nitter and the like), paywall bypasses, logged-in scraping, or WhatsApp/Signal. Telegram: public channels via the t.me/s web preview only. A robots.txt exemption needs the publisher's documented permission, recorded in the adapter's SourceSpec.tos_notes; the RTR API is the only one so far. If a source can't be accessed cleanly, say so and propose legal alternatives instead of working around it.
2. Every claim has evidence. No claim without a document link.
3. Fairness. Neutral wording. Legal matters carry the procedural stage and "Es gilt die Unschuldsvermutung". Automatically detected legal claims stay in review (review_status = pending). Connections are not accusations. Personal data only in the person's public role.
4. Not public yet. The instance is private as of October 2026. The X deletion sync (roadmap) is deferred until launch, but must exist before the instance goes public or an X token is configured.
5. Commits: only when the owner asks, as Simon Diesenreiter <simon@disi.dev>, not the global (work) address: git -c user.email=simon@disi.dev commit …, unless the repo-local config sets it.

Invariants that are easy to break

• Four stages. Adapters fetch only through Request objects handled by the framework's Fetcher (robots, throttling, TDM, archive), never with their own HTTP client. parse() is pure: no database, no network. Only the Loader writes claims. Enrichers derive from stored data.
• Claim identity. Claims with a facet get their fingerprint from it. Claims without a facet need an explicit key, or duplicates and lost updates follow.
• Parser changes: bump parser_version. Otherwise unchanged responses are skipped and the fix never applies. politrack reparse <adapter> rebuilds from the raw archive and deletes claims the new parser no longer emits.
• Golden fixtures: new or changed parsers come with a fixture (politrack fixture …). Review the .expected.json diff; don't blindly regenerate with UPDATE_GOLDEN=1.
• Schema: 0001_schema.sql and 0002_seed.sql are applied in existing databases. Change the schema only with a new numbered migration, with COMMENTs on new tables and columns.
• API types: after changing a response model, regenerate the frontend types: cd backend && politrack openapi --ts ../web/src/lib/api-types.ts.
• UI text: every new key goes into both web/messages/de.json and en.json.
• Backend dependencies: a dependency added to or changed in backend/pyproject.toml also goes into the Debian package list in backend/tests/Dockerfile.debian (usually python3-<name>). That list builds both the PyPI-free test image and the agent sandbox image; without the entry, tests there fail on the import. If Debian doesn't package it, add it to the exceptions in the file's header.

Commands

cd backend && pytest                                  # unit + golden tests, no DB or network needed
politrack sources list                                # adapters, enrichers, schedules, queues
politrack run <adapter> --max-requests 20 -p key=val  # limit requests when testing against live sources
politrack reparse <adapter>                           # re-parse archived responses
politrack enrich | status | review | migrate           # enrichers, recent runs, legal review, schema

In the agent-envs sandbox

You are in the sandbox if the repository is at /workspace and HTTPS_PROXY is http://egress:8080 (details). There:

• Network: only allowlisted hosts. HTTP 403 "Blocked by egress policy" means the host isn't allowed: tell the owner which host you need instead of working around it.
• Git: pushing and Gitea API calls over HTTPS work without credentials; the proxy adds them. Don't set up tokens or credential helpers. The commit identity comes from the repository's config.
• Tests: the sandbox image has the backend's dependencies from Debian packages, so cd backend && python3 -m pytest -q works without pip install. It comes from backend/tests/Dockerfile.debian (see "Backend dependencies" above).
• No Docker, no database. Leave adapter runs against the database and the compose stack to the owner. The notes below on Git's safe.directory don't apply in the sandbox; the PyPI and npm block does when the session runs on the owner's PC.

The owner's development machine

These notes apply to the owner's Windows PC (Docker Desktop, Git Bash, PowerShell):

• Git reports "dubious ownership" for this folder. Don't change the global git config; prefix git commands with git -c safe.directory=C:/Users/disi0/source/repos/politics-tracking.
• PyPI and npm downloads failed there in October 2026 (a network filter blocks files.pythonhosted.org and registry.npmjs.org). Until that is fixed, run the tests with the Debian test image (testing.md, "Without PyPI access"). The real images, npm install and the workers can't be built or run there.
• Local database: docker compose up -d db (port 5433, passwords in the gitignored .env, data in data/). It has the schema and sources but no scraped data yet, and no job-queue schema until the real migrate image runs. Postgres on a Windows bind mount is slow: the first start takes minutes.
• DATA_DIR, not APPDATA: Windows already defines APPDATA, so the compose file uses DATA_DIR.