politics-tracking/AGENTS.md

AGENTS.md

Guidance for AI coding agents working on this repository. Humans: start with README.md and CONTRIBUTING.md; everything below links there instead of repeating it.

Politrack tracks Austrian politicians from EU to municipal level (roles, votes, money, statements, meetings, social media, legal proceedings). Every fact is a claim backed by evidence links. Backend: Python (FastAPI, Postgres 16 + pgvector, Procrastinate workers, MCP server). Frontend: Next.js 15 with next-intl, German by default, English at /en. Deployed with Docker Compose on the owner's Unraid server.

Before you start

Task	Read
Pick up open work	docs/roadmap.md: Next, Not yet verified, Known limitations
Add a data source	docs/contributing/adapters.md, legal checklist in §0 first
Derive facts from stored data	docs/contributing/enrichers.md
API endpoint or MCP tool	docs/contributing/api.md
UI page or new claim type in the UI	docs/contributing/frontend.md
Tests and fixtures	docs/contributing/testing.md
Why it is built this way	docs/architecture/02-comparison-and-decisions.md
Tables, claim types, identifier schemes	docs/data-model.md
What each source allows	docs/sources.md, docs/legal.md

Ground rules

1. Legal access only. Collect as much as possible, but only through official APIs, open data and pages whose terms and robots.txt allow it. Never use X proxies (Nitter and the like), paywall bypasses, logged-in scraping, or WhatsApp/Signal. Telegram: public channels via the t.me/s web preview only. A robots.txt exemption needs the publisher's documented permission, recorded in the adapter's SourceSpec.tos_notes; the RTR API is the only one so far. If a source can't be accessed cleanly, say so and propose legal alternatives instead of working around it.
2. Every claim has evidence. No claim without a document link.
3. Fairness. Neutral wording. Legal matters carry the procedural stage and "Es gilt die Unschuldsvermutung". Automatically detected legal claims stay in review (review_status = pending). Connections are not accusations. Personal data only in the person's public role.
4. Not public yet. The instance is private as of October 2026. The X deletion sync (roadmap) is deferred until launch, but must exist before the instance goes public or an X token is configured.
5. Commits: only when the owner asks, as Simon Diesenreiter <simon@disi.dev>, not the global (work) address: git -c user.email=simon@disi.dev commit …, unless the repo-local config sets it.

Invariants that are easy to break

• Four stages. Adapters fetch only through Request objects handled by the framework's Fetcher (robots, throttling, TDM, archive), never with their own HTTP client. parse() is pure: no database, no network. Only the Loader writes claims. Enrichers derive from stored data.
• Claim identity. Claims with a facet get their fingerprint from it. Claims without a facet need an explicit key, or duplicates and lost updates follow.
• Parser changes: bump parser_version. Otherwise unchanged responses are skipped and the fix never applies. politrack reparse <adapter> rebuilds from the raw archive and deletes claims the new parser no longer emits.
• Golden fixtures: new or changed parsers come with a fixture (politrack fixture …). Review the .expected.json diff; don't blindly regenerate with UPDATE_GOLDEN=1.
• Schema: 0001_schema.sql and 0002_seed.sql are applied in existing databases. Change the schema only with a new numbered migration, with COMMENTs on new tables and columns.
• API types: after changing a response model, regenerate the frontend types: cd backend && politrack openapi --ts ../web/src/lib/api-types.ts.
• UI text: every new key goes into both web/messages/de.json and en.json.

Commands

cd backend && pytest                                  # unit + golden tests, no DB or network needed
politrack sources list                                # adapters, enrichers, schedules, queues
politrack run <adapter> --max-requests 20 -p key=val  # limit requests when testing against live sources
politrack reparse <adapter>                           # re-parse archived responses
politrack enrich | status | review | migrate           # enrichers, recent runs, legal review, schema

The owner's development machine

These notes apply to the owner's Windows PC (Docker Desktop, Git Bash, PowerShell):

• Git reports "dubious ownership" for this folder. Don't change the global git config; prefix git commands with git -c safe.directory=C:/Users/disi0/source/repos/politics-tracking.
• PyPI and npm downloads failed there in October 2026 (a network filter blocks files.pythonhosted.org and registry.npmjs.org). Until that is fixed, run the tests with the Debian test image (testing.md, "Without PyPI access"). The real images, npm install and the workers can't be built or run there.
• Local database: docker compose up -d db (port 5433, passwords in the gitignored .env, data in data/). It has the schema and sources but no scraped data yet, and no job-queue schema until the real migrate image runs. Postgres on a Windows bind mount is slow: the first start takes minutes.
• DATA_DIR, not APPDATA: Windows already defines APPDATA, so the compose file uses DATA_DIR.