Politrack: track Austrian politicians from public sources

Initial import of Politrack, a self-hosted service that tracks Austrian
politicians from federal to municipal level - roles, votes, money flows,
statements, meetings, social media and legal proceedings - with every claim
linked to its sources.

- backend: FastAPI + Postgres/pgvector; adapter framework (fetch -> parse ->
  load -> enrich) with robots.txt/TDM-aware fetching, a raw-response archive
  for reparsing, and a claim/evidence model with corroboration
- sources: Parliament, Wikidata, EU Parliament/Commission, Council of
  Ministers, Rechnungshof party reports and donations, procurement, RTR media
  transparency, political ad libraries, WKStA, news feeds, Wikipedia edits,
  Bluesky, Mastodon, public Telegram channels, YouTube and X (official API)
- web: Next.js frontend in German and English with typed API client
- mcp: MCP server for AI agents and RAG
- docs: architecture, data model, sources, legal notes, Unraid deployment and
  contributor guides; Docker Compose setup for Unraid

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
simonandClaude Opus 5.5 committed 2026-10-28 20:02:01 +01:00
commit 583ccba31c
187 files changed
+39890

No files matched your search

+75
View File
@@ -0,0 +1,75 @@
# Roadmap
## Done
**Foundation (v1)** – claim/evidence model, source trust and topic filters, parliament (members,
mandates, functions, side jobs, speeches, faction votes, roll calls, immunity waivers, questions),
Wikidata (state/local offices, party membership, careers), Rechnungshof donations, news feeds and
sitemaps (headline-level, TDM-aware), entity linking, travel/meeting extraction, legal-matter detection
with review, corroboration, activity status, former politicians, calendar, timeline, REST API, MCP server,
Next.js frontend (de/en), Docker Compose for Unraid.
**Re-architecture (v2)** – see [architecture/02-comparison-and-decisions.md](architecture/02-comparison-and-decisions.md):
- [x] Adapter framework: `SourceSpec` in code, pure `parse()`, Fetcher with robots/TDM/throttle/conditional GET
- [x] Raw archive + `politrack reparse`; change detection (body hash + parser version)
- [x] Single writer (Loader), identity from resolved entities, `identifier` table, resolution hints
(roll-call surnames, office holders on a date), legal-form-insensitive organisation matching
- [x] Registry + entry points; periodic tasks generated from adapter metadata; queues scrape/social/bulk/enrich
- [x] Typed API (Pydantic) → OpenAPI → generated TypeScript; golden-fixture tests
- [x] Contributor documentation
**New data (v2)**:
- [x] Rechnungshof annual party reports (totals, public funding, loans, holdings, affiliated organisations)
- [x] Public procurement core data (contract awards, eForms)
- [x] RTR media transparency (government ads, media subsidies)
- [x] Council of Ministers minutes (agenda items per reporting minister)
- [x] U-committee hearings (who testified when)
- [x] WKStA press releases (documents for search; legal claims only after review)
- [x] European Parliament: Austrian MEPs, declarations of interests
- [x] European Commission meetings with interest representatives
- [x] Wikipedia edits from parliament/government IP ranges
- [x] Political ads: Google (bundle), Meta Ad Library (opt-in token)
- [x] Social: Bluesky, Mastodon, public Telegram channels, YouTube (key), X via official API (token);
party/institution channels; curated `social_accounts.yaml`
- [x] Pages: organisations, money explorer, social feed, Wikipedia edits, person network ("follow the money"),
say-vs-vote on roll calls
## Next
**Data**
- [ ] X deletion sync (Developer Agreement: remove posts deleted on X) – required before enabling `social.x` in production
- [ ] Full speech texts and interjections from the protocols; committee memberships (list 250)
- [ ] HowTheyVote.eu / EP roll-call votes of Austrian MEPs
- [ ] EU Council participant lists, ministries' press releases, Federal President's schedule
- [ ] Election results and candidate lists; AMA/TAM subsidy recipients (after checking bulk access and terms)
- [ ] Cabinet staff and appointments to state-owned companies (from answers to parliamentary questions)
- [ ] One adapter per Landtag (OGD where available); municipal councils and mayors
- [ ] Lobbying register – if the BMJ ever offers stable URLs or an export
**Quality**
- [ ] Golden fixtures for every adapter (the framework and first cases are in place)
- [ ] Better event dates (explicit dates in the text instead of publication dates)
- [ ] Entity resolution for donors/companies via Firmenbuch numbers from procurement notices
- [ ] Wayback Machine archiving of evidence URLs (`document.archive_url`)
**Product**
- [ ] Community link submissions and a corrections/right-of-reply form
- [ ] Alerts (e-mail/RSS) for followed politicians
- [ ] Optional accounts via reverse-proxy SSO using `app_user`
- [ ] Partnerships: MeineAbgeordneten, Parteispenden.at, APA (OTS full-text licence)
- [ ] Map view of trips and appearances
## Known limitations
- Votes in the Nationalrat are faction-level except roll calls (an Austrian parliamentary practice).
- Split votes (factions voting differently on parts of a bill) keep the official text but no faction
positions, because the summary cannot be attributed reliably.
- Activity status: parliament and EP data are authoritative (`status_verified`). Open-ended roles from
Wikidata count as current for 15 years after their start, so some state/local politicians may show
wrong status until per-state adapters exist.
- Roll-call surnames that are ambiguous without a link in the protocol are skipped, not guessed.
- Travel/meeting extraction is rule-based; dates are publication dates of the source.
- News coverage is headline-level; mentions are linked by exact name or "office title + surname".
- Telegram: only channels with the public web preview; only the latest posts per run.
- Council of Ministers items are attributed to the office holder on that date; if the resolver cannot
find who held the office (missing role data), the item is skipped and counted.