Simon DiesenreiterandClaude Opus 5.5
9da5a1dc48
Initial import of Politrack, a self-hosted service that tracks Austrian politicians from federal to municipal level - roles, votes, money flows, statements, meetings, social media and legal proceedings - with every claim linked to its sources. - backend: FastAPI + Postgres/pgvector; adapter framework (fetch -> parse -> load -> enrich) with robots.txt/TDM-aware fetching, a raw-response archive for reparsing, and a claim/evidence model with corroboration - sources: Parliament, Wikidata, EU Parliament/Commission, Council of Ministers, Rechnungshof party reports and donations, procurement, RTR media transparency, political ad libraries, WKStA, news feeds, Wikipedia edits, Bluesky, Mastodon, public Telegram channels, YouTube and X (official API) - web: Next.js frontend in German and English with typed API client - mcp: MCP server for AI agents and RAG - docs: architecture, data model, sources, legal notes, Unraid deployment and contributor guides; Docker Compose setup for Unraid Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
4.8 KiB
4.8 KiB
Roadmap
Done
Foundation (v1) – claim/evidence model, source trust and topic filters, parliament (members, mandates, functions, side jobs, speeches, faction votes, roll calls, immunity waivers, questions), Wikidata (state/local offices, party membership, careers), Rechnungshof donations, news feeds and sitemaps (headline-level, TDM-aware), entity linking, travel/meeting extraction, legal-matter detection with review, corroboration, activity status, former politicians, calendar, timeline, REST API, MCP server, Next.js frontend (de/en), Docker Compose for Unraid.
Re-architecture (v2) – see architecture/02-comparison-and-decisions.md:
- Adapter framework:
SourceSpecin code, pureparse(), Fetcher with robots/TDM/throttle/conditional GET - Raw archive +
politrack reparse; change detection (body hash + parser version) - Single writer (Loader), identity from resolved entities,
identifiertable, resolution hints (roll-call surnames, office holders on a date), legal-form-insensitive organisation matching - Registry + entry points; periodic tasks generated from adapter metadata; queues scrape/social/bulk/enrich
- Typed API (Pydantic) → OpenAPI → generated TypeScript; golden-fixture tests
- Contributor documentation
New data (v2):
- Rechnungshof annual party reports (totals, public funding, loans, holdings, affiliated organisations)
- Public procurement core data (contract awards, eForms)
- RTR media transparency (government ads, media subsidies)
- Council of Ministers minutes (agenda items per reporting minister)
- U-committee hearings (who testified when)
- WKStA press releases (documents for search; legal claims only after review)
- European Parliament: Austrian MEPs, declarations of interests
- European Commission meetings with interest representatives
- Wikipedia edits from parliament/government IP ranges
- Political ads: Google (bundle), Meta Ad Library (opt-in token)
- Social: Bluesky, Mastodon, public Telegram channels, YouTube (key), X via official API (token);
party/institution channels; curated
social_accounts.yaml - Pages: organisations, money explorer, social feed, Wikipedia edits, person network ("follow the money"), say-vs-vote on roll calls
Next
Data
- X deletion sync (Developer Agreement: remove posts deleted on X) – required before enabling
social.xin production - Full speech texts and interjections from the protocols; committee memberships (list 250)
- HowTheyVote.eu / EP roll-call votes of Austrian MEPs
- EU Council participant lists, ministries' press releases, Federal President's schedule
- Election results and candidate lists; AMA/TAM subsidy recipients (after checking bulk access and terms)
- Cabinet staff and appointments to state-owned companies (from answers to parliamentary questions)
- One adapter per Landtag (OGD where available); municipal councils and mayors
- Lobbying register – if the BMJ ever offers stable URLs or an export
Quality
- Golden fixtures for every adapter (the framework and first cases are in place)
- Better event dates (explicit dates in the text instead of publication dates)
- Entity resolution for donors/companies via Firmenbuch numbers from procurement notices
- Wayback Machine archiving of evidence URLs (
document.archive_url)
Product
- Community link submissions and a corrections/right-of-reply form
- Alerts (e-mail/RSS) for followed politicians
- Optional accounts via reverse-proxy SSO using
app_user - Partnerships: MeineAbgeordneten, Parteispenden.at, APA (OTS full-text licence)
- Map view of trips and appearances
Known limitations
- Votes in the Nationalrat are faction-level except roll calls (an Austrian parliamentary practice).
- Split votes (factions voting differently on parts of a bill) keep the official text but no faction positions, because the summary cannot be attributed reliably.
- Activity status: parliament and EP data are authoritative (
status_verified). Open-ended roles from Wikidata count as current for 15 years after their start, so some state/local politicians may show wrong status until per-state adapters exist. - Roll-call surnames that are ambiguous without a link in the protocol are skipped, not guessed.
- Travel/meeting extraction is rule-based; dates are publication dates of the source.
- News coverage is headline-level; mentions are linked by exact name or "office title + surname".
- Telegram: only channels with the public web preview; only the latest posts per run.
- Council of Ministers items are attributed to the office holder on that date; if the resolver cannot find who held the office (missing role data), the item is skipped and counted.