Politrack: track Austrian politicians from public sources
Initial import of Politrack, a self-hosted service that tracks Austrian politicians from federal to municipal level - roles, votes, money flows, statements, meetings, social media and legal proceedings - with every claim linked to its sources. - backend: FastAPI + Postgres/pgvector; adapter framework (fetch -> parse -> load -> enrich) with robots.txt/TDM-aware fetching, a raw-response archive for reparsing, and a claim/evidence model with corroboration - sources: Parliament, Wikidata, EU Parliament/Commission, Council of Ministers, Rechnungshof party reports and donations, procurement, RTR media transparency, political ad libraries, WKStA, news feeds, Wikipedia edits, Bluesky, Mastodon, public Telegram channels, YouTube and X (official API) - web: Next.js frontend in German and English with typed API client - mcp: MCP server for AI agents and RAG - docs: architecture, data model, sources, legal notes, Unraid deployment and contributor guides; Docker Compose setup for Unraid Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
commit
583ccba31c
187 files changed
+39890
No files matched your search
@@ -0,0 +1,75 @@
|
||||
# Roadmap
|
||||
|
||||
## Done
|
||||
|
||||
**Foundation (v1)** – claim/evidence model, source trust and topic filters, parliament (members,
|
||||
mandates, functions, side jobs, speeches, faction votes, roll calls, immunity waivers, questions),
|
||||
Wikidata (state/local offices, party membership, careers), Rechnungshof donations, news feeds and
|
||||
sitemaps (headline-level, TDM-aware), entity linking, travel/meeting extraction, legal-matter detection
|
||||
with review, corroboration, activity status, former politicians, calendar, timeline, REST API, MCP server,
|
||||
Next.js frontend (de/en), Docker Compose for Unraid.
|
||||
|
||||
**Re-architecture (v2)** – see [architecture/02-comparison-and-decisions.md](architecture/02-comparison-and-decisions.md):
|
||||
- [x] Adapter framework: `SourceSpec` in code, pure `parse()`, Fetcher with robots/TDM/throttle/conditional GET
|
||||
- [x] Raw archive + `politrack reparse`; change detection (body hash + parser version)
|
||||
- [x] Single writer (Loader), identity from resolved entities, `identifier` table, resolution hints
|
||||
(roll-call surnames, office holders on a date), legal-form-insensitive organisation matching
|
||||
- [x] Registry + entry points; periodic tasks generated from adapter metadata; queues scrape/social/bulk/enrich
|
||||
- [x] Typed API (Pydantic) → OpenAPI → generated TypeScript; golden-fixture tests
|
||||
- [x] Contributor documentation
|
||||
|
||||
**New data (v2)**:
|
||||
- [x] Rechnungshof annual party reports (totals, public funding, loans, holdings, affiliated organisations)
|
||||
- [x] Public procurement core data (contract awards, eForms)
|
||||
- [x] RTR media transparency (government ads, media subsidies)
|
||||
- [x] Council of Ministers minutes (agenda items per reporting minister)
|
||||
- [x] U-committee hearings (who testified when)
|
||||
- [x] WKStA press releases (documents for search; legal claims only after review)
|
||||
- [x] European Parliament: Austrian MEPs, declarations of interests
|
||||
- [x] European Commission meetings with interest representatives
|
||||
- [x] Wikipedia edits from parliament/government IP ranges
|
||||
- [x] Political ads: Google (bundle), Meta Ad Library (opt-in token)
|
||||
- [x] Social: Bluesky, Mastodon, public Telegram channels, YouTube (key), X via official API (token);
|
||||
party/institution channels; curated `social_accounts.yaml`
|
||||
- [x] Pages: organisations, money explorer, social feed, Wikipedia edits, person network ("follow the money"),
|
||||
say-vs-vote on roll calls
|
||||
|
||||
## Next
|
||||
|
||||
**Data**
|
||||
- [ ] X deletion sync (Developer Agreement: remove posts deleted on X) – required before enabling `social.x` in production
|
||||
- [ ] Full speech texts and interjections from the protocols; committee memberships (list 250)
|
||||
- [ ] HowTheyVote.eu / EP roll-call votes of Austrian MEPs
|
||||
- [ ] EU Council participant lists, ministries' press releases, Federal President's schedule
|
||||
- [ ] Election results and candidate lists; AMA/TAM subsidy recipients (after checking bulk access and terms)
|
||||
- [ ] Cabinet staff and appointments to state-owned companies (from answers to parliamentary questions)
|
||||
- [ ] One adapter per Landtag (OGD where available); municipal councils and mayors
|
||||
- [ ] Lobbying register – if the BMJ ever offers stable URLs or an export
|
||||
|
||||
**Quality**
|
||||
- [ ] Golden fixtures for every adapter (the framework and first cases are in place)
|
||||
- [ ] Better event dates (explicit dates in the text instead of publication dates)
|
||||
- [ ] Entity resolution for donors/companies via Firmenbuch numbers from procurement notices
|
||||
- [ ] Wayback Machine archiving of evidence URLs (`document.archive_url`)
|
||||
|
||||
**Product**
|
||||
- [ ] Community link submissions and a corrections/right-of-reply form
|
||||
- [ ] Alerts (e-mail/RSS) for followed politicians
|
||||
- [ ] Optional accounts via reverse-proxy SSO using `app_user`
|
||||
- [ ] Partnerships: MeineAbgeordneten, Parteispenden.at, APA (OTS full-text licence)
|
||||
- [ ] Map view of trips and appearances
|
||||
|
||||
## Known limitations
|
||||
|
||||
- Votes in the Nationalrat are faction-level except roll calls (an Austrian parliamentary practice).
|
||||
- Split votes (factions voting differently on parts of a bill) keep the official text but no faction
|
||||
positions, because the summary cannot be attributed reliably.
|
||||
- Activity status: parliament and EP data are authoritative (`status_verified`). Open-ended roles from
|
||||
Wikidata count as current for 15 years after their start, so some state/local politicians may show
|
||||
wrong status until per-state adapters exist.
|
||||
- Roll-call surnames that are ambiguous without a link in the protocol are skipped, not guessed.
|
||||
- Travel/meeting extraction is rule-based; dates are publication dates of the source.
|
||||
- News coverage is headline-level; mentions are linked by exact name or "office title + surname".
|
||||
- Telegram: only channels with the public web preview; only the latest posts per run.
|
||||
- Council of Ministers items are attributed to the office holder on that date; if the resolver cannot
|
||||
find who held the office (missing role data), the item is skipped and counted.
|
||||
Reference in new issue
Block a user