simonandClaude Opus 5.5
45514624f4
- AGENTS.md: ground rules (legal access, evidence, fairness), invariants that are easy to break, commands, and notes on the owner's dev machine; CLAUDE.md imports it - backend/tests/Dockerfile.debian: test image from Debian packages for networks where PyPI is unreachable, documented in testing.md - roadmap: list what v2 has not been verified against yet Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
88 lines
5.5 KiB
Markdown
88 lines
5.5 KiB
Markdown
# Roadmap
|
||
|
||
## Done
|
||
|
||
**Foundation (v1)** – claim/evidence model, source trust and topic filters, parliament (members,
|
||
mandates, functions, side jobs, speeches, faction votes, roll calls, immunity waivers, questions),
|
||
Wikidata (state/local offices, party membership, careers), Rechnungshof donations, news feeds and
|
||
sitemaps (headline-level, TDM-aware), entity linking, travel/meeting extraction, legal-matter detection
|
||
with review, corroboration, activity status, former politicians, calendar, timeline, REST API, MCP server,
|
||
Next.js frontend (de/en), Docker Compose for Unraid.
|
||
|
||
**Re-architecture (v2)** – see [architecture/02-comparison-and-decisions.md](architecture/02-comparison-and-decisions.md):
|
||
- [x] Adapter framework: `SourceSpec` in code, pure `parse()`, Fetcher with robots/TDM/throttle/conditional GET
|
||
- [x] Raw archive + `politrack reparse`; change detection (body hash + parser version)
|
||
- [x] Single writer (Loader), identity from resolved entities, `identifier` table, resolution hints
|
||
(roll-call surnames, office holders on a date), legal-form-insensitive organisation matching
|
||
- [x] Registry + entry points; periodic tasks generated from adapter metadata; queues scrape/social/bulk/enrich
|
||
- [x] Typed API (Pydantic) → OpenAPI → generated TypeScript; golden-fixture tests
|
||
- [x] Contributor documentation
|
||
|
||
**New data (v2)**:
|
||
- [x] Rechnungshof annual party reports (totals, public funding, loans, holdings, affiliated organisations)
|
||
- [x] Public procurement core data (contract awards, eForms)
|
||
- [x] RTR media transparency (government ads, media subsidies)
|
||
- [x] Council of Ministers minutes (agenda items per reporting minister)
|
||
- [x] U-committee hearings (who testified when)
|
||
- [x] WKStA press releases (documents for search; legal claims only after review)
|
||
- [x] European Parliament: Austrian MEPs, declarations of interests
|
||
- [x] European Commission meetings with interest representatives
|
||
- [x] Wikipedia edits from parliament/government IP ranges
|
||
- [x] Political ads: Google (bundle), Meta Ad Library (opt-in token)
|
||
- [x] Social: Bluesky, Mastodon, public Telegram channels, YouTube (key), X via official API (token);
|
||
party/institution channels; curated `social_accounts.yaml`
|
||
- [x] Pages: organisations, money explorer, social feed, Wikipedia edits, person network ("follow the money"),
|
||
say-vs-vote on roll calls
|
||
|
||
## Next
|
||
|
||
**Data**
|
||
- [ ] X deletion sync (Developer Agreement: remove posts deleted on X) – required before enabling `social.x` in production
|
||
- [ ] Full speech texts and interjections from the protocols; committee memberships (list 250)
|
||
- [ ] HowTheyVote.eu / EP roll-call votes of Austrian MEPs
|
||
- [ ] EU Council participant lists, ministries' press releases, Federal President's schedule
|
||
- [ ] Election results and candidate lists; AMA/TAM subsidy recipients (after checking bulk access and terms)
|
||
- [ ] Cabinet staff and appointments to state-owned companies (from answers to parliamentary questions)
|
||
- [ ] One adapter per Landtag (OGD where available); municipal councils and mayors
|
||
- [ ] Lobbying register – if the BMJ ever offers stable URLs or an export
|
||
|
||
**Quality**
|
||
- [ ] Golden fixtures for every adapter (the framework and first cases are in place)
|
||
- [ ] Better event dates (explicit dates in the text instead of publication dates)
|
||
- [ ] Entity resolution for donors/companies via Firmenbuch numbers from procurement notices
|
||
- [ ] Wayback Machine archiving of evidence URLs (`document.archive_url`)
|
||
|
||
**Product**
|
||
- [ ] Community link submissions and a corrections/right-of-reply form
|
||
- [ ] Alerts (e-mail/RSS) for followed politicians
|
||
- [ ] Optional accounts via reverse-proxy SSO using `app_user`
|
||
- [ ] Partnerships: MeineAbgeordneten, Parteispenden.at, APA (OTS full-text licence)
|
||
- [ ] Map view of trips and appearances
|
||
|
||
## Not yet verified (v2)
|
||
|
||
Tested so far: unit and golden tests, a live pipeline run of every adapter with `--max-requests`
|
||
against a test database, every API endpoint, the frontend type check against the generated types.
|
||
Not run yet, because PyPI/npm were unreachable on the development network (October 2026):
|
||
|
||
- [ ] Building the `backend` and `web` images (`docker compose build`), `npm run build`
|
||
- [ ] Procrastinate workers and the periodic schedule; the job-queue schema (`politrack migrate` installs it)
|
||
- [ ] The MCP server at runtime
|
||
- [ ] Mastodon (no account in the test sample), YouTube, X and Meta Ad Library (no credentials)
|
||
- [ ] GDELT (rate-limited during the test), EP declarations for MEPs whose API calls returned HTTP 500
|
||
|
||
## Known limitations
|
||
|
||
- Votes in the Nationalrat are faction-level except roll calls (an Austrian parliamentary practice).
|
||
- Split votes (factions voting differently on parts of a bill) keep the official text but no faction
|
||
positions, because the summary cannot be attributed reliably.
|
||
- Activity status: parliament and EP data are authoritative (`status_verified`). Open-ended roles from
|
||
Wikidata count as current for 15 years after their start, so some state/local politicians may show
|
||
wrong status until per-state adapters exist.
|
||
- Roll-call surnames that are ambiguous without a link in the protocol are skipped, not guessed.
|
||
- Travel/meeting extraction is rule-based; dates are publication dates of the source.
|
||
- News coverage is headline-level; mentions are linked by exact name or "office title + surname".
|
||
- Telegram: only channels with the public web preview; only the latest posts per run.
|
||
- Council of Ministers items are attributed to the office holder on that date; if the resolver cannot
|
||
find who held the office (missing role data), the item is skipped and counted.
|