Files
simonandClaude Opus 5.5 583ccba31c Politrack: track Austrian politicians from public sources
Initial import of Politrack, a self-hosted service that tracks Austrian
politicians from federal to municipal level - roles, votes, money flows,
statements, meetings, social media and legal proceedings - with every claim
linked to its sources.

- backend: FastAPI + Postgres/pgvector; adapter framework (fetch -> parse ->
  load -> enrich) with robots.txt/TDM-aware fetching, a raw-response archive
  for reparsing, and a claim/evidence model with corroboration
- sources: Parliament, Wikidata, EU Parliament/Commission, Council of
  Ministers, Rechnungshof party reports and donations, procurement, RTR media
  transparency, political ad libraries, WKStA, news feeds, Wikipedia edits,
  Bluesky, Mastodon, public Telegram channels, YouTube and X (official API)
- web: Next.js frontend in German and English with typed API client
- mcp: MCP server for AI agents and RAG
- docs: architecture, data model, sources, legal notes, Unraid deployment and
  contributor guides; Docker Compose setup for Unraid

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:02:01 +01:00

5.8 KiB
Raw Permalink Blame History

Legal & ethical guard-rails

This is an engineering summary, not legal advice. Before publishing the site publicly, have an Austrian media/data-protection lawyer review it. Partner NGOs such as epicenter.works, Forum Informationsfreiheit and MeineAbgeordneten can help.

  • Official works (laws, official decrees and publications, §7 UrhG) are not protected by copyright. Parliament open data is CC BY 4.0 and needs attribution (shown on the method page and in every evidence link).
  • Text & data mining (§42h UrhG / Art. 4 DSM Directive) is allowed for lawfully accessible content unless the rightholder reserved it in machine-readable form. The Fetcher detects TDMRep reservations for every origin (no text and no raw archive is kept for reserved sites), and media content is limited to headline, teaser and link (see sources.md).
  • Database right (§76c ff. UrhG): systematically extracting substantial parts of someone else's curated database (for example MeineAbgeordneten) is not allowed without permission, so we ask for partnerships instead.
  • ToS: we never scrape platforms whose terms prohibit it (X, Meta, most newspapers). We use official APIs or leave the source out. Each source's assessment is recorded in its SourceSpec.tos_notes (shown on the method page and in /api/v1/sources).
  • Raw archive: responses are archived only where the publisher's terms allow it (SourceSpec.keep_raw) and never for TDM-reserved sites. The archive serves re-parsing and auditability, not republication.
  • X API: the Developer Agreement requires removing content of posts that were deleted on X. social.x keeps no raw archive or full text, but the deletion sync is still open – don't enable it in production before it exists (see roadmap).
  • Messaging platforms: only public Telegram broadcast channels through Telegram's public web preview. No WhatsApp (no public read access, terms forbid automated collection), no Signal, no private groups.

2. Personal data (GDPR / DSG)

  • Politicians are public figures, but this is still personal data. Our legal basis is legitimate interest / public interest in political transparency (Art. 6(1)(f) GDPR). Freedom of expression and information (Art. 85 GDPR) supports this, but Austria's implementation of the media privilege (§9 DSG) was challenged before the Constitutional Court and amended. Check the current version.
  • Scope: only the public role of a person: offices, votes, statements, public appearances, declared side jobs, donations to parties, and public court matters. No private addresses, family members (unless they hold office themselves) or health data.
  • Data on criminal offences (Art. 10 GDPR) is especially restricted. We only republish what official bodies or reputable media have already made public, always with the procedural stage and the presumption of innocence, and never infer anything ourselves.
  • Private donors named in the Rechnungshof lists: republishing information the law requires to be published is defensible, but show it exactly as published, link to the source, and honour erasure requests if the official publication is withdrawn. Private donors stay plain names (no person entity, no address – the record types have no address field); only companies and associations become organisations.
  • Wikipedia edits from institutional IP ranges are attributed to the network owner (parliament, government), never to a person.
  • Connections are not accusations: network and "follow the money" views say so explicitly; a side job or meeting is shown as what it is.
  • Provide a privacy policy, a contact for data-subject requests (access, rectification, objection) and a documented balancing test (LIA).

3. Media law (if the site is public)

  • Impressum/Offenlegung (§§24, 25 MedienG) is required for a website providing information.
  • §7b MedienG (presumption of innocence): anyone not finally convicted must not be presented as guilty. Implemented rule: every legal_proceeding claim shows "Es gilt die Unschuldsvermutung." unless its stage is convicted_final and it is not disputed. Acquittals, dropped proceedings, diversion and immunity requests keep the notice, since they confirm there is no finding of guilt. The MCP instructions tell agents the same. Summaries are phrased neutrally and attributed ("Ermittlungen (laut Bericht)", "nicht rechtskräftig verurteilt").
  • Review before publication: legal matters detected automatically in headlines are stored with review_status = 'pending' and are not shown until a person approves them (politrack review, or POST /api/v1/review/{id} with POLITRACK_ADMIN_TOKEN). Official data (Parliament immunity requests) is published directly, because the source itself is the official record.
  • Defamation (§111 StGB, §1330 ABGB): only publish what a source states, attributed to that source ("laut DER STANDARD …"), with a link. Disputed claims are labelled as such.
  • Right of reply / corrections: offer a corrections form so politicians can submit counter-evidence. Counter-evidence becomes stance = 'contradicts' evidence, which marks the claim disputed (it is never silently deleted).

4. Operational rules

  • Identify the crawler honestly (User-Agent with contact address) and respond to abuse mails.
  • Keep raw HTTP fetches minimal and do not mirror third-party content.
  • Keep an audit trail: every claim records created_by (which scraper or moderator) and when.
  • Keep humans in the loop for anything sensitive (legal matters): automatically detected claims wait in the review queue (review_status = 'pending') until a moderator approves them.
  • Every run is logged (run table) and every fetch too (fetch_log: URL with secrets redacted, status, hash) – useful when a publisher asks what was accessed.