Politrack: track Austrian politicians from public sources
Initial import of Politrack, a self-hosted service that tracks Austrian politicians from federal to municipal level - roles, votes, money flows, statements, meetings, social media and legal proceedings - with every claim linked to its sources. - backend: FastAPI + Postgres/pgvector; adapter framework (fetch -> parse -> load -> enrich) with robots.txt/TDM-aware fetching, a raw-response archive for reparsing, and a claim/evidence model with corroboration - sources: Parliament, Wikidata, EU Parliament/Commission, Council of Ministers, Rechnungshof party reports and donations, procurement, RTR media transparency, political ad libraries, WKStA, news feeds, Wikipedia edits, Bluesky, Mastodon, public Telegram channels, YouTube and X (official API) - web: Next.js frontend in German and English with typed API client - mcp: MCP server for AI agents and RAG - docs: architecture, data model, sources, legal notes, Unraid deployment and contributor guides; Docker Compose setup for Unraid Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
commit
583ccba31c
187 files changed
+39890
No files matched your search
@@ -0,0 +1,97 @@
|
||||
# Politrack 🇦🇹
|
||||
|
||||
**What Austrian politicians do, with every claim linked to its sources.**
|
||||
|
||||
Politrack collects publicly available information about Austrian politicians and parties, from the EU and
|
||||
federal level down to municipalities: mandates and functions, how factions and individuals voted, party
|
||||
donations and finances, public contracts, government advertising, side jobs, lobby meetings, speeches,
|
||||
questions and public posts, media reports, trips and meetings, and legal proceedings. Every fact is a
|
||||
**claim** with one or more pieces of **evidence** (links), so readers can verify it and decide which
|
||||
sources they trust.
|
||||
|
||||
* 🇩🇪 German UI by default (`/`), English at `/en`
|
||||
* Users choose **which sources they trust** and **which topics they care about**
|
||||
* Pages per **person**, **party** and **organisation**, a **money** explorer, **calendar** and **timeline**
|
||||
* **Votes**: faction votes for every bill, **individual roll-call votes** with a dissent flag, and
|
||||
"say vs. vote" (did the person speak on the item?)
|
||||
* **Follow the money**: donations, party funding, loans and holdings, public contracts, government ads,
|
||||
political ads – and the organisations a politician is connected to
|
||||
* **Corroboration**: the same fact from independent sources becomes one claim with several sources
|
||||
* **Former politicians**: what they are doing now
|
||||
* Legal matters always carry the procedural stage and *"Es gilt die Unschuldsvermutung"*; automatically
|
||||
detected ones need human review first
|
||||
* **Agent-ready**: MCP server, typed REST/OpenAPI, read-only SQL role, pgvector chunks for RAG
|
||||
* Runs as Docker containers on a homelab (Unraid); workers scale horizontally
|
||||
* **Legal by construction**: robots.txt, rate limits, TDM reservations and storage rules are enforced in
|
||||
the framework, not left to each scraper
|
||||
|
||||
## Documentation
|
||||
|
||||
| Doc | Contents |
|
||||
|---|---|
|
||||
| [docs/architecture/01-design-from-scratch.md](docs/architecture/01-design-from-scratch.md) | The architecture: quality goals, data model, adapter framework, serving, testing, guard-rails |
|
||||
| [docs/architecture/02-comparison-and-decisions.md](docs/architecture/02-comparison-and-decisions.md) | Why it is built this way (decision records) |
|
||||
| [docs/data-model.md](docs/data-model.md) | Tables, claim types, identifier schemes, views |
|
||||
| [docs/sources.md](docs/sources.md) | Every source with access method, licence/ToS assessment, and what we deliberately don't collect (incl. Telegram/WhatsApp/Signal) |
|
||||
| [docs/legal.md](docs/legal.md) | Copyright/TDM, GDPR, MedienG §7b (presumption of innocence), defamation |
|
||||
| [docs/deployment-unraid.md](docs/deployment-unraid.md) | Step-by-step Unraid setup, backups, exposure |
|
||||
| [docs/investigative-data.md](docs/investigative-data.md) | Data points for investigative work and the correlations they enable |
|
||||
| [docs/market-research.md](docs/market-research.md) | Existing services and the gaps Politrack fills |
|
||||
| [docs/roadmap.md](docs/roadmap.md) | What's done and what comes next |
|
||||
| [CONTRIBUTING.md](CONTRIBUTING.md) | How to contribute: adapters, enrichers, API, frontend, tests |
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
cp .env.example .env # set passwords and a real contact address for the crawler
|
||||
docker compose up -d --build # db, migrate, api, web, mcp, workers
|
||||
docker compose exec api politrack sources list # all adapters/enrichers and their schedules
|
||||
docker compose exec api politrack run parlament.persons # members, mandates, side jobs, speeches
|
||||
docker compose exec api politrack run wikidata.politicians # state/local offices, parties, careers, social handles
|
||||
docker compose exec api politrack run parlament.items # votes, roll calls, immunity, questions, hearings
|
||||
docker compose exec api politrack run rechnungshof.donations # party donations
|
||||
docker compose exec api politrack enrich # linking, events, legal detection, corroboration
|
||||
docker compose exec api politrack review # approve/reject automatically detected legal matters
|
||||
docker compose exec api politrack status # recent runs with stats and errors
|
||||
```
|
||||
|
||||
The workers then keep everything fresh on their own: each adapter declares its cron schedule and queue.
|
||||
|
||||
* Web: http://localhost:3000
|
||||
* API docs: http://localhost:8000/api/docs
|
||||
* MCP (streamable HTTP): http://localhost:8001/mcp
|
||||
* SQL (read-only): `postgresql://politrack_reader:<POLITRACK_READER_PASSWORD>@localhost:5433/politrack`
|
||||
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
backend/ Python package `politrack` (one image for API, workers, MCP)
|
||||
migrations/ SQL schema (commented for agents) + seed (chambers, parties, topics)
|
||||
config/ feeds.yaml, social_accounts.yaml, ip_ranges.yaml, meta_pages.yaml
|
||||
politrack/
|
||||
core/ the framework: SourceSpec, Adapter/Enricher, Request/Response, Fetcher
|
||||
(policy + archive), records, Resolver, Loader, registry, runner
|
||||
sources/ one module per publisher (parlament/, wikidata, rechnungshof, bundesregierung,
|
||||
procurement, rtr, eu, wikipedia, political_ads, justiz, news, gdelt, social)
|
||||
enrich/ linker, events, legal, topics, maintenance, corroborate, embed, curated
|
||||
queries/ read queries shared by API and MCP
|
||||
api/ FastAPI app + Pydantic response models
|
||||
mcp_server.py MCP tools for agents
|
||||
worker.py Procrastinate tasks generated from the registry
|
||||
tests/ unit tests + golden fixtures
|
||||
web/ Next.js 15 + next-intl frontend (types generated from the API)
|
||||
docs/
|
||||
docker-compose.yml
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
Adding a source is one module with a `SourceSpec`, `start()` and a pure `parse()` – see
|
||||
[docs/contributing/adapters.md](docs/contributing/adapters.md) and [CONTRIBUTING.md](CONTRIBUTING.md).
|
||||
|
||||
## Data licence & attribution
|
||||
|
||||
Data from the Austrian Parliament: © Parlamentsdirektion, CC BY 4.0. Wikidata: CC0. Rechnungshof,
|
||||
Bundeskanzleramt and WKStA publications are official works (§7 UrhG). Open government data from
|
||||
data.gv.at and RTR under their respective licences (mostly CC BY 4.0). Media content is linked, not
|
||||
reproduced.
|
||||
Reference in new issue
Block a user