Current members of eight Landtage from their official lists

One adapter per state (landtag.noe, .ooe, .stmk, .tirol, .vbg, .sbg, .bgld,
.wien) reads the official member list: name, party, district and mandate
start where published; no contact details or addresses. enrich.landtage
closes the roles of members who leave a list (skipping incomplete runs) and
records which states are covered; migration 0007 makes person_status trust
those lists instead of open-ended Wikidata Landtag roles. Kärnten is left
out because ktn.gv.at disallows all crawlers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
simonandClaude Opus 5.5 committed 2026-11-17 22:19:32 +01:00
1 parent d9c6c3649e
commit 3e6b1328b4
5 files changed
+482 -5

No files matched your search

@@ -0,0 +1,50 @@
-- State MPs: trust the official Landtag member lists (adapters landtag.*) where they exist. Wikidata often has no
-- end date for state mandates, so former members looked current; for states that enrich.landtage marks as covered,
-- open-ended Wikidata Landtag roles no longer count, and official Landtag roles count as verified.
CREATE OR REPLACE FUNCTION landtag_key(role_title TEXT) RETURNS TEXT LANGUAGE sql IMMUTABLE AS $$
SELECT CASE
WHEN role_title ILIKE '%%landtag%%' AND role_title ILIKE '%%niederösterreich%%' THEN 'noe'
WHEN role_title ILIKE '%%landtag%%' AND role_title ILIKE '%%oberösterreich%%' THEN 'ooe'
WHEN role_title ILIKE '%%landtag%%' AND (role_title ILIKE '%%steir%%' OR role_title ILIKE '%%steiermark%%') THEN 'stmk'
WHEN role_title ILIKE '%%landtag%%' AND role_title ILIKE '%%tirol%%' THEN 'tirol'
WHEN role_title ILIKE '%%landtag%%' AND role_title ILIKE '%%vorarlberg%%' THEN 'vbg'
WHEN role_title ILIKE '%%landtag%%' AND role_title ILIKE '%%salzburg%%' THEN 'sbg'
WHEN role_title ILIKE '%%landtag%%' AND role_title ILIKE '%%burgenl%%' THEN 'bgld'
WHEN role_title ILIKE '%%landtag%%' AND role_title ILIKE '%%wien%%' THEN 'wien'
WHEN role_title ILIKE '%%landtag%%' AND role_title ILIKE '%%kärnt%%' THEN 'ktn'
END
$$;
COMMENT ON FUNCTION landtag_key(TEXT) IS 'State key (noe, ooe, ...) of a Landtag role title from Wikidata, NULL for other roles.';
CREATE OR REPLACE VIEW person_status AS
WITH covered AS (
SELECT jsonb_array_elements_text(value) AS key FROM state WHERE adapter = 'enrich.landtage' AND key = 'covered'
),
current_roles AS (
SELECT r.person_id, r.level, r.role, r.party_id, r.start_date, c.created_by
FROM role r JOIN claim c ON c.id = r.claim_id
WHERE c.merged_into IS NULL AND (r.end_date IS NULL OR r.end_date >= current_date)
AND (c.created_by LIKE 'parlament%%' OR c.created_by LIKE 'europarl%%' OR c.created_by LIKE 'landtag.%%'
OR r.end_date IS NOT NULL OR r.level = 'party'
OR (r.start_date IS NOT NULL AND r.start_date >= current_date - interval '15 years'))
AND NOT (c.created_by = 'wikidata.politicians' AND r.end_date IS NULL AND r.level = 'state'
AND landtag_key(r.role) IN (SELECT key FROM covered))
)
SELECT p.id AS person_id,
CASE
WHEN p.death_date IS NOT NULL THEN 'deceased'
WHEN EXISTS (SELECT 1 FROM current_roles cr WHERE cr.person_id = p.id AND cr.level <> 'party') THEN 'active'
WHEN EXISTS (SELECT 1 FROM current_roles cr WHERE cr.person_id = p.id AND cr.level = 'party') THEN 'party_member'
ELSE 'former'
END AS status,
EXISTS (SELECT 1 FROM current_roles cr WHERE cr.person_id = p.id AND cr.level <> 'party'
AND (cr.created_by LIKE 'parlament%%' OR cr.created_by LIKE 'europarl%%' OR cr.created_by LIKE 'landtag.%%'))
AS verified,
(SELECT max(r.end_date) FROM role r JOIN claim c ON c.id = r.claim_id
WHERE r.person_id = p.id AND c.merged_into IS NULL AND r.level <> 'party') AS last_office_end,
(SELECT array_agg(DISTINCT cr.role) FROM current_roles cr WHERE cr.person_id = p.id AND cr.level <> 'party')
AS current_roles,
(SELECT array_agg(DISTINCT cr.party_id) FROM current_roles cr WHERE cr.person_id = p.id AND cr.party_id IS NOT NULL)
AS current_party_ids
FROM person p;
COMMENT ON VIEW person_status IS 'active = current office (verified = confirmed by Parliament/EP data or an official Landtag list); party_member = only party membership; former; deceased. For states covered by an official Landtag list (enrich.landtage), open-ended Wikidata Landtag roles are ignored.';
+359
View File
@@ -0,0 +1,359 @@
"""Current members of the state parliaments (Landtage), from each Landtag's official member list.
Wikidata knows most state MPs but often without an end date, so former members looked current. These adapters
read the official lists; `enrich.landtage` closes the official role of anyone who has left a list and records
which states are covered, and the person_status view then trusts the official lists for those states (migration
0007). One adapter per state, because every Landtag publishes its list differently:
noe noe-landtag.gv.at/personen (3 pages) + one profile per member (birth date, start of mandate)
ooe land-oberoesterreich.gv.at/12748.htm (party, district, "Mitglied des Landtags seit ...")
stmk landtag.steiermark.at member table
tirol portrait gallery ("Name (Party)")
vbg vorarlberg.at/web/landtag/abgeordnete-alle
sbg salzburg.gv.at, one page per Landtag club
bgld bgld-landtag.at/organe/abgeordnete (club sections; the postal addresses printed there are not kept)
wien wien.gv.at politician search (allowed in robots.txt), Landtag = Gemeinderat
Kärnten is missing on purpose: ktn.gv.at disallows all crawlers in robots.txt, so its members stay Wikidata-only.
Only names, party, district/region and mandate dates are kept (public role); no contact details or addresses.
"""
from __future__ import annotations
import re
from collections.abc import Iterator
from datetime import date
from typing import Any
from urllib.parse import parse_qs, urljoin, urlsplit
from bs4 import BeautifulSoup
from politrack.core import Adapter, Enricher, Request, Response, RunContext, SourceSpec, register, register_spec
from politrack.core.records import Claim, Evidence, OrgRef, PersonRecord, PersonRef, Role
from politrack.core.spec import DERIVED
from politrack.sources.bundesregierung import german_date
from politrack.text import make_slug, strip_titles
LANDTAGE = {
# key: (region, name of the parliament, seats, member list URL)
"noe": ("Niederösterreich", "Landtag Niederösterreich", 56, "https://noe-landtag.gv.at/personen"),
"ooe": ("Oberösterreich", "Oberösterreichischer Landtag", 56, "https://www.land-oberoesterreich.gv.at/12748.htm"),
"stmk": ("Steiermark", "Landtag Steiermark", 48, "https://www.landtag.steiermark.at/cms/ziel/31866695/DE/"),
"tirol": ("Tirol", "Tiroler Landtag", 36,
"https://www.tirol.gv.at/landtag/unser-parlament/die-abgeordneten/bildergalerie/"),
"vbg": ("Vorarlberg", "Vorarlberger Landtag", 36, "https://vorarlberg.at/web/landtag/abgeordnete-alle"),
"sbg": ("Salzburg", "Salzburger Landtag", 36, "https://www.salzburg.gv.at/pol/landtag/lt-klubs"),
"bgld": ("Burgenland", "Burgenländischer Landtag", 36, "https://www.bgld-landtag.at/organe/abgeordnete/"),
"wien": ("Wien", "Wiener Landtag", 100,
"https://www.wien.gv.at/advuew/internet/AdvPrSrv.asp?Layout=polstelle_at&Type=K"
"&stellecd=1997031808394662&Hlayout=politikersuche&AUSSEN"),
}
SBG_CLUBS = {"lt-oevp": "ÖVP", "lt-fpoe": "FPÖ", "lt-spoe": "SPÖ", "lt-kpoeplus": "KPÖ", "lt-diegruenen": "GRÜNE"}
PARTY_ALIASES = {"KEIN KLUB": None, "VP TIROL": "ÖVP", "VP": "ÖVP", "VOLKSPARTEI": "ÖVP", "GRÜNE": "GRÜNE", "DIE GRÜNEN": "GRÜNE",
"KPÖ PLUS": "KPÖ", "KPÖ": "KPÖ", "FRITZ": "FRITZ", "LISTE FRITZ": "FRITZ"}
# Functions written in front of names: Klubobmann, Landtagspräsidentin, KO-Stellvertreter, ...
ROLE_PREFIX = re.compile(
r"^(?:(?:Landtagspräsident(?:in)?|Präsident(?:in)?|(?:Zweite|Dritte)[rn]? Präsident(?:in)?|Klubobmann|Klubobfrau|"
r"Klubobleute|KO-Stellvertreter(?:in)?|KO|VP(?:in)?|LTPräs(?:in)?|LAbg\.|Abg\.|Bgm\.?(?:in)?|KommR|HR|Prof\.)\s+)+")
TITLE_TOKENS = {"Mag", "Maga", "MMag", "Dr", "DI", "DIin", "Dipl", "Ing", "Prof", "HR", "BA", "MA", "BSc", "MSc", "MBA",
"BEd", "B.Ed", "B.A", "M.A", "LL.M", "LLM", "MPA", "PhD", "KommR", "Bgm", "Päd", "Univ", "Mag.a",
"Dr.in", "Mag.(FH)", "(FH)", "MAS", "MSc.", "M.A.I.S", "Dipl.-Ing", "Dipl.Soz.-Wiss.in", "Ing.in"}
SUFFIX_TOKENS = {"a", "in"} # "Mag. a", "Dr. in"
def _is_title(token: str) -> bool:
t = token.strip(",.")
return t in TITLE_TOKENS or re.fullmatch(r"(?:Dipl|Univ|Mag|Dr)\.[\w.\-]*", token.strip(",")) is not None
def clean_name(name: str) -> str:
"""'KO Mag. Markus Abwerzger' -> 'Markus Abwerzger'; drops functions, academic titles and stray punctuation.
Titles are matched as whole tokens, so a first name like "Ingrid" is never taken for "Ing."."""
name = ROLE_PREFIX.sub("", re.sub(r"\s+", " ", name).strip())
name = strip_titles(name)
kept = []
for token in name.replace(",", " , ").split():
# "a"/"in" are the feminine endings of split titles ("Mag. a", "Dr. in"); never part of a name
if token == "," or _is_title(token) or token in SUFFIX_TOKENS:
continue
kept.append(token)
return re.sub(r"\s+", " ", " ".join(kept)).strip(" ,.")
def surname_first(text: str) -> str:
"""'Höglinger Tobias, Mag.' / 'AMON Werner, MBA' -> 'Tobias Höglinger' / 'Werner Amon'."""
head = text.split(",", 1)[0].strip()
parts = head.split()
if not parts:
return ""
upper = [p for p in parts if p.isupper() and len(p) > 1]
if upper and len(upper) < len(parts): # Steiermark: surname in capitals, possibly several words
surname = " ".join(p.title() if p.isupper() else p for p in upper)
given = " ".join(p for p in parts if p not in upper)
else:
surname, given = parts[0], " ".join(parts[1:])
return clean_name(f"{given} {surname}")
def party_short(text: str | None) -> str | None:
if not text:
return None
t = re.sub(r"[()\s]+", " ", text).strip()
if t.upper() in PARTY_ALIASES:
return PARTY_ALIASES[t.upper()]
return t.upper() if len(t) <= 6 else t
def member(name: str, member_id: str | None = None, party: str | None = None, region: str | None = None,
since: date | None = None, birth_date: date | None = None, url: str | None = None) -> dict[str, Any]:
return {"name": name, "id": member_id or make_slug(name), "party": party_short(party), "district": region,
"since": since, "birth_date": birth_date, "url": url}
# -- parsers (pure) -------------------------------------------------------------------------------------------
def parse_noe_list(html: str, base: str) -> tuple[list[str], list[str]]:
"""Profile URLs and further list pages of the NÖ member list."""
soup = BeautifulSoup(html, "html.parser")
profiles = sorted({urljoin(base, a["href"]) for a in soup.find_all("a", href=re.compile(r"^/personen/[a-z0-9_\-]+$"))})
pages = sorted({urljoin(base, a["href"]) for a in soup.find_all("a", href=re.compile(r"/personen\?seite=\d+$"))})
return profiles, pages
def parse_noe_profile(html: str, url: str) -> dict[str, Any] | None:
soup = BeautifulSoup(html, "html.parser")
h1 = soup.find("h1")
text = re.sub(r"\s+", " ", soup.get_text(" "))
mandate = re.search(r"Abgeordnete[r]? zum NÖ Landtag \(\s*([^)]+?)\s*\) seit (\d{2}\.\d{2}\.\d{4})", text)
if not h1 or not mandate:
return None
born = re.search(r"Geb\.:\s*(\d{2})\.(\d{2})\.(\d{4})", text)
district = re.search(r"Wahlkreis:\s*([^:]+?)\s+(?:Klub/Fraktion|Wahlpartei|Sitz)", text)
d, m, y = (int(x) for x in mandate.group(2).split("."))
return member(clean_name(h1.get_text(" ", strip=True)), url.rstrip("/").rsplit("/", 1)[-1], mandate.group(1),
district.group(1).strip() if district else None, date(y, m, d),
date(int(born.group(3)), int(born.group(2)), int(born.group(1))) if born else None, url)
def parse_ooe(html: str, base: str) -> list[dict[str, Any]]:
soup = BeautifulSoup(html, "html.parser")
out = []
for li in soup.find_all("li", attrs={"data-title": True, "data-partei": True}):
text = re.sub(r"\s+", " ", li.get_text(" "))
since = re.search(r"Mitglied des Landtags seit (\d{1,2}\. \w+ \d{4})", text)
district = re.search(r"Wahlkreis \d+ - ([^M]+?)\s+Mitglied", text)
link = li.find("a", href=True)
out.append(member(surname_first(li["data-title"]), None, li["data-partei"],
district.group(1).strip() if district else None,
german_date(since.group(1)) if since else None, None,
urljoin(base, link["href"]) if link else None))
return out
def parse_stmk(html: str, base: str) -> list[dict[str, Any]]:
soup = BeautifulSoup(html, "html.parser")
out = []
for row in soup.find_all("tr"):
cells = row.find_all("td")
link = row.find("a", href=re.compile(r"\.pdf$", re.IGNORECASE))
if len(cells) < 2 or not link:
continue
label = link.get_text(" ", strip=True)
out.append(member(surname_first(label), make_slug(label.split(",")[0]), cells[-1].get_text(" ", strip=True),
url=base))
return out
def parse_tirol(html: str, base: str) -> list[dict[str, Any]]:
soup = BeautifulSoup(html, "html.parser")
seen, out = set(), []
for img in soup.find_all("img", alt=re.compile(r"\(.+\)\s*$")):
m = re.match(r"(.+?)\s*\(\s*([^()]+?)\s*\)\s*$", img["alt"])
name = clean_name(m.group(1)) if m else ""
if name and name not in seen and "Mandat" not in name:
seen.add(name)
out.append(member(name, party=m.group(2), url=base))
return out
def parse_vbg(html: str, base: str) -> list[dict[str, Any]]:
soup = BeautifulSoup(html, "html.parser")
out = []
for box in soup.find_all("div", class_="avlr-standard-ap-mit-bild"):
link = box.find("a", href=re.compile(r"/web/landtag/-/"))
h3 = box.find("h3")
# the party is the last parenthesis ("Mag. (FH)" or "(WU)" in degrees come earlier)
parties = re.findall(r"\(([^()]{2,15})\)", box.get_text(" "))
if link and h3 and parties:
out.append(member(clean_name(h3.get_text(" ", strip=True)), link["href"].rstrip("/").rsplit("/", 1)[-1],
parties[-1], url=urljoin(base, link["href"])))
return out
def parse_sbg_club(html: str, base: str, party: str) -> list[dict[str, Any]]:
soup = BeautifulSoup(html, "html.parser")
out = []
for a in soup.find_all("a", href=re.compile(r"/politiker-detail-view/")):
img = a.find("img", alt=True)
if img:
out.append(member(clean_name(img["alt"]), a["href"].rstrip("/").rsplit("/", 1)[-1], party,
url=urljoin(base, a["href"])))
return out
BGLD_MEMBER = re.compile(
r"(?:@[\w.\-]+\.at|Bgld\. Landtag)\s+(.{3,90}?)\s+(?:\d\.\s)?(?:Landtagsabgeordnete[r]?|Landtagspräsident(?:in)?|"
r"Klubobmann|Klubobfrau|Klubvorsitzende[r]?)\s+(SPÖ|ÖVP|FPÖ|GRÜNE|Grüne|NEOS)\s+\d{4}\s")
def parse_bgld(html: str, base: str) -> list[dict[str, Any]]:
"""Club sections list "<name> <function> <party> <postcode> <place> <e-mail>"; only name and party are kept."""
text = re.sub(r"[\s​]+", " ", BeautifulSoup(html, "html.parser").get_text(" "))
out, seen = [], set()
for m in BGLD_MEMBER.finditer(text):
name = clean_name(m.group(1))
if name and name not in seen:
seen.add(name)
out.append(member(name, party=m.group(2), url=base))
return out
def parse_wien(body: bytes, base: str) -> list[dict[str, Any]]:
soup = BeautifulSoup(body, "html.parser") # the page declares its (legacy) charset itself
out = []
for a in soup.find_all("a", href=re.compile(r"Layout=politiker_at")):
cell = a.find_parent("td")
after = [t for t in (cell.stripped_strings if cell else []) if t not in a.stripped_strings]
party = after[-1].split()[-1] if after else None # "SPÖ" or "Klubvorsitzende NEOS"
person = parse_qs(urlsplit(a["href"]).query).get("personcd", [None])[0]
out.append(member(surname_first(a.get_text(" ", strip=True)), person, party, url=urljoin(base, a["href"])))
return out
# -- adapters -------------------------------------------------------------------------------------------------
def landtag_spec(key: str) -> SourceSpec:
region, name, _, url = LANDTAGE[key]
return register_spec(SourceSpec(
slug=f"landtag-{key}", name=name, kind="official", homepage=url, license="Amtliche Information",
access="scrape", trust=90, min_interval=2.0, store_fulltext=False,
tos_notes=f"Official member list of the {name} (public role data: name, party, district, mandate dates). "
"robots.txt allows the pages used; no contact details or addresses are stored."))
class LandtagMembers(Adapter):
key: str = ""
schedule = "35 5 * * *"
queue = "scrape"
creates_persons = True
def start(self, ctx: RunContext) -> Iterator[Request]:
yield Request(url=LANDTAGE[self.key][3], kind="list")
def members(self, ctx: RunContext, response: Response) -> Iterator[Any]:
raise NotImplementedError
def parse(self, ctx: RunContext, response: Response) -> Iterator[Any]:
for item in self.members(ctx, response):
if isinstance(item, Request):
yield item
elif item and item.get("name"):
yield from self.records(item)
else:
ctx.stats["entries_skipped"] += 1
def records(self, m: dict[str, Any]) -> Iterator[Any]:
region, body_name, _, list_url = LANDTAGE[self.key]
scheme = f"landtag-{self.key}"
ref = PersonRef(ids={scheme: m["id"]}, name=m["name"])
yield PersonRecord(ref=ref, full_name=m["name"], birth_date=m["birth_date"], match_by_name=True)
body = OrgRef(slug=f"landtag-{self.key}", name=body_name, kind="parliament")
party = OrgRef(short_name=m["party"], kind="party", create=False) if m["party"] else None
label = f" ({m['party']})" if m["party"] else ""
yield Claim(
claim_type="role", person=PersonRef(ids={scheme: m["id"]}), org=body, valid_from=m["since"],
summary_de=f"{m['name']}: Abgeordnete/r zum {body_name}{label}",
summary_en=f"{m['name']}: member of the {body_name}{label}",
data={"landtag": self.key, "party": m["party"], "district": m["district"]},
evidence=[Evidence(url=m["url"] or list_url, doc_type="register_entry", title=f"{body_name}: Abgeordnete",
locator=f"member:{m['id']}")],
facet=Role(role="mp", level="state", body=body, party=party, region=region, start=m["since"]),
)
def _register(key: str, members_fn) -> None:
cls = type(f"Landtag_{key}", (LandtagMembers,), {
"key": key, "spec": landtag_spec(key), "name": f"landtag.{key}",
"description": f"Current members of the {LANDTAGE[key][1]} (official list)", "members": members_fn,
})
register(cls)
def _noe(self, ctx, response):
if response.request.kind == "list":
profiles, pages = parse_noe_list(response.text, response.url)
if not response.meta.get("page"):
for page in pages:
yield Request(url=page, kind="list", meta={"page": True})
for url in profiles:
yield Request(url=url, kind="profile")
else:
yield parse_noe_profile(response.text, response.request.url)
def _sbg(self, ctx, response):
if response.request.kind == "list":
for club in SBG_CLUBS:
yield Request(url=f"https://www.salzburg.gv.at/pol/landtag/lt-klubs/{club}", kind="club",
meta={"party": SBG_CLUBS[club]})
else:
yield from parse_sbg_club(response.text, response.url, response.meta["party"])
_register("noe", _noe)
_register("ooe", lambda self, ctx, r: iter(parse_ooe(r.text, r.url)))
_register("stmk", lambda self, ctx, r: iter(parse_stmk(r.text, r.url)))
_register("tirol", lambda self, ctx, r: iter(parse_tirol(r.text, r.url)))
_register("vbg", lambda self, ctx, r: iter(parse_vbg(r.text, r.url)))
_register("sbg", _sbg)
_register("bgld", lambda self, ctx, r: iter(parse_bgld(r.text, r.url)))
_register("wien", lambda self, ctx, r: iter(parse_wien(r.body, r.url)))
@register
class LandtagStatus(Enricher):
"""Closes official Landtag roles of members who are no longer on their list and records which states have a
current official list (state key `covered`), which the person_status view uses instead of Wikidata."""
spec = DERIVED
name = "enrich.landtage"
description = "Close roles of members who left a Landtag list; mark states covered by an official list"
schedule = "15 6 * * *"
MIN_SHARE = 0.6 # a run that confirmed fewer members than this share of the seats is treated as broken
def run(self, ctx: RunContext) -> None:
conn = ctx.conn
covered = []
for key, (_, _, seats, _) in LANDTAGE.items():
last = conn.execute(
"SELECT started_at FROM run WHERE adapter = %s AND status IN ('ok', 'partial') "
"ORDER BY id DESC LIMIT 1", (f"landtag.{key}",)).fetchone()
if not last:
continue
confirmed = conn.execute(
"SELECT count(*) AS n FROM claim WHERE created_by = %s AND claim_type = 'role' AND updated_at >= %s",
(f"landtag.{key}", last["started_at"])).fetchone()["n"]
if confirmed < seats * self.MIN_SHARE:
ctx.stats[f"{key}_incomplete"] = confirmed
continue
covered.append(key)
closed = conn.execute(
"""WITH gone AS (
SELECT r.claim_id FROM role r JOIN claim c ON c.id = r.claim_id
WHERE c.created_by = %(adapter)s AND r.end_date IS NULL AND c.updated_at < %(since)s)
UPDATE role SET end_date = %(today)s WHERE claim_id IN (SELECT claim_id FROM gone)""",
{"adapter": f"landtag.{key}", "since": last["started_at"], "today": date.today()}).rowcount
ctx.stats[f"{key}_closed"] = closed
ctx.state.set("covered", covered)
ctx.stats["covered"] = len(covered)
return None
+68
View File
@@ -0,0 +1,68 @@
"""Landtag member-list parsers, on snippets of each Landtag's real markup (pages fetched November 2026)."""
from datetime import date
from politrack.sources import landtage as L
def test_clean_name_drops_functions_and_titles_but_not_names():
assert L.clean_name("KO Mag. Markus Abwerzger") == "Markus Abwerzger"
assert L.clean_name("Dipl. Päd., DI Carina Laschober-Luif") == "Carina Laschober-Luif"
assert L.clean_name("Mag. a Astrid Eisenkopf") == "Astrid Eisenkopf"
assert L.clean_name("Ingrid Korosec") == "Ingrid Korosec" # "Ing" is a title, "Ingrid" is not
assert L.surname_first("Höglinger Tobias, Mag.") == "Tobias Höglinger"
assert L.surname_first("EISEL-EISELSBERG Detlev") == "Detlev Eisel-Eiselsberg"
def test_noe_profile():
html = """<html><body><h1>Otto Auer</h1><h2>Lebenslauf</h2><p>Geb.: 04.01.1966 , Hainburg/Donau Beruf: Landwirt
Wahlkreis: Bruck an der Leitha Klub/Fraktion: Landtagsklub der Volkspartei Niederösterreich</p>
<table><tr><td>Abgeordneter zum NÖ Landtag ( ÖVP )</td><td>seit 23.03.2023</td></tr></table></body></html>"""
m = L.parse_noe_profile(html, "https://noe-landtag.gv.at/personen/otto_auer")
assert (m["name"], m["id"], m["party"], m["district"]) == ("Otto Auer", "otto_auer", "ÖVP", "Bruck an der Leitha")
assert (m["since"], m["birth_date"]) == (date(2023, 3, 23), date(1966, 1, 4))
def test_ooe():
html = """<ul><li class="list-item" data-groups="GRÜNE" data-title="Huber-Reiter Brigitte" data-partei="GRÜNE">
<h4><a href="/abg/huber">Huber-Reiter Brigitte</a></h4><p>Betriebsratsvorsitzende</p>
<p class="mb-2">Bezirk Ried<br> Wahlkreis 2 - Innviertel</p>
<p class="mb-0">Mitglied des Landtags seit <span class="text-nowrap">13. November 2025</span></p></li></ul>"""
[m] = L.parse_ooe(html, "https://www.land-oberoesterreich.gv.at/12748.htm")
assert (m["name"], m["party"], m["district"], m["since"]) == ("Brigitte Huber-Reiter", "GRÜNE", "Innviertel",
date(2025, 11, 13))
def test_stmk_tirol_vbg():
stmk = """<table><tr><td><a href="/cms/dokumente/1/AMON%20Werner.pdf">AMON Werner, MBA</a></td><td>&nbsp;ÖVP</td></tr>
</table>"""
assert [(m["name"], m["party"]) for m in L.parse_stmk(stmk, "https://x")] == [("Werner Amon", "ÖVP")]
tirol = """<img alt="KO Mag. Markus Abwerzger (FPÖ)"><img alt="Christoph Appler (VP Tirol)">
<img alt="Dr. Georg Dornauer (kein Klub)">"""
assert [(m["name"], m["party"]) for m in L.parse_tirol(tirol, "https://x")] == [
("Markus Abwerzger", "FPÖ"), ("Christoph Appler", "ÖVP"), ("Georg Dornauer", None)]
vbg = """<div class="avlr-standard-ap-mit-bild"><a href="https://vorarlberg.at/web/landtag/-/auer-manuela">
<figcaption>Bildrechte: mathis.studio</figcaption></a><h3>Mag. (FH) Manuela Auer</h3>
<div>Landtagsabgeordnete (SPÖ)</div></div>"""
assert [(m["name"], m["id"], m["party"]) for m in L.parse_vbg(vbg, "https://x")] == [
("Manuela Auer", "auer-manuela", "SPÖ")]
def test_sbg_bgld_wien():
sbg = """<ul><li><a href="/pol/landtag/lt-klubs/politiker-detail-view/pallauf-brigitta">
<img alt="Brigitta Pallauf"></a></li></ul>"""
assert [(m["name"], m["id"], m["party"]) for m in L.parse_sbg_club(sbg, "https://x", "ÖVP")] == [
("Brigitta Pallauf", "pallauf-brigitta", "ÖVP")]
bgld = """<p>SPÖ-Abgeordnete zum Bgld. Landtag Gerhard Bachmann Landtagsabgeordneter SPÖ 2423 Deutsch Jahrndorf
gerhard.bachmann@spoeklub-bgld.at Mag. a Astrid Eisenkopf Landtagspräsidentin SPÖ 7035 Steinbrunn
astrid.eisenkopf@bgld-landtag.at Claudia Schlager 3. Landtagspräsidentin SPÖ 7210 Mattersburg x@y.at</p>"""
members = L.parse_bgld(bgld, "https://x")
assert [(m["name"], m["party"]) for m in members] == [
("Gerhard Bachmann", "SPÖ"), ("Astrid Eisenkopf", "SPÖ"), ("Claudia Schlager", "SPÖ")]
assert all(m["district"] is None for m in members) # postal addresses are never kept
wien = """<table><tr><td><a href="/advuew/internet/AdvPrSrv.asp?Layout=politiker_at&amp;personcd=2015111618241279">
Abrahamczik Nina, Mag<sup>a</sup></a><br>SPÖ</td><td>+43 1 4000 81925</td></tr>
<tr><td><a href="/advuew/internet/AdvPrSrv.asp?Layout=politiker_at&amp;personcd=1">Arapovic Selma, Dipl.-Ing in</a>
<br>Klubvorsitzende NEOS</td><td>+43 1</td></tr></table>""".encode("utf-8")
assert [(m["name"], m["id"], m["party"]) for m in L.parse_wien(wien, "https://www.wien.gv.at/")] == [
("Nina Abrahamczik", "2015111618241279", "SPÖ"), ("Selma Arapovic", "1", "NEOS")]
+4 -4
View File
@@ -35,6 +35,7 @@ Next.js frontend (de/en), Docker Compose for Unraid.
- [x] EP roll-call votes of Austrian MEPs, including amendments and part votes (`europarl.votes`)
- [x] Committee memberships of the configured legislative periods (list 250, `parlament.persons`)
- [x] Full speech texts and interjections from the protocols (`parlament.speeches`)
- [x] Current members of eight Landtage from their official lists (`landtag.*`); status of state MPs from those lists
- [x] Event dates from the text (gestern, am Montag, 12. November) instead of publication dates
- [x] Election results and preference votes: NR 2024, NR 2019, NR 2017, EU 2024, EU 2019 preference votes (`bmi.elections`)
- [x] Former Austrian MEPs of the 2019-2024 term; EC meetings of the 2014-2019 Commission; Bluesky/Mastodon history paging
@@ -58,7 +59,7 @@ Next.js frontend (de/en), Docker Compose for Unraid.
- [ ] Election results before 2019, NR 2019 state lists, Landtag and municipal elections
- [ ] AMA/TAM subsidy recipients (after checking bulk access and terms)
- [ ] Cabinet staff and appointments to state-owned companies (from answers to parliamentary questions)
- [ ] One adapter per Landtag (OGD where available); municipal councils and mayors
- [ ] Landtag votes (only Steiermark has an API) and state election results; municipal councils and mayors
- [ ] Lobbying register – if the BMJ ever offers stable URLs or an export
**Quality**
@@ -95,9 +96,8 @@ Not run yet, because PyPI/npm were unreachable on the development network (Octob
- Votes in the Nationalrat are faction-level except roll calls (an Austrian parliamentary practice).
- Split votes (factions voting differently on parts of a bill) keep the official text but no faction
positions, because the summary cannot be attributed reliably.
- Activity status: parliament and EP data are authoritative (`status_verified`). Open-ended roles from
Wikidata count as current for 15 years after their start, so some state/local politicians may show
wrong status until per-state adapters exist.
- Activity status: parliament, EP and the official Landtag lists are authoritative (`status_verified`). Kärnten
and local offices still rely on Wikidata, whose open-ended roles count as current for 15 years after their start.
- Roll-call surnames that are ambiguous without a link in the protocol are skipped, not guessed.
- Travel/meeting extraction is rule-based; dates are publication dates of the source.
- News coverage is headline-level; mentions are linked by exact name or "office title + surname".
+1 -1
View File
@@ -54,7 +54,7 @@ Status: ✅ implemented (adapter name in `code`) · 🟡 planned · ⛔ delibera
| Meta Ad Library API | political ads of configured pages (`config/meta_pages.yaml`) | Graph API, token after identity confirmation | Meta platform terms | ✅ `ads.meta` (opt-in) |
| RIS (data.bka.gv.at) | laws, court decisions | OGD API | CC BY 4.0 | 🟡 decisions are anonymised; only linkable via a case number cited elsewhere |
| Lobbying- und Interessenvertretungs-Register (BMJ) | lobbying firms & clients | Lotus Notes web application without stable URLs (deep links return 404) | official work | ⛔ technically not reliably accessible; revisit if an export appears |
| State parliaments & governments (9 Länder) | Landtag members, votes | varies; several publish OGD | mostly CC BY 4.0 | 🟡 one adapter per state; until then via Wikidata |
| State parliaments (Landtage) | **current members** with party, district and mandate start where published; roles of members who leave a list are closed (`enrich.landtage`) | official member pages of eight Landtage (robots.txt checked per site) | official information | ✅ `landtag.noe`, `landtag.ooe`, `landtag.stmk`, `landtag.tirol`, `landtag.vbg`, `landtag.sbg`, `landtag.bgld`, `landtag.wien`; ⛔ Kärnten (ktn.gv.at disallows all crawlers); 🟡 Landtag votes and state election results |
| Municipalities | mayors, councils | fragmented | – | 🟡 via Wikidata |
| EU Council participant lists | which minister attended which Council | PDF per meeting | EU re-use policy | 🟡 not implemented yet |
| AMA farm subsidies, EU state-aid register (TAM) | subsidy recipients | search interfaces | – | 🟡 not implemented yet (bulk access and terms still to be checked) |