Features/#1310 ethos builda integration

[jh-RLI]

Fixes #1310. Addresses the residential half of #1311.

Residential buildings were selected by their OSM tags alone, which overestimates
the stock by about a third against Zensus 2022. ETHOS.BUILDA supplies one point
per residential building, so this PR decides "is this building residential?" by
intersecting OSM building polygons with those points instead.

New dataset EthosBuilda (society.egon_ethos_builda_buildings) downloads
one CSV per NUTS-1 region from Zenodo (record 13771740, ETHOS.BUILDA v2.0.0,
ODbL-1.0), verifies the published MD5, and imports it through an unlogged
staging table — the four building attributes are JSON objects
{"value","source","lineage"}, not scalars, and are unpacked there. Regions are
loaded one at a time with the staging table truncated in between, which bounds
it to the largest single CSV (1.6 GB) instead of the full 7.5 GB. In test mode
only Schleswig-Holstein (DEF) is fetched and imported.

osm_buildings_filter_residential.sql is rewritten as the three-step
cascade of Till Krebber's method (RLI):

1. ETHOS point inside the OSM polygon → source = 'ethos_intersect'
2. remainder: nearest neighbour within 10 m of ST_PointOnSurface →
   source = 'ethos_nearest'
3. remainder: building tag on the residential whitelist →
   source = 'osm_tagging'

osm_buildings_residential gains 7 columns: source, ethos_id,
match_distance and the four ETHOS attributes (construction_year,
size_class, refurbishment_state, tabula_type). The census gap filler
osm_buildings_extend_residential.sql stays as a safety net and marks its rows
source = 'census_gap_fill'; its INSERT ... SELECT * had to become an explicit
column list because the target table is now wider than
osm_buildings_filtered.

Downstream consequence, and the reason this PR is not confined to the building
dataset: residential buildings are no longer a subset of the filtered ones.
Measured on SH, 2,165 residential buildings (0.35 %) sit outside
osm_buildings_filtered — 1,621 plausible dwellings whose OSM tag the filter
list does not carry, plus 544 ancillary buildings that ETHOS hits and that are
kept deliberately. So everywhere the filtered table stood for "all buildings
that can carry a demand", the residential table is now read alongside it:
loadareas_add_demand_hh.sql (both UNION blocks), mapping.py, and
pv_rooftop_buildings.py. The first two use UNION (not UNION ALL), verified
free of double counting: the 616,043 overlapping rows have 0 deviations in
geom_point. The third merges with pd.concat in pandas rather than in SQL, so
its index is deduplicated explicitly.

Also adds the missing PRIMARY KEY (id) on
boundaries.egon_map_zensus_buildings_residential — the only one of the three
census mapping tables without one.

Measured on Schleswig-Holstein

Task	Time	Result
EthosBuilda.to_postgres	73.2 s	581,222 rows
filter_buildings_residential	45.5 s	618,208 buildings, 618,208 distinct ids

618,208 = 0.713 × Zensus 2022, against 607,490 = 0.700 for the reference
implementation (+1.8 %). Sources: ethos_intersect 579,076 (93.67 %),
osm_tagging 38,084 (6.16 %), ethos_nearest 1,048 (0.17 %).

📋 Pull Request Guidelines

Please read the Pull Request Guidelines carefully before creating your PR.

---

🧑‍💻 Contributor Checklist

Before requesting a review, make sure you've completed all of the following:

• All tests pass locally or via CI
  (for more information on local test, check tox in the Contributing section)
  (CI tests are automatically executed when creating a PR, you can see the results of the checks below)
• Workflow has run at least once in Test mode
  (optional if no dataset changes are involved)
• Relevant documentation is updated (API, new features, etc.)
• Dataset-versions are updated when existing datasets are adjusted.
• Added a note to CHANGELOG.rst about the changes
• Added yourself to AUTHORS.rst

Optional:

• Changes have been tested in Everything mode
• Extend the checklist for reviewers: Which aspects should be reviewed in particular?

Please focus on these four, in this order:

1. `pv_rooftop_buildings.py` — `load_building_data()`. This is the riskiest
   change in the PR and the one least exercised so far. `osm_buildings()` now
   reads two tables that overlap heavily, and they are merged with `pd.concat`,
   not a SQL UNION. Without the index dedup, `buildings_gdf.loc[building_ids]`
   returns the 616,043 shared buildings twice and doubles their roof potential —
   a worse bug than the one being fixed. Please check the dedup sits where the
   index must be unique and that no earlier `.loc`/`.reindex` runs before it.

2. The three UNION additions (`loadareas_add_demand_hh.sql` ×2, `mapping.py`).
   `UNION` is deliberate; `UNION ALL` would double-count. `mapping.py` resolves
   the table via `cts_s.tables[...]`, which is why `osm_buildings_residential`
   had to be added to `CtsDemandBuildings.sources`.

3. `osm_buildings_extend_residential.sql` — the explicit column list. It has
   only been exercised against three synthetic census cells so far (see notes),
   so a second pair of eyes on the column/type mapping is worth more than usual.

4. Stage 3 of the cascade omits `building='yes'`. This is a measurement, not a
   preference: with it, 531,674 buildings return and the result lands at
   1.33 × Zensus 2022, back in the overestimation the intersection removes.

---

🔍 Reviewer Checklist

During your review, please check the following:

• Is the code clean, readable, and efficient? Are there any oddities or obvious inefficiencies?
• Does the code work as expected? (should already be verified by contributor)
• Do all tests pass? (see CI results)
• Is the documentation complete and up to date?
• Is CHANGELOG.rst updated accordingly?
• Is all necessary metadata complete and correct?
  • If metadata is pending: Is there an appropriate issue filed?

---

📝 Additional Notes (optional)

Two checkboxes are deliberately left unchecked. Here is exactly what was and
was not run.

What ran: the changed tasks individually against a Schleswig-Holstein
database — EthosBuilda.download, EthosBuilda.to_postgres,
filter_buildings_residential, extend_buildings_residential and
create_buildings_residential_zensus_mapping, with the numbers above.

What did not run: a full DAG run in test mode, and therefore none of the three
touched downstream datasets (LoadArea, CtsDemandBuildings, PowerPlants).
The local environment has a virtualenv built for a different branch — omi
without omi.dialects, which breaks import egon.data.metadata and with it
pipeline.py — so tox and an Airflow run were not possible. pipeline.py
compiles and EthosBuilda instantiates correctly inside a DAG context
(ethos_builda.download, ethos_builda.to-postgres). Reviewers should treat
the downstream changes as unexecuted, in particular point 1 of the focus list.

extend_buildings_residential and the census mapping were validated against a
stand-in table carrying the real schema and three real test cells (100 m
envelopes around commercial buildings with no residential building inside):
3 rows inserted, source = 'census_gap_fill', all six ETHOS columns NULL.
That is a structural validation of the column list, not a quantity validation.

OSM snapshot caveat. eGon-data pins germany-250101.osm.pbf, which is the
snapshot the reference implementation used — that is what makes an ID-level
comparison meaningful. The local workdir, however, held
schleswig-holstein-240101.osm.pbf, so the 618,208 above may rest on the 2024
extract rather than the 2025 one. download() fetches the pinned file (the
target filename differs), so a fresh test-mode run settles it. Until then, treat
the comparison against 607,490 as indicative rather than snapshot-matched.

Deliberate deviations from the reference implementation, all measured on SH,
so a reviewer can attribute differences rather than guess:

• Duplicate rule — the reference has none; here a building appears at most
  once and an ETHOS point claims at most one building, resolved geometrically
  and deterministically. Without it the primary key breaks. Affects 180
  buildings (0.031 %) and 9 points (0.002 %).
• Stage 2 measures to ST_PointOnSurface, not the centroid, to stay faithful
  to the reference code — the centroid would match 342 points more.
• Ancillary buildings are kept when ETHOS hits them (~500 in SH, 0.08 %): the
  point belongs to a real dwelling and merely sits on the wrong polygon, so
  dropping it would lose the building entirely.
• Care homes captured via amenity are lost. The previous residential filter
  also matched amenity IN ('retirement_home','nursing_home','assisted_living', 'group_home') and amenity='social_facility' with the corresponding
  social_facility tag. The reference whitelist carries those values too, but
  checks them against the building column, where OSM almost never puts them
  (8 buildings in SH). Measured on SH: the old clause matched 389 buildings,
  of which 80 are no longer residential — all building='yes', 50
  social_facility and 30 nursing_home; the other 309 are picked up by the
  intersection anyway. The loss arises only from the combination of dropping
  building='yes' in stage 3 and checking the whitelist against building
  alone. All 80 remain in osm_buildings_filtered, so they stay in the
  downstream union and merely count as CTS rather than residential. A targeted
  amenity branch in stage 3 would recover exactly those 80 without
  reintroducing building='yes' — say so in review if you want it.
• The 10 m threshold was recomputed, not assumed: in EPSG:3035 without the
  reference implementation's per-state decomposition the knee stays at 9 m, and
  up to 10 m the reference matches 1,010 points against 1,014 here.

Metadata. oemetadata resources for both tables
(society_egon_ethos_builda_buildings,
openstreetmap_osm_buildings_residential) are written and validated as
complete against the assembler, but they are not in this PR: the resource
store exists only on features/#1177-update-to-oemetadata-v2 (20 resource YAMLs
there, none on dev), so they will be added on that branch. Worth noting for
the metadata checkbox above: the store's dataset template declares
dl-by-de/2.0, which a resource without its own block inherits silently — the
result here is a Derivative Database of two ODbL-1.0 sources (OSM and ETHOS) and
must declare ODbL-1.0 explicitly. TABULA attribution is mandatory, not
courtesy.

Full-Germany run and validation against the reference results per federal
state and Zensus 2022 are not part of this PR; they run on a workstation
afterwards. The intersection stages are cheap on SH, but stage 2 is a nearest
neighbour over ST_DWithin and the building count rises by roughly a factor of
28 nationwide, so runtime there is estimated, not measured.

Out of scope, deliberately: the missing residential tags in
osm_buildings_filter.sql. bungalow alone is 60 % of the gap between the two
tables and stands in the reference whitelist but not in eGon's filter list. That
is a real bug, but it predates ETHOS, has nationwide effects on CTS
distribution and PV potentials, and needs its own validation — it will be filed
separately.