Skip to content

[BUG] osm_buildings_filter whitelist drops clearly residential buildings #1522

Description

@jh-RLI

Describe the bug

The tag whitelist in osm_buildings_filter.sql contradicts the one in
osm_buildings_filter_residential.sql and misses several OSM tag variants. As a
result, clearly inhabited buildings never reach
openstreetmap.osm_buildings_filtered, and they disappear from everything
downstream without a warning.

Three inconsistencies:

  • semidetached_house is whitelisted, the variants semidetached and
    semi_detached are not.
  • bungalow and farmhouse are on the residential whitelist in
    osm_buildings_filter_residential.sql, but not in osm_buildings_filter.sql.
  • Further unambiguously residential tags are absent altogether:
    dwelling_house, detached_house, houseboat.

Measured on a full Germany run: 134,725 buildings (0.71 %) that are
classified as residential do not appear in osm_buildings_filtered. The largest
groups carrying an unambiguously residential tag are bungalow (37,402),
cabin (6,591) and dwelling_house (4,849). The remainder is dominated by
outbuilding tags such as garage and shed — a separate matter, not this bug.

osm_buildings_filtered is the basis for load area aggregation, CTS building
selection, the MVGD building mapping and the rooftop PV potentials. A building
missing from it silently loses three things:

  1. Household load. loadarea/loadareas_add_demand_hh.sql joins peak loads
    against osm_buildings_filtered; with no geometry match the load drops out
    of demand.egon_loadarea. No warning, just less demand.
  2. Grid assignment. electricity_demand_timeseries/mapping.py writes no row
    to boundaries.egon_map_zensus_mvgd_buildings, so the building gets no
    MVGD/bus.
  3. Rooftop PV. power_plants/pv_rooftop_buildings.py intersects against the
    same set, so the building gets no rooftop potential.

To Reproduce

  1. Run the OsmBuildingsStreets dataset.
  2. Run:
SELECT b.building,
       count(*)    AS in_osm_buildings,
       count(f.id) AS in_osm_buildings_filtered
FROM openstreetmap.osm_buildings b
LEFT JOIN openstreetmap.osm_buildings_filtered f USING (id)
WHERE b.building IN ('bungalow', 'farmhouse', 'semidetached', 'semi_detached',
                     'dwelling_house', 'detached_house', 'houseboat')
GROUP BY b.building
ORDER BY in_osm_buildings DESC;
  1. in_osm_buildings_filtered is 0 for every row.

Expected behavior

Buildings carrying these tags end up in osm_buildings_filtered, the way
semidetached_house, terraced_house, static_caravan and plain
building=yes already do.


Screenshots

n/a


Environment:

  • OS: not relevant — the defect is in SQL, not in the runtime
  • Python version: not relevant
  • Scenario: independent of scenario
  • Dataset Boundary: Everything (the figures above); the gap is visible on
    Schleswig-Holstein as well

Proposed solution (optional)

Extend the whitelist in osm_buildings_filter.sql with the missing residential
tags, cross-checked against osm_buildings_filter_residential.sql so the two
whitelists stop drifting apart.

Two parts should be separated when doing this:

  • The three inconsistencies above are a straightforward correction.
  • Whether debatable tags such as cabin, stilt_house or villa belong on the
    list is a modelling decision, not a defect, and can be decided independently.

Additional context

The fix is not free. osm_buildings_filtered is also the CTS candidate set: in
census cells with CTS demand but no amenities, cts_buildings.py spreads the
demand over all filtered buildings in the cell, so adding buildings lowers
the demand per building. The effect is expected to be small, but it should be
measured rather than assumed — which is why this belongs in its own change
rather than riding along with #1310.

Part of #1311. Found while working on #1310, where residential buildings are
determined by intersecting with an external building dataset instead of by OSM
tag — which is how the gap became visible.

Possibly related: #1234 reports missing buildings in the residential layer. That
is a different table, but some of the reported gaps may share this cause. Not
verified.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    🐛 bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions