Skip to content

Architecture

Two independent programs share one thing: the layer format. openwater writes layers and projects in that format into the viewer's data directory; the viewer shows whatever projects it finds there. Neither knows anything about a particular layer.

flowchart LR
    subgraph OW["openwater (Python package)"]
        SPEC["openwater.spec<br/>layer format<br/>(layer + project schema)"]
        API["define · ops · osm<br/>build · export"]
    end
    SCRIPT["layers/kiel.py<br/>(layer script)"]
    OSM[("OpenStreetMap<br/>Overpass")]
    EXT[("Zensus 2022 grid<br/>GTFS timetable")]
    CACHE[("data/cache/<br/>overpass · downloads · datasets")]

    subgraph DIR["data directory (VIEWER_DATA_DIR)"]
        PKG["projects/&lt;id&gt;.json<br/>layers/&lt;id&gt;/<br/>layer.json · data.geojson"]
        VIEW["layers/&lt;id&gt;/<br/>style.json · tiles.pmtiles"]
    end

    subgraph VW["viewer"]
        SRV["server<br/>(projects, layers, tiles, styles)"]
        WEB["web app<br/>(map, layer style, Settings, About)"]
    end
    DOCS["docs<br/>(page per layer)"]

    SCRIPT -- imports --> API
    API --> CACHE
    OSM --> API
    EXT --> API
    API -- "writes, validated<br/>against" --> PKG
    SPEC -. defines .-> PKG
    PKG --> SRV
    SRV <--> VIEW
    SRV --> WEB
    PKG --> DOCS
Part Owns Knows of the others
openwater (openwater/) Defining layers and projects (Python API), feature sources (OSM, grid), datasets (CSV points, GTFS), download cache, computing attributes and rules, exporting layer packages and projects, and the layer format itself (openwater.spec) Nothing
Layer scripts (layers/*.py) The curated content: which OSM objects, which attributes, values, rules, assumptions, sources; which layers form a project, in which order openwater's API
Data directory All projects and layer packages plus the viewer's files for them –
viewer (viewer/) Showing projects, building tiles, map display, styles (colours, shapes, popups); site texts and defaults in viewer/viewer.yaml The layer format (schema files), nothing else
docs (docs/, docs-server/) This site, plus one generated page per layer, listed by project The layer format

The data directory

data/viewer/                     (VIEWER_DATA_DIR, any host path)
├── projects/
│   └── kiel.json                openwater: the project's layers in drawing order
└── layers/
    └── kiel-hydrants/
        ├── layer.json           openwater: what the layer is (attributes, values, rules, stats)
        ├── data.geojson         openwater: the features
        ├── style.json           viewer: how it looks (edited in the layer style)
        ├── tiles.pmtiles        viewer: vector tiles, rebuilt when data.geojson changes
        └── .tiles.json          viewer: checksum of the data the tiles were built from

Each layer package is self-contained: its own features, texts, rules, assumptions and sources, even if that repeats something another layer also contains. openwater only ever replaces layer.json and data.geojson, so a re-export keeps the layer's style.

From OpenStreetMap to the map

  1. A layer script (e.g. layers/kiel.py) defines layers with openwater's API and calls openwater.run(...). See Writing layers.
  2. Download: for OSM layers, openwater builds an Overpass query from the layer's selection and region. Responses are cached per query in data/cache/overpass/; layers or scripts with the same query share one download. Grid layers need the region's outline (also from Overpass). Datasets (Zensus grid, GTFS timetable) are downloaded once into data/cache/downloads/; the points extracted for a region are cached in data/cache/datasets/. --refresh downloads again.
  3. Build: the layer's source yields the features (OSM elements, or grid cells clipped to the region); the attributes are computed in the order they are defined, each over all features: tag values, matching rules, distances, sums of nearby points, indices, classifications, values from other layers. A layer another layer depends on (ops.nearest, ops.sum_within, ops.value_at) is built first, even if it isn't exported itself. keep= then drops features (e.g. empty grid cells).
  4. Export: openwater validates layer.json against the layer format and every feature against the attribute definitions. If anything is wrong, nothing is written. Otherwise data.geojson and layer.json are replaced atomically.
  5. Viewer: the server lists the layer, and builds tiles.pmtiles with tippecanoe the first time the tiles are requested after the data changed. The web app loads the layer's layer.json and style.json and draws it. See Viewer.
  6. Docs: the docs container generates one page per layer when it starts.

Classification of the Kiel hydrants

Defined in layers/kiel.py with generic operations; the layer's page under Layers shows the exact rules.

flowchart TD
    H["Hydrant from OSM"] --> R{"excluded_by not empty?<br/>(rule set exclusion)"}
    R -- yes --> EXC["status = excluded"]
    R -- no --> D{"dist_public_water_m ≤ 300 m?<br/>(nearest public drinking water)"}
    D -- yes --> COV["status = candidate_covered"]
    D -- "no, or none in region" --> GAP["status = candidate_gap"]

Updates and reproducibility

  • Refresh data: make layers ARGS=--refresh downloads OSM data again and re-exports. The viewer picks up the new data on the next page load and rebuilds the tiles; restart docs to regenerate the layer pages.
  • Reproducibility: data/cache/overpass/ keeps the exact Overpass responses. Without --refresh, a script produces the same layers again (apart from timestamps).
  • License: every layer package is derived from OSM and falls under the ODbL. Each layer lists its sources with attribution; the map shows them in its attribution line.

Known limitations

Limitation Effect Planned fix
Distances are straight-line, not walking routes Coverage is overestimated across rivers, railways and the Kiel Fjord A walking-distance method for ops.nearest (M4)
OSM has few drinking-water points mapped in Kiel (6, only 2 public) candidate_gap is strongly overstated Second source for fountains (Refill / Karte von morgen)
Pedestrian potential is modelled, not measured Ranking by footfall is an estimate; weights are assumptions Check against hystreet counts (internally) and counts from the city
Rules are assumptions Classification is not yet backed by regulation Requirements matrix from M2 research → status verified
No water-quality information (M3) Every network hydrant is treated as suitable Proxies such as dead-end streets; utility data
No demand data (population, heat) No ranking among candidates New operations for grid and raster data (M4); scoring (M5)
Overpass is suitable for single cities only No national run yet An .osm.pbf source for openwater