Writing layers¶

A layer is defined in a small Python script that imports openwater, describes the layer, and exports it into the viewer's data directory. Layers that are shown together form a project, also defined in the script: the viewer shows one project at a time, with the project's layers in the project's order. The Kiel study is in layers/kiel.py; use it as a template.

A minimal script¶

import openwater as ow
from openwater import ops, osm

KIEL = ow.Region("kiel", "Kiel", '["boundary"="administrative"]["de:amtlicher_gemeindeschluessel"="01002000"]')

benches = ow.Layer(
    id="kiel-benches",
    title="Benches (Kiel)",
    description="Public benches from OpenStreetMap.",
    region=KIEL,
    source=osm.select('nwr["amenity"="bench"]'),
    keep_osm_tags=True,
    attributes=[
        ow.Attribute("osm_id", "string", "OSM object", ops.osm_id()),
        ow.Attribute("backrest", "string", "Backrest", ops.tag("backrest", default="unknown"),
                     values={"yes": "With backrest", "no": "Without backrest", "unknown": "Not tagged"}),
    ],
    sources=[ow.Source("osm", "OpenStreetMap", "ODbL 1.0", attribution="© OpenStreetMap contributors")],
)

project = ow.Project(
    id="kiel-benches",
    title="Benches in Kiel",
    description="Where to sit down in Kiel.",
    layers=[benches],          # drawing order: the first is drawn on top
)

if __name__ == "__main__":
    ow.run([project])          # exports the layers and projects/kiel-benches.json

Run it with make layers SCRIPTS=layers/benches.py, or without Docker after pip install -e openwater:

python layers/benches.py                # export into data/viewer, using the download cache
python layers/benches.py --check        # only check the definitions
python layers/benches.py --refresh      # download OSM data again
python layers/benches.py --only kiel-benches --data-dir /srv/viewer-data

Option	Default	Meaning
`--data-dir`	env `OPENWATER_DATA_DIR`, else `data/viewer`	The viewer's data directory
`--cache-dir`	env `OPENWATER_CACHE_DIR`, else `data/cache`	Download cache, shared by all scripts
`--refresh`	off	Download again instead of using the cache
`--only ID`	all	Export only these layers (dependencies are still built)
`--check`	off	Validate the definitions, no download, nothing written

The new project appears in the viewer's Project list (and under Settings → Projects). ow.run([layer_a, layer_b]) without a project exports the layers only; the map can't show them then. A layer may belong to several projects, e.g. a project that compares two regions can list layers of both: define it in a script that imports the other scripts' layers.

The study template¶

The Offene Quelle study (boundary, drinking water, attractors, pedestrian potential, hydrants) is defined once in layers/_study.py. A region script only names the region and its population and transit data, and gets one project with the five layers:

import openwater as ow
import _study

REGION = _study.german_city("flensburg", "Flensburg", ags="01001000")
PROJECT = _study.project(REGION, slug="flensburg", population=_study.ZENSUS, transit=_study.GTFS_DE)

if __name__ == "__main__":
    ow.run([PROJECT])

slug becomes the project id and the start of the layer ids (flensburg-hydrants). The project draws drinking water and hydrants on top, then the attractors (off at start), the pedestrian potential and the boundary. _study.layers(...) returns the five layers alone, for projects of your own. Outside Germany, define a _study.Population (a population grid dataset, the radius that assigns its points to the 100 m cells, a description) and a _study.Transit (a GTFS dataset), see layers/zuerich.py. Files starting with _ are helpers: make layers and make check skip them. The regions and their results are compared in Regions compared.

Building blocks¶

Class	Purpose
`Project(id, title, description, layers, hidden=[])`	Layers shown together. `layers` in drawing order (first on top); `hidden` are listed but switched off at start. Exported as `projects/<id>.json`
`Region(id, name, osm_area)`	The area: an Overpass filter that selects the boundary relation
`Layer(...)`	One exported layer: id (lowercase, hyphens), title, description, region, `source` (where the features come from), attributes, rule sets, parameters, assumptions, planned items, sources. `keep_osm_tags=True` adds all raw tags as `osm:<key>`; `keep={...}` exports only features matching these conditions (e.g. grid cells with a value)
`Attribute(id, type, label, op, ...)`	One property of every feature. `type`: `string`, `integer`, `number`, `boolean`. Optional: `description`, `unit`, `values` (known values with labels), `closed=True` (only those values), `optional`
`Value(label, description)`	Label and explanation of a value; a plain string means label only
`RuleSet(id, title, description, rules)` / `Rule(id, title, description, where, source, status)`	Named conditions; `status` is `assumption`, `verified` or `rejected`
`Parameter(value, description, unit)`	A value the layer is based on, e.g. a distance; exported so viewers can use it (e.g. as a circle radius)
`Statement(id, title, description, status)`	An assumption behind the layer
`Planned(id, title, description, step)`	Something defined in the method but not implemented yet
`Source(id, title, license, ...)`	A data source with license and attribution

Feature sources (source=):

Source	Features
`osm.select('nwr[...]', ...)`	OSM elements inside the region, as points (ways and relations become their centre point)
`osm.region_boundary()`	The region's outline as a line
`grid.cells(size_m=100)`	Square cells covering the region (clipped to its boundary), in EPSG:3035 and aligned with the Zensus 2022 grid. Cell ids look like `CRS3035RES100mN5967800E4337500`

Datasets are external point data that operations can use (not features of a layer themselves). Files are downloaded once into the cache (data/cache/downloads/), extracted points are cached in data/cache/datasets/:

Dataset	Points
`datasets.csv_points(id, title, url, x, y, value, crs=…, member=…, delimiter=…, offset=0)`	Rows of a CSV file (also inside a ZIP; `member` defaults to its only CSV file), with coordinates in any CRS and one numeric value. `delimiter=None` detects it from the header. `offset=50` moves lower-left corners of 100 m cells to their centre. Examples: Zensus 2022 (Germany), BFS STATPOP (Switzerland)
`datasets.inspire_grid(id, title, url, member=…)`	An INSPIRE Population Distribution file (GML, also inside a ZIP): one point per grid cell (`…CRS3035RES100mN…E…`) at its centre, with its value. Example: Statistik Austria's 100 m population grid
`datasets.gtfs_departures(id, title, url, day=None, weekday=1)`	Stops of a GTFS timetable, valued by their departures on one day (default: the next Tuesday the feed covers). Reading a Germany-wide feed takes about a minute

Operations¶

Every attribute gets its value from one operation. Operations work for any layer; there is no layer-specific code.

Operation	Value
`ops.osm_id()`	OSM element reference, e.g. `node/123` (every layer needs an `osm_id` attribute)
`ops.tag(key, default=None)`	The OSM tag's value, cast to the attribute type; `default` if the tag is missing (without a default the attribute is absent then)
`ops.matching_rules(rule_set)`	Comma-separated ids of the rules in the set that match; empty if none
`ops.none_matched(rule_set)`	`True` if no rule in the set matches
`ops.nearest(layer, where=None)`	Straight-line distance in metres to the nearest feature of another point layer, optionally only features matching `where`; absent if there is none
`ops.classify([(value, where), ...], default)`	The value of the first case whose conditions hold, else `default`
`ops.feature_id()`	The feature's id: OSM reference or grid cell id
`ops.sum_within(source, value=None, radius_m=300, decay="linear", where=None)`	Sum of `value` over the points of a layer or dataset within `radius_m` (each point counts 1 without `value`). `decay="linear"`: a point counts fully at the feature and not at all at `radius_m`; `"none"`: all count fully
`ops.index({attribute: weight, ...}, cap_percentile=95, scale=100)`	Combines several numbers into one index: each attribute is divided by its 95th percentile over the layer (non-zero values, capped at 1), then the weighted mean is scaled to 0–100
`ops.value_at(layer, value)`	The value of an attribute in the polygon of another layer that contains the feature, e.g. the grid cell a hydrant lies in; absent outside

Attributes are computed in the order they are listed, each over all features before the next. An operation may only use attributes listed before it; --check reports violations. So list tag attributes first, then rule results, then distances, sums, indices and classifications.

Conditions¶

Used in rules (where=), in ops.nearest(where=...) and in ops.classify cases. Each key is an attribute id; all conditions must hold.

Condition	Holds if the attribute…
`"underground"`	equals the value
`["pipe", "pond"]`	equals one of the values
`ops.not_empty()`	is present and not `""`
`ops.empty()`	is absent or `""`
`ops.at_least(x)` / `ops.at_most(x)`	is a number ≥ x / ≤ x

A condition on an absent attribute is false (except empty). With default="unknown" on tag attributes, a missing tag never matches a rule, so features with incomplete tags are not excluded by accident.

Example: classification in three steps¶

From layers/kiel.py:

ow.Attribute("excluded_by", "string", "Excluded by", ops.matching_rules("exclusion")),
ow.Attribute("dist_public_water_m", "integer", "Nearest public drinking water (straight line)",
             ops.nearest(drinking_water, where={"public": True}), unit="m"),
ow.Attribute("status", "string", "Potential status",
             ops.classify([
                 ("excluded", {"excluded_by": ops.not_empty()}),
                 ("candidate_covered", {"dist_public_water_m": ops.at_most(WALK_M)}),
             ], default="candidate_gap"),
             closed=True, values={...}),

drinking_water is another Layer object in the same script; openwater builds it first.

Example: a grid index from several sources¶

From layers/kiel.py (pedestrian potential):

ZENSUS = datasets.csv_points(id="zensus2022-population-100m", title="Zensus 2022 population, 100 m grid",
    url="https://www.destatis.de/static/DE/zensus/gitterdaten/Zensus2022_Bevoelkerungszahl.zip",
    member="Zensus2022_Bevoelkerungszahl_100m-Gitter.csv", delimiter=";",
    x="x_mp_100m", y="y_mp_100m", value="Einwohner", crs="EPSG:3035")
GTFS = datasets.gtfs_departures(id="gtfs-de-nv", title="Public transport departures",
    url="https://download.gtfs.de/germany/nv_free/latest.zip")

potential = ow.Layer(
    id="kiel-pedestrian-potential", ..., source=grid.cells(100), keep={"potential": ops.at_least(1)},
    attributes=[
        ow.Attribute("cell_id", "string", "Grid cell", ops.feature_id()),
        ow.Attribute("attractors", "number", "Attractors nearby", ops.sum_within(attractors, value="weight", radius_m=300)),
        ow.Attribute("transit", "number", "Departures nearby", ops.sum_within(GTFS, radius_m=400)),
        ow.Attribute("residents", "number", "Residents", ops.sum_within(ZENSUS, radius_m=60, decay="none")),
        ow.Attribute("potential", "number", "Pedestrian potential",
                     ops.index({"attractors": 0.5, "transit": 0.3, "residents": 0.2})),
        ...
    ])

attractors is another layer of the same script (points with a weight per category). The hydrants then get ops.value_at(potential, "potential").

When a layer needs something new¶

If an analysis step can't be expressed with these operations, add a new operation to openwater/src/openwater/ops.py (or a new source or dataset type) that works for any layer, e.g. "value of a raster at the feature", never code for one specific layer. Add it to the layer format's list of sources (openwater/spec/layer.schema.json) and a test in openwater/tests/.