Building road-surface tiles¶

The map's road-surface layer answers "what is under my tyres?" for every road OpenStreetMap knows about, and just as usefully "which roads has nobody answered for yet?". This is how that layer is built.

It shares its source with Harvesting OSM coverage, the same Geofabrik extracts, the same osmium tags-filter step, and diverges completely after that. Coverage produces points and loads them into PostGIS. Surface produces lines, and touches no database at all: points go through coverage_poi because the serve endpoints and the dedupe need SQL, and lines need neither, so ways stream from the PBF straight to GeoJSONL and on into tippecanoe. That is what makes country-scale line data affordable.

Running it¶

make surface-tiles regions=$(make -s coverage-regions)

One pass over each region's PBF produces three artifacts, because they answer three different questions and a vector tile is fetched whole, a layer that is off by default still costs its bytes to every rider who fetches the tile it is folded into:

Artifact	What it holds	Zooms
`surface.pmtiles`	every way with a `surface` tag, canonicalised into the contract's surface classes	z10-13
`surface-todo.pmtiles`	ways with no surface tag, in the classes where the answer is genuinely unknown	z11-13
`surface-gaps.pmtiles`	one square per ~6 km with the km of unrecorded network inside it	z4-11

The classified floor is z10 because the build runs with no tile-size limit, so a low-zoom tile carries every classified way under it: measured over Wallonia, one tile was about 1.1 MB at z8 against roughly 90 KB at z10, and a screen is about sixteen tiles at any zoom. Below z10 the gaps grid answers the planning question instead. The client carries the same floor (CLASSIFIED_MIN_ZOOM), and a cross-language test pins the two together.

Why the "needs recording" arm is not simply every untagged road¶

An arm holding every untagged road costs about as much as the entire classified layer: measured on Belgium, the two came out within a few percent of each other. What decides the set instead is what those roads actually turn out to be. Of Belgian ways somebody has tagged, these are the shares that came back unpaved:

highway	unpaved when tagged	on the to-do list?
`track`	88.2 %	yes
`path`	50.1 %	yes
`unclassified`	6.8 %	yes
`living_street`	20.0 %	no
`residential`	7.2 %	no
`tertiary`	2.5 %	no
`secondary`	0.5 %	no
`primary`	0.1 %	no
`cycleway`	0.0 %	no

Measured on our own Belgian extract: of the ways somebody has tagged, this is the share that came back unpaved. A rider sent to record an untagged primary is being sent to confirm asphalt, one tagged primary in a thousand is anything else, and because mappers tag the surprising road first, an untagged one is safer still than its bar suggests. Only track, path and unclassified are kept, which holds the arm to under a third of the bytes while making every line in it a road where riding actually settles something. Note it is not a threshold: unclassified is kept at 6.8 % while living_street is dropped at 20 %. An unclassified road is a rural lane where the answer genuinely varies; a living street is in a town, and its 20 % is mostly setts nobody rides for the surface.

The set is contract data (surface.todo.highways), not code, so widening it is a rebuild rather than a release.

The grid: "where should I scan?" without shipping roads¶

Below z11 the same question is answered by squares instead of geometry. Each cell carries the kilometres of unrecorded to-do network inside it, that as a share of the cell's network, and a road count, and a cell where everything is already recorded is omitted, because a square drawn over finished work reads as "there is something to do here", which is the one thing the layer must never say.

Kilometres, not way counts: a way is an arbitrary unit. A rural track runs unbroken for 3 km while a village lane is split at every junction, so counting ways would make dense villages look like more work than the gravel network around them.

The expensive half of a continental build is walking the ways, and it happens once. Each way is written out and dropped as it arrives, so a country never accumulates in memory, the reason a 5 GB France extract builds on a laptop-sized machine at a peak of about 2 GB, most of which is osmium's node-location index rather than our ways. Only the grid keeps state, and it is bounded by area rather than by roads: about a thousand cells per country, three numbers each.

The grid stops at exactly the zoom the lines start, and both sides are pinned to that number by a cross-language test, a mismatch would leave a band of zoom showing neither.

Running it at continental scale¶

Two knobs exist for exactly that:

# One region at a time: a failure costs one country, not the queue.
make surface-tiles regions=europe/germany ARGS=--extract-only

# Then tile every extract on disk, once, from the PBFs already in the workdir.
make surface-tiles offline=1 regions=$(make -s coverage-regions)   # every onboarded region, as above

ARGS=--extract-only stops after the GeoJSONL. Without it, every per-region pass ends in a full tippecanoe build of the countries done so far: tiling Germany a dozen times to discard each result.
offline=1 (COVERAGE_PBF_OFFLINE=1 inside the container) uses the PBFs already in the workdir, and a region whose PBF is not there fails the run instead of fetching it. A PMTiles archive cannot be appended to, so adding one country means tiling all of them again, and that pass walks the whole region list. It is explicit rather than inferred from a warm cache: a run that skips the refresh has to say so, or "the data is current" quietly becomes "the data is whatever was here last time".

The per-region extract is cached and reused only while it is newer than both the PBF it came from and the contract that shaped it, adding a highway class changes what should be in the file while leaving the PBF untouched, and a stale extract would then be tiled as if it were current.

Each run reports its own peak memory. That matters here: ways stream to disk rather than accumulating, and measuring from outside the container (/usr/bin/time docker compose run) measures the docker client, which is how a continental build can appear to use 12 MB.

Publishing: one file per country, one manifest¶

A finished build is uploaded by the same run that made it, one file per country, under a versioned prefix that carries the country code:

surface/be/20260924-1013/classified.pmtiles
surface/be/20260924-1013/todo.pmtiles
surface/lu/20260924-1013/classified.pmtiles
surface/lu/20260924-1013/todo.pmtiles
surface/gaps/20260924-1013/gaps.pmtiles   <- one world file, no country split
surface/manifest.json                     <- stable key, repointed last

Symfony reads that manifest server-side (App\Coverage\SurfaceManifest, cached one hour, 30-second negative TTL, 5-second timeout) and injects the per-country URLs into the map page. So a rebuild goes live within the hour with no config change, no cache clear and no deploy, the same contract as the coverage artifact.

Three properties are load-bearing:

The artifacts are immutable, the manifest is not. Versioned keys are Cache-Control: immutable for a year; the manifest carries max-age=300. A rebuild must never overwrite a live artifact, a rider mid-session holds an offset into the PMTiles directory, and changing the bytes underneath them reads as corruption, not as an update.
A country's own two arms move together. One stamp, one manifest entry, one publish, for that country. They are two readings of a single walk over the same ways, and serving one build's classified skin beside another's to-do arm would tell riders that roads they have just recorded still need recording. A publish only ever replaces the countries it built: another country's entry is left exactly as it was. The world gap grid is merged from the current cell counts of every onboarded region, in the run or not, and is left as it was while any onboarded region has none.
The env vars pin, they do not configure. ROAD_SURFACE_TILES_URL, ROAD_SURFACE_TODO_URL and ROAD_SURFACE_GAPS_URL override the manifest when set, serving one world archive to every country: the hatch for bisecting a rendering problem or serving an artifact that was never published. Empty (the default) means "follow the manifest", and ROAD_SURFACE_MANIFEST_URL names the manifest itself. Leaving a pin set by accident is how a map ends up serving last month's tiles.

Old builds are pruned to the newest three per country, whole prefixes at a time, never one arm of a build, which would leave a manifest pointing at a layer that is no longer there. Three rather than coverage's four because a surface build runs to several gigabytes per country against coverage's well under one, and the reason to keep any is a fast rollback, not history.

A half country is never published, and dropping one is explicit¶

A PMTiles archive cannot be appended to, so a country's tiling step builds from exactly the regions it was given. Three onboarded countries span more than one Geofabrik extract (US = california + colorado, CA = british-columbia + quebec, GB = great-britain + ireland-and-northern-ireland). When a run does not include every one of a country's onboarded regions, that country is skipped with a log line rather than published from a partial extract:

make surface-tiles regions=north-america/us/california   # Colorado not included

There is no shrink guard to reason about, because a run can only ever add or refresh the countries it was given; it can never take another country's live entry off the manifest. The only way to drop a country on purpose is naming it:

make surface-tiles regions=$(make -s coverage-regions) ARGS="--retire lu"

--retire <cc> is the only thing that ever removes a country's entry from a manifest. Use ARGS=--no-publish for size experiments on one country without touching the live manifest at all.

The routes build, and why it runs first¶

The cycle-route network layer (signed route=bicycle/route=mtb corridors plus knooppunt numbers) is a sibling build with the same shape, same extracts, no database, its own artifact and manifest:

make routes-tiles regions=$(make -s coverage-regions)   # the same full list as the surface build

It differs from the surface build in one structural way: routes are OSM relations, and a way cannot know its relations, so the extractor walks the filtered PBF twice, relations first for membership, then ways with locations (pipeline/coverage/routes.py). It publishes, per country, routes/<cc>/<stamp>/routes.pmtiles + a stable routes/manifest.json (read by App\Coverage\RoutesManifest; pin: ROUTES_TILES_URL), with the same immutable-artifact, complete-region-set and whole-prefix-prune rules as above.

Order matters when rebuilding both. The routes run drops a routes_<region>_wayids.txt per region into the workdir, the set of ways that carry a signed route, and the surface build reads it to make its to-do arm route-aware (an untagged way on a signed route is homework whatever its highway class). The way-id file is a declared input of the surface extract, so a fresh routes run automatically invalidates the surface extracts it would change; run make routes-tiles first, make surface-tiles second, and the cache does the rest. A surface run with no way-id files still builds; it says so in the log, and the to-do arm is class-gated only.

Try it¶

Hands-on: read the manifest before you rebuild anything

Every claim on this page is visible in one file. Read it first, because it is also the file a publish repoints.

curl -s http://localhost:9100/cc-maps/surface/manifest.json \
  | jq '{version, countries: (.countries | keys), gaps: .gaps.url}'

{
  "version": 2,
  "countries": ["be", "lu", "nl"],
  "gaps": "http://localhost:9100/cc-maps/surface/gaps/20260924-1013/gaps.pmtiles"
}

One key per country that has ever been published, plus the one world gaps file. Read one country's own entry to see what it built:

curl -s http://localhost:9100/cc-maps/surface/manifest.json | jq '.countries.lu'

{
  "stamp": "20260924-1013",
  "built_at": "2026-09-24T10:13:49+00:00",
  "inputs": "52b31fe009cc87d4",
  "bounds": [5.731844, 49.448692, 6.529054, 50.182749],
  "counts": { "classified": 49737, "todo": 19514 },
  "tiles": {
    "classified": "http://localhost:9100/cc-maps/surface/lu/20260924-1013/classified.pmtiles",
    "todo": "http://localhost:9100/cc-maps/surface/lu/20260924-1013/todo.pmtiles"
  }
}

Two arms under tiles, which is the design this page opens with: one for the roads whose surface is recorded, one for the roads worth recording. The world gaps grid answers the same question below the classified floor. counts are kilometres, not way counts, for the reason the page gives.

Run the same build again with nothing changed, and inputs is why it costs almost nothing the second time:

make surface-tiles regions=europe/belgium,europe/netherlands,europe/luxembourg

[surface] BE: unchanged, not rebuilt
[surface] LU: unchanged, not rebuilt
[surface] NL: unchanged, not rebuilt

Each country's own fingerprint (inputs above) decided that on its own; no .pmtiles is uploaded and the manifest is not rewritten for any of the three. A country whose inputs would differ (its extract changed, or the contract or the border outlines near it did) is the only one that rebuilds and re-publishes.

Where to go deeper¶

A-road-surface.md: the rider-facing item: what a surface submission is, how a stretch is drawn, what moderators see.
map-and-search.md: how the three artifacts render, the legend row that governs all of them, and the zoom handover.
Harvesting OSM coverage: the point pipeline this shares its extracts with, and the md5 mirror-lag gotcha that bites both.