The map page, request by request¶
Open /map with the browser's network tab showing and you see about thirty
requests to four different places. This page says what each one is, who
answers it, and in what order they happen. It is the map read from the wire,
where Putting it on screen is the map read from the code.
Who serves what¶
Three parties answer the page. The application builds the HTML and answers every JSON call. A tile host holds the archives the pipeline built. A few external hosts supply the base map, satellite imagery, street-level photos and geocoding.
flowchart LR
P[Pipeline<br/>pipeline/coverage] -->|writes archives<br/>and manifests| T
A[Symfony app<br/>MapController, CoverageController] -->|reads the three manifests<br/>server side, once an hour| T[Tile host<br/>coverage, surface, routes<br/>.pmtiles archives + manifest.json]
B[Browser<br/>MapLibre GL + pmtiles protocol<br/>icons drawn in JS] -->|GET /map, then<br/>catalog, stamps, slices,<br/>boundaries, counts, drawer data| A
B -->|HTTP range reads,<br/>a few KB per tile| T
B -->|base map style and tiles,<br/>satellite, street-level, geocoding| X[External hosts<br/>OpenFreeMap, Esri, Mapillary, Photon]
Two facts on that picture explain most of what follows. The pipeline writes both database rows and tile archives. The browser reads the archives straight from the tile host, with no application server in that path.
Boot order¶
GET /map.web/src/Controller/MapController.phpreads the coverage, surface and routes manifests throughweb/src/Coverage/CoverageManifest.php,web/src/Coverage/SurfaceManifest.phpandweb/src/Coverage/RoutesManifest.php, each cached for an hour, and embeds the result in the page aswindow.CC_TILES: one archive URL per country per family ({coverage: {"be": {...}, "nl": {...}}, surface: {...}, routes: {...}}), or one entry under the key"*"when the artifact serves every country from a single archive (a v1 manifest, or a pin). The page also carries the catalog version tag, which catalog regions a link needs (window.CC_CATALOG_BOOT), the basemap icon registry, the viewer's home area and the UI strings.- Scripts. MapLibre GL and the pmtiles library from our own origin, then
web/assets/map/map.jsand its modules through the importmap. - MapLibre boots with the style
https://tiles.openfreemap.org/styles/liberty. That one request returns the style document; the base map tiles follow as the view needs them. - Stamps, then the regions on screen. The page preloads
/map/catalog/stamps.json, which is small and always revalidated.web/assets/map/catalog-load.jsreads it and fetches/map/catalog/region/{rid}.json?v=<stamp>for every region of the active scope, plus the regions a link points into. Nothing downloads the whole world: "Search everywhere" asks/v1/search?q=for our own items, and a picked hit loads its one region. - Icons.
web/assets/map/icons.jsdraws each pin on a canvas and hands the pixels to MapLibre withmap.addImage. No image file is requested. - Archives.
web/assets/map/tile-sources.jsowns the pmtiles protocol registration andmountInView(), called bycoverage.js,surface-tiles.jsandroutes-tiles.js: it adds apmtiles://source, one per country, for every country whose bounds meet the current viewport, from the URLs inwindow.CC_TILES(step 1). A country's source is added once, on the first view that reaches it, and never removed as the rider pans elsewhere. The pmtiles protocol reads each mounted archive's header once, then fetches only the tiles in view, each as an HTTP range request. Surface and routes sources mount only once their panel is first toggled on, not at boot. - Scope. When a region is chosen,
web/assets/map/spotlight.jsfetches/map/region/{slug}/boundaryfor the region and/map/scope/boundary?rids=for its neighbours, andweb/assets/map/coverage.jsfetches/map/coverage/counts?rids=for the letter counts in the icon rail. - On demand. Everything else is fetched when the visitor acts: a click on a point, a search, the drawer, the Curated facet, community actions.
Every request¶
Three tables, one per party. Paths are shortened to what tells them apart; the host is in the heading.
Answered by the application, MapController and CoverageController:
| Request | What it carries | Cached |
|---|---|---|
GET /map |
the page, with the window.CC_* values embedded |
private |
/map/catalog/stamps.json |
{ "<region id>": "<stamp>" }; key "0" is the rows that belong to no region |
no-cache, ETag, preloaded |
/map/catalog/region/{rid}.json?v= |
one region's rows, in the catalog's shapes | one day, ETag, URL-versioned |
/v1/search?q= |
our items by name, anywhere, each with its region, plus routes when the "Routes" switch is on (&routes=include); the "Search everywhere" half the region files cannot answer |
five minutes, ETag |
/map/region/{slug}/boundary |
one region's border as GeoJSON | public |
/map/scope/boundary?rids= |
several borders in one call | public |
/map/coverage/counts?rids= |
how many coverage points of each letter fall in the scope | one hour |
/map/coverage/poi/{type}/{id} |
one point's details for the drawer | public |
/map/coverage/photo/{type}/{id} |
the photo attached to a point | public |
/map/coverage/nearby?lat=&lng= |
points around a place, for the places panel | public |
/map/coverage/search?q= |
point search by name | public |
/map/heat.json |
heat points, off by default | public |
/map/best-of?season=&bike= |
ranked verified routes for the Curated facet | five minutes |
/map/item/{id}/history |
an item's change log for the drawer | one minute |
/map/item/{id}/hidden-photos |
photos withheld from an item, for curators | private |
/items/{id}/…, /routes/{id}/… |
community actions, confirmations, decisions | private, POST |
Read from the tile host by HTTP range, a few kilobytes per tile. Every
key is immutable: a new build is a new key. Each family publishes one file
per country (<cc> lowercase); only a country whose bounds meet the viewport
is ever requested.
| Archive | What it carries |
|---|---|
coverage/<cc>/<stamp>/points.pmtiles |
one country's coverage points, layer <letter>_<cc> |
surface/<cc>/<stamp>/classified.pmtiles |
one country's roads whose surface tag is known, canonicalised |
surface/<cc>/<stamp>/todo.pmtiles |
one country's roads whose surface nobody has recorded |
surface/gaps/<stamp>/gaps.pmtiles |
the same question as a world grid, for planning zoom |
routes/<cc>/<stamp>/routes.pmtiles |
one country's cycle-route network lines and knooppunt numbers |
External hosts, each tied to one feature:
| Host | Request | What it carries |
|---|---|---|
| OpenFreeMap | styles/liberty |
base map style document |
| OpenFreeMap | planet/<date>/{z}/{x}/{y}.pbf |
base map vector tiles |
| Esri | World_Imagery/.../{z}/{y}/{x} |
satellite tiles, only after the viewer turns satellite on |
| Mapillary | mly1_public/2/{z}/{x}/{y} |
street-level coverage lines |
| Mapillary | graph.mapillary.com/images |
"is there a photo here" lookups |
| Komoot | photon.komoot.io/api |
geocoding for the search box |
Every host the page may talk to is listed in the Content Security Policy, in
web/src/EventSubscriber/CspSubscriber.php. A new host is a CSP change first.
The stamps document is what keeps the catalog fresh without redownloading
it. A region's slice is a few tens of kilobytes; every region in one document
would be about a megabyte gzipped. A visitor downloads the slices of their own
area. A
curator's decision moves one region's stamp, and a visitor in that region
fetches that slice and nothing else. A stamp also moves once a day, because a
pin's freshness colour depends on the day. The database counts the changes
itself, by trigger, and the server keeps each built slice in Redis until its
stamp moves (docs/specs/catalog-data-model.md §9.1).
What you will not see, and why¶
- Icons. They are drawn on a canvas and pushed into MapLibre. There is no sprite file and no icon request.
manifest.json. The application reads it on the server. The browser only ever sees the archive URL that came out of it.- A
data:image/svg+xml,...entry with a crosshair path andfill=#333. That is MapLibre's own stylesheet inlining the icon of its "locate me" control. Nobody serves it. - Whole archives. An archive is tens of megabytes, but the browser reads a small header and then a few kilobytes per tile. The size the network tab shows is what was read, not the file.
Where the tile data comes from¶
flowchart LR
G[Geofabrik extract] --> H[harvest: filter, parse,<br/>diff-merge into coverage_poi]
H --> E[export one GeoJSONL<br/>per letter per country]
E --> K[tippecanoe, per country]
K --> AR[coverage/<cc>/<stamp>/points.pmtiles<br/>one file per country]
K --> M[coverage/manifest.json<br/>one entry per country:<br/>stamp, built_at, inputs, bounds, counts, tiles]
M -->|read on the server,<br/>once an hour| APP[app] --> PAGE[window.CC_TILES.coverage]
The surface and routes archives take the same path without the database: the extract is their input, one archive per country plus their own manifest is their output. A country rebuilds only when its own extract's fingerprint has changed since the live manifest, so most nights most countries are skipped. The stamp in a country's key is that country's own build time, so every rebuild is a new immutable URL for that country and the manifest is the only thing that moves. The runbooks in Data operations cover how the archives are built.
Development against production¶
| Development stack | Staging and production | |
|---|---|---|
| Application | localhost:8001 |
the site |
| Tile host | MinIO, localhost:9100/cc-maps/... |
the object storage behind COVERAGE_PUBLIC_BASE_URL |
| Base map, satellite, Mapillary, geocoding | the same external hosts | the same |
The page is identical. Only the two origins differ, and the network tab shows you which is which.