Skip to content

fix: delete only cache tiles that can touch the previous footprint (MAPCO-12035) - #124

Open
CL-SHLOMIKONCHA wants to merge 6 commits into
masterfrom
fix/MAPCO-12035-cache-deletion-intersection
Open

CL-SHLOMIKONCHA wants to merge 6 commits into
masterfrom
fix/MAPCO-12035-cache-deletion-intersection

Conversation

@CL-SHLOMIKONCHA

@CL-SHLOMIKONCHA CL-SHLOMIKONCHA commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor
Question Answer
Bug fix ✔
New feature ✖
Breaking change ✖
Deprecations ✖
Documentation ✖
Tests added ✔
Chore ✖

Related issues: MAPCO-12035

Further information:

Problem. The update cache deletion job walks the whole update area from zoom 0 to the configured max zoom (21). An update that adds a new area next to an existing layer therefore produces millions of deletion tiles, almost all above zoom 16, for ground that was never served and so never cached in redis.

Change.

  • updateJobHandler: at init, before the polygon parts are merged into the catalog, the current catalog footprint is stored on the job as additionalParams.previousFootprint. This is best effort; if the catalog lookup fails a warning is logged and the deletion falls back to the full walk.
  • cacheDeletionJobCreator: with a previous footprint on the job, the walk runs per zoom and clips the update area's tile ranges to the tiles touching the previous footprint's bbox. The box uses closed bounds (ceil - 1 on the min side, floor on the max side), so an edge lying exactly on a grid line claims the tile on either side, the same rule footprintToTileRanges applies to footprints. This is plain range arithmetic in grid units, exact at any latitude, with no buffer. Because the bbox of an irregular footprint also holds ground the layer never covered, each clipped range is then checked against the footprint itself with booleanIntersects and dropped when its ground does not touch it. The first zoom with no surviving range ends the walk, because a child tile cannot be shared by both areas when its parent is not.
  • jobParameters.schema: previousFootprint (Polygon or MultiPolygon, optional) on the update and swap finalize params.

What gets deleted. The result is exactly the update tiles whose squares touch the previous footprint, at every zoom, which is every tile MapProxy could have rendered for the old layer inside the update. Whether a given tile is actually in redis is not known here; the cleaner unlinks the key either way. Tiles MapProxy may have served beyond the catalog footprint are left to the cache TTL, see below.

Example. A layer near 34.3°E 31.35°N updated with a disjoint polygon near 35.5°E 33.9°N, 2.7° away, now produces four ranges, one tile per zoom from 0 to 3, instead of about 1.9 million tiles across zooms 0 to 21. From zoom 4 on no tile covers both footprints and the walk stops.

Trade-offs, kept on purpose.

  • Zooms above the layer's resolution inside the overlap are still enumerated up to max zoom.
  • Tiles outside the previous footprint that MapProxy upscaled into redis are left to the cache TTL.

Tests. Unit tests cover the fallback walk, a disjoint update (walk stops at the last zoom with a shared tile, exhaustive oracle for tiles touching both), an overlapping update (reaches max zoom but stays inside the tiles touching the previous bbox), the same at latitude 32, an update in the notch of an L-shaped layer (stops early, all tiles touching both arms covered), and the footprint recording and its fallback in the update handler.

…APCO-12035)

The update cache deletion job walked the whole update area from zoom 0 to
the configured max zoom, so an update that adds a new area next to an
existing layer produced millions of tiles above zoom 16 for ground that
was never served and never cached.

The update job now records the catalog footprint as previousFootprint at
init, before the polygon parts are merged. The cache deletion creator
clips the update area per zoom to the previous footprint buffered by one
tile diagonal, and stops the walk at the first zoom whose clip is empty,
since a deeper tile cannot touch both areas when its parent does not.
When the footprint cannot be recorded the whole update area is walked as
before.

The buffer is taken in grid degrees: turf measures along the globe, and a
buffer of many degrees around a footprint away from the equator bends
around the sphere and misses the footprint, which ended the walk after
zoom 0 for any real layer.
… coarse zooms (MAPCO-12035)

turf measures a buffer along the globe, which is a superset of the grid
distance as long as the buffer is not wide enough to bend around the
sphere. The coarse zooms whose reach exceeds ten degrees are therefore
walked over the whole update area, a handful of tiles that are nearly
always shared, and the deeper zooms buffer the footprint as it is.

This removes the flattening of the footprint onto an equatorial patch
and the coordinate mapping that came with it.
… clip at coarse zooms (MAPCO-12035)"

This reverts commit fd9d42f.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant