Skip to content

EC-FR: the 2018 campaign, from EuroCrops - #288

Merged
m-mohr merged 6 commits into
split/lv-backfillfrom
split/ec_fr
Sep 17, 2026
Merged

m-mohr merged 6 commits into
split/lv-backfillfrom
split/ec_fr

Conversation

@ivorbosloper

@ivorbosloper ivorbosloper commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Based on #286, which adds the split_multipart this uses.

IGN never released an RPG archive for 2018 — the downloads run 2017 and 2019 onwards — so fr has a gap there. EuroCrops publishes that campaign with its HCAT columns already resolved, so this is a collection of its own rather than a variant of fr.

  • 9,517,878 parcels, ID_PARCEL unique. 13 are multi-part and are split before the id is checked.
  • SURF_PARC is rounded to 0.01 ha, so 4,913 parcels under 50 m² read as zero and take their area from the geometry.
  • EuroCrops converters now download from Zenodo with requests: it answers with two Content-Type headers, which the aiohttp client behind the base class rejects with a 400. No ec_* converter could download on a cold cache.

Closes fieldsoftheworld/harmonized-field-data-catalog#13 once published.

ivorbosloper and others added 3 commits September 15, 2026 20:07
IGN never released an RPG archive for 2018; the downloads run 2017 and 2019
onwards. EuroCrops publishes it with the HCAT columns already resolved, so the
collection is theirs rather than a variant of fr.

9,517,878 parcels, ids unique. SURF_PARC is rounded to 0.01 ha, so 4,913 parcels
under 50 m2 read as zero and get their area from the geometry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVx9uQV2QPM8ecPAY3ZjXG
Zenodo answers with two Content-Type headers and the aiohttp client behind the
base class rejects that with a 400, so no ec_* converter can download on a cold
cache. The failed attempt also left a zero-byte file behind, which the next run
would have taken for a cached archive.
@ivorbosloper
ivorbosloper force-pushed the split/ec_fr branch 2 times, most recently from 59e4ed0 to 2ce50ce Compare September 15, 2026 18:07
@ivorbosloper
ivorbosloper changed the base branch from main to split/lv-backfill September 15, 2026 18:08
Splitting early only makes that possible; it does not do it. Nine parcels are
multi-part, so thirteen rows shared an id.
Comment thread fiboa_cli/datasets/commons/ec.py Outdated
ivorbosloper and others added 2 commits September 16, 2026 16:15
Zenodo answers with two Content-Type headers. aiohttp rejected that until 3.13.5,
which skips the RFC 9110 singleton check in lax mode -- the default for responses
(aio-libs/aiohttp#12302). The lock held 3.13.3, so no ec_* converter could
download on a cold cache.

The requests fallback added for it goes again.
@m-mohr
m-mohr merged commit 49491a6 into split/lv-backfill Sep 17, 2026
@m-mohr
m-mohr deleted the split/ec_fr branch September 17, 2026 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fr 2018: no RPG archive exists, but EuroCrops publishes FR_2018

2 participants