FR: editions 2017-2024, and the crop codes they need - #234
Open
ivorbosloper wants to merge 2 commits into
Open
ivorbosloper wants to merge 2 commits into
ivorbosloper wants to merge 2 commits into
Conversation
This was referenced Sep 11, 2026
ivorbosloper
force-pushed
the
split/fr
branch
2 times, most recently
from
September 12, 2026 15:10
e2ee28e to
3fcdcf5
Compare
ivorbosloper
changed the title
Three things, all found by converting the years the catalog was missing. **The crop mapping stopped at 2018.** EuroCrops splits France over two tables — fr_2018.csv holds that campaign's code list and fr_other_years.csv the codes that appeared later — and the converter named only the first. 725,902 fields of the 2024 edition (7.50%) carried a crop code that matched nothing. `ec_mapping_supplements` lets a converter name tables that fill the gaps in its main one; `crop:code_list` keeps pointing at the main table, which is the one that describes the source's own codes. **JAC was still missing from both.** It is the single fallow code that replaced J5M, J6S and J6P in the 2023 campaign, and at 604,122 fields it is the most common code in the whole file. fiboa.org/code/fr/fr_supplement.csv now carries it and ten others, each settled by a sibling code in EuroCrops' own tables rather than by judgement — JAC beside J5M and JNO, FVP beside FVL and FFO, the aromatic and medicinal codes beside PPA and PPP. Ten codes that no existing row settles stay NULL rather than being guessed at. **RPG's own parcel id does not identify a field in 2024.** 361 ids cover 736 of its 9,679,888 rows: 231 of those rows are exact duplicates and the rest are two declarations sharing an id — parcel 10308017 is declared both BOR and PTR, each 0.01 ha. It is published as `parcel_id` in every edition now, and `id` falls back to the row index in an edition where it repeats. Result over the seven editions: no field without an HCAT code up to 2022, 0.54% in 2023 and 0.63% in 2024, against 7.50% before. Field counts are unchanged, which is what a mapping-only change should do. The test pins the 2022 variant, which is what the fixture archive holds; the converter's default is the newest edition. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DVx9uQV2QPM8ecPAY3ZjXG
ivorbosloper
force-pushed
the
split/fr
branch
from
September 12, 2026 16:12
3fcdcf5 to
75054d2
Compare
crop:code_list is published in the data, and it pointed at EuroCrops' fr_2018.csv while the mapping came from three files: 640,000 fields of 2024 carry a code that list does not contain, JAC (jachere) alone 604,122 of them. The three are merged into https://fiboa.org/code/fr/fr.csv, 333 rows, each recording where it came from. The mapping is unchanged except for CSE, the one code both EuroCrops tables carry: they disagree, and until now fr_other_years won by list order alone. The official RPG reference reads "Chanvre sans etiquette conforme", so it maps to hemp, not to not_known_and_other. ec_mapping_supplements existed for this converter only, and goes with it.
m-mohr
reviewed
Sep 15, 2026
| # RPG's parcel id identifies a field in every edition but 2024, where | ||
| # 361 ids cover 736 of the 9,679,888 rows. It is published as parcel_id | ||
| # throughout; id falls back to the row index where it repeats. | ||
| gdf["parcel_id"] = gdf["id_parcel"] |
Contributor
There was a problem hiding this comment.
Same as in the other PR, should the ID be consistent across variants?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Handle 2017-2024 properly, with a single code-list, fixed mapping, unique ids