EC-SI: require only what the source actually fills

CHANGELOG.md

@@ -41,6 +41,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 - ES-CL: the ITACyL server is https-only, the 2025 shapefiles sit in province subfolders, and C_REFREC is the identifier
 - A test refuses a fixture above 5 MB, committed or merely lying in the fixture folder, because a failing convert test downloads the real source there
 - ES-MD: find RECINTO.shp wherever the archive puts it
+- EC-SI: stop requiring columns the source leaves empty, and require only what every parcel carries
 - EC-LV: stop requiring columns the source leaves empty
 - ES-AN: CD_USO is the land-use column, with the determination date from the variant year
 - HR: editions 2011-2024; the archives leave their CRS undefined and carry ARKOD's own parcel id, which the row index used to overwrite

fiboa_cli/datasets/ec_si.py

@@ -31,7 +31,11 @@ class Converter(EuroCropsConverterMixin, FiboaBaseConverter):
     }
 
     missing_schemas = {
-        "required": ["gerk_pid", "crop_type_class", "rastlina", "crop_lat_e", "color"],
+        # Four parcels of the 828,161 in the 2021 release carry no crop at all:
+        # SIFRA_KMRS, RASTLINA, CROP_LAT_E, COLOR and the EC_* columns are empty
+        # for them (and three parcels have no EC_NUTS3). Only the GERK parcel id
+        # is always there.
+        "required": ["gerk_pid"],
         "properties": {
             "gerk_pid": {"type": "uint64"},
             "crop_type_class": {"type": "string"},

tests/data-files/convert/ec_si/si_2021.csv

@@ -0,0 +1,158 @@
+original_code,original_name,latin_name,translated_name,HCAT3_name,HCAT3_code
+000,ni v uporabi,Not known,Not known,not_known_and_other,3399000000
+001,p?enica (jara),Triticum aestivum L.,Common wheat spring,spring_common_soft_wheat,3301010102
+002,r? (jara),Secale cereale L.,Rye (spring),spring_rye,3301010302
+003,pira (jara),Triticum spelta L.,Spring Spelt,spring_spelt,3301011002
+004,ajda,Fagopyrum esculentum Moench,Buckwheat,buckwheat,3301150200
+005,koruza za zrnje,Zea mays L.,Grain maize,grain_maize_corn_popcorn,3301010600
+006,koruza za sila?o,Zea mays L.,Silo Maize,green_silo_maize,3301090400
+007,tritikala (jara),X Triticosecale Wittmack (Triticum x Secale),Spring Triticale,spring_triticale,3301010802
+008,oves (jari),Avena sativa L.,Oats (spring),spring_oats,3301010502
+009,je?men (jari),"Hordeum vulgare L., spring barley",spring barley,spring_barley,3301010402
+010,proso,Panicum miliaceum L.,Proso millet,millet_sorghum,3301010900
+011,me?anice ?it (jara),Mixture of cereals (spring),Mixture of cereals (spring),spring_unspecified_cereals,3301011502
+012,son?nice,Helianthus annus L.,Common Sunflower,sunflower,3301060500
+013,oljna bu?a,Cucurbita pepo var.pepo,Winter squash pumkin,pumpkin_squash_gourd,3301140400
+014,oljna ogr??ica (jara),Brassica napus var.napus,Rapeseed spring,spring_rapeseed_rape,3301060402
+017,krmni bob,Vicia faba L. var. minor Harz,Field bean,beans,3301020100
+019,sladkorna pesa,Beta vulgaris L. subsp. vulgaris var. Altissima,Sugar beet,sugar_beet,3301290700
+020,krompir (pozni),Solanum tuberosum L.,potatoes (late),potatoes,3301030000
+022,krompir (zgodnji),Solanum tuberosum L.,potatoes (early),potatoes,3301030000
+024,sirek,Sorghum bicolor (L.) Moench,sorghum,millet_sorghum,3301010900
+025,trda p?enica (jara),Triticum durum Desf.,Durum wheat spring,spring_durum_hard_wheat,3301010202
+026,praha,Fallow land,Fallow land,fallow_land_not_crop,3301110000
+027,konoplja,Canabis sativa var.sativa,Cannabis,hemp_cannabis,3301061000
+028,lan,Linum usitatissimum L.,Flax,flax_linen,3301060701
+029,ukoreni??e hmeljnih sadik,The rooting of hop seedlings,The rooting of hop seedlings,hops,3301060200
+030,soja,Glycine max (L.) Merr.,Soybean,soy_soybeans,3301160000
+031,vrtni mak (jari),Papaver somniferum L. subsp. somniferum,Spring Poppy,summer_poppy,3301060602
+033,krmni grah (jari),Pisum sativum L.,Pea spring,peas,3301020600
+035,p?enica horasan (jara),Triticum turanicum Jakubz.,Khorasan wheat spring,other_cereals,3301019900
+036,ri?ek,Camelina sativa L. Crantz,Camelina,camelina,3301061500
+037,amarant,Amaranthus caudatus L.,Pendant amaranth,amaranth,3301150100
+038,repa,Brassica rapa L.  var. rapa (L.) Thell.,Turnip,turnips,3301290800
+049,sladka koruza,Zea mays L. convar. saccharata Koerm.,Sweet corn,grain_maize_corn_popcorn,3301010600
+052,me?anica medonosnih rastlin,A mixture of honey plants,A mixture of honey plants,fallow_land_not_crop,3301110000
+053,me?anica medonosnih rastlin z drugimi kmetijskimi rastlinami,A mixture of honey plants with other agricultural plants,A mixture of honey plants with other agricultural plants,fallow_land_not_crop,3301110000
+054,bela gorju?ica - medonosna praha,Sinapis alba L.,White mustard ,mustard,3301210100
+055,oljna redkev - medonosna praha,Raphanus sativus L. var. oleiformis Pers.,Oilseed radish,radish,3301290600
+056,facelija - medonosna praha,Phacelia tanacetifolia,Phacelia,phacelia,3301061400
+057,ajda - medonosna praha,Fagopyrum esculentum Moench,Buckwheat,buckwheat,3301150200
+058,son?nice - medonosna praha,Helianthus annus L.,Sunflower,sunflower,3301060500
+100,vinska trta,Vitis vinifera L.,Common grape vine,vineyards_wine_vine_rebland_grapes,3303060000
+101,krmna pesa,Beta vulgaris spp.vulgaris,Beet fodder,mangelwurzel_fodder_beet,3301290400
+102,krmna repa,Brassica rapa L. var. rapa (L.) Thell,Turnip,turnips,3301290800
+103,oljna repica,Brassica rapa L. subsp. campestris,Oilseed rape,rapeseed_rape,3301060400
+104,krmna repica (jara),Brassica rapa L. ssp. sylvestris f. autumnalis,Oilseed rape spring,spring_rapeseed_rape,3301060402
+105,krmni ohrovt,"Brassica oleracea L., convar.: acephala var. medullosa Thell.",Fodder cabbage,other_brassica_oleracea_cabbage,3301210299
+106,krmni radi?,Cichorium intybus L. var. sativum DC.  Bischoff,Common Chicory,chicory_chicories,3301310200
+107,krmno korenje,Daucus carota,Daucus,carrots_daucus,3301290300
+108,podzemna koleraba,Brassica napus L. var. napobrassica (L.) Rchb.,Swede,swede_rutabaga,3301210500
+109,krmni sirek,Sorghum bicolor (L.) Moench.,Sorghum,millet_sorghum,3301010900
+110,gra?ica (jara),Vicia sativa L.,Common peas spring,peas,3301020600
+111,bela gorju?ica,Sinapis alba L.,White mustard,mustard,3301210100
+112,krmna ogr??ica (jara),Brassica napus L. var. napus f. biennis,Spring oilseed rape,spring_rapeseed_rape,3301060402
+113,oljna redkev,Raphanus sativus L. var. oleiformis Pers.,Fodder Radish,radish,3301290600
+114,druge rastline za krmo na njivah,Other fodder crops on arable land,Other fodder crops on arable land,other_arable_land_crops,3301990000
+116,sudanska trava,Sorghum sudannense P.,Sudan grass,millet_sorghum,3301010900
+129,"druge me?anice z rastlinami, ki ve?ejo du?ik",Other mixtures with nitrogen-fixing crops,Other mixtures with nitrogen-fixing crops,not_known_and_other,3399000000
+200,trave za pridelavo semena,Grass for seed production,Grass for seed production,temporary_grass,3301090100
+201,trave,Grass on arable land,Grass on arable land,temporary_grass,3301090100
+202,travna ru?a (travni tepih),Green cover (grass turf),Green cover (grass turf),sod_turf,3301090207
+203,travnodeteljne me?anice,Grass clover mixture,Grass clover mixture,temporary_grass,3301090100
+204,trajno travinje,Permanent grassland,Permanent grassland,pasture_meadow_grassland_grass,3302000000
+206,deteljnotravne me?anice,Clover grass mixture,Clover grass mixture,clover,3301090303
+207,detelja,Trifolium pratense L.,Red clover,clover,3301090303
+208,lucerna,Medicago sativa L.,Alfalfa,alfalfa_lucerne,3301090301
+210,vol?ji bob,Lupinus albus L.,White lupin,sweet_lupins,3301020700
+219,facelija,Phacelia tanacetifolia,Lacy phacelia,phacelia,3301061400
+221,perzijska detelja,Trifolium resupinatum L.,Reversed clover,clover,3301090303
+222,inkarnatka,Trifolium incarnatum L.,Crimson clover,clover,3301090303
+333,tehni?no ali drugo sredstvo,Not mantained because of technical obstacle,Not mantained because of technical obstacle,unmaintained,3308000000
+402,zelenjadnice,Mixed vegetables,Mixed vegetables,fresh_vegetables,3301070000
+403,razli?na trajna zeli??a,Mixed permanent herbs,Mixed permanent herbs,aromatic_medicinal_culinary_plants_spices_herbs,3301061200
+404,enoletna in dvoletna njivska zeli??a,One-year and two-year herbs on fields,One-year herbs on fields,aromatic_medicinal_culinary_plants_spices_herbs,3301061200
+405,"me?ana raba (zelenjadnice, polj??ine, di?avnice in zdravilna zeli??a)","Mixed use (vegetables, crops, aromatic plants and medicinal herbs)","Mixed use (vegetables, crops, aromatic plants and medicinal herbs)",arable_crops,3301000000
+409,"me?ane zelenjadnice pod 0,1 ha","Mixed vegetables on field under 0,1 ha size",Mixed vegetables on field under 0,fresh_vegetables,3301070000
+444,pridelava ni v tleh,Production not in the soil,Production not in the soil,not_known_and_other,3399000000
+611,jablana,Malus domestica Borkh.,Apple,apples,3303010200
+612,hru?ka,Pyrus communis L.,European pear,pears,3303011200
+613,kutina,Cydonia oblonga Mill.,Quince,quinces,3303011500
+614,nashi,Pyrus pyrifolia (Burm.f.) Nakai,Asian pear,pears,3303011200
+615,granatno jabolko,Punica granatum L.,Pomegranate,pomegranate,3303011400
+616,ne?plja,Mespilus germanica L.,Common medlar,medlar_loquat,3303010800
+618,?i?ula,Ziziphus sativa Gaert.,Jujube red date,orchards_fruits,3303010000
+619,feioja,Feijoa sellowiana,Feijoa,feijoa,3303010500
+621,breskev,Prunus persica BATSCH,Peach,peach,3303011100
+622,nektarina,Prunus persica (L.) Batsch. var. nucipersica (Suckow) Schneid.,Nectarine,nectarine,3303010900
+623,sliva/?e?plja,Prunus domestica L.,Common plum,plums,3303011300
+624,marelica,Prunus armeniaca L.,Armenian plum,plums,3303011300
+625,?e?nja,Prunus avium L.,Sweet cherry,cherry_cherries,3303010400
+626,vi?nja,Prunus cerasus L. (Cerasus vulgaris Mill.),Sour cherry,cherry_cherries,3303010400
+631,oreh,Juglans regia,English walnut,walnuts,3303030600
+632,leska,Corylus avellana,Common hazel,hazelnuts_hazel,3303030200
+633,mandelj,Prunus dulcis (Mill.) D. A. Webb,Almond,almond,3303030100
+634,pekan oreh,Carya illinoensis,Pecan,pecan,3303030300
+642,kivi,Actinidia chinensis Planch. (Actinidia deliciosa),Golden kiwifruit,kiwi,3303010700
+643,kaki,Diospyros kaki L.,Japanese Persimmon,orchards_fruits,3303010000
+644,kostanj,Castanea sativa,Chestnut,sweet_chestnuts,3303030500
+646,bezeg,Sambucus L.,Elderberry,elder_elderberry,3303080400
+647,smokva (figa),Ficus carica L.,Fig,fig,3303010600
+648,asimina,Asimina triloba,Pawpaw,pawpaw,3303011000
+649,rakitovec,Hippophae rhamnoides L.,Seaberry,hippophae_sea_buckthorns_seaberry,3303020800
+651,jagoda,Fragaria L.,Strawberries,strawberries,3301130000
+652,ameri?ka borovnica,Vaccinium corymbosum L.,American blueberry,blueberry,3303020400
+653,malina,Rubus idaeus L.,Red raspberry,raspberry_raspberries,3303021000
+654,robida,Rubus fruticosus L.,Blackberry,blackberry,3303020200
+655,rde?i ribez,Ribes rubrum L.,Redcurrant,redcurrant,3303021100
+656,?rni ribez,Ribes nigrum L.,Blackcurrant,blackcurrant_cassis,3303020300
+657,aronija,Aronia melanocarpa,Black chokeberry,aronia_chokeberries,3303020100
+658,murva,Morus sp.,White mulberry,berries_berry_species,3303020000
+659,goji jagoda,Lycium barbarum L.,Goji berry,shrubberries_shrubs,3303080000
+660,?rni ribez x kosmulja,Ribes nidigrolaria,Jostaberry,jostaberry,3303020900
+661,namizno grozdje,Vitis vinifera,Common grape vine,vineyards_wine_vine_rebland_grapes,3303060000
+662,robida x malina,Rubus fruticosus x Rubus idaeus,Tayberry,tayberry,3303021400
+671,limonovec,Citrus limon (L.) Burm. f. lemon,Lemon,citrus_plantations,3303040000
+674,mandarinovec,Citrus reticulata Blanco,Mandarin orange,citrus_plantations,3303040000
+675,dren,Cornus mas L.,Cornelian cherry,dogwood_cornus,3306040000
+676,kosmulja,Ribes uva-crispa L.,European gooseberry,gooseberry_gooseberries_cranberries,3303020700
+677,skor?,Sorbus domestica L.,Sorb tree,other_tree_wood_forest,3306990000
+678,u?itno modro kosteni?je,Lonicera caerulea var. Kamtschatica,Kamchatka honeysuckle,honeysuckle,3303080500
+680,?ipek,Rosa canina L.,Rosa canina,roses,3301083700
+682,?marna hru?ica,Amelanchier spp.,Serviceberry,amelanchier_serviceberry,3303010100
+698,oreh in kostanj,Castanea sativa,Sweet chestnut,sweet_chestnuts,3303030500
+699,me?ane sadne vrste,Mixed fruit plants,Mixed fruit plants,orchards_fruits,3303010000
+702,drevesnice,Nurseries,Nurseries,nurseries_nursery,3303070000
+703,?parglji,Asparagus officinalis,Garden asparagus,asparagus,3301200000
+704,trsnice,Vine nurseries,Vine nurseries,vineyards_wine_vine_rebland_grapes,3303070000
+705,"me?ane trajne rastline pod 0,1 ha","Mixed permanent crops on field under 0,1 ha size",Mixed permanent crops on field under 01 ha size,permanent_crops_perennial,3303000000
+706,"trta za drugo rabo, ki ni vino ali namizno grozdje",Vines for other uses than for wine or fruits,Vines for other uses than for wine or fruits,vineyards_wine_vine_rebland_grapes,3303060000
+707,mati?njak,Root-stock nursery,Root-stock nursery,nurseries_nursery,3303070000
+710,me?ane rastline za rejo pol?ev,Mixed plants for snail farming,Mixed plants for snail farming,other_arable_land_crops,3301990000
+720,"hitro rasto?i panjevec (vrba, topol)","Short rotation coppice (willow tree, poplar)","Short rotation coppice (willow tree, poplar)",tree_wood_forest,3306000000
+721,drugi hitro rasto?i panjevci,Other short rotation coppices,Other short rotation coppices,tree_wood_forest,3306000000
+722,miskant,Miscanthus,Silvergrass,miscanthus_silvergrass,3301083000
+733,arti?oka,Cynara cardunculus var. Scolymus L.,Globe artichocke,artichoke,3301270000
+734,rabarbara,Rheum rhabarbarum L.,Garden rhubarb,rhubarb,3301230000
+735,okrasne rastline,Ornamental plants,Ornamental plants,flowers_ornamental_plants,3301080000
+736,vrtnice,Roses,Roses,roses,3301083700
+737,sivka,Lavandula spica L.,Lavandula,lavender_lavandula,3301061219
+738,ameri?ki slamnik,Echinacea purpurea (L.) Moench Echinacea angustifolia,Purple coneflower,echinacea_sun_hat,3301081500
+777,povr?ina v odstopu,Crop not known,Crop not known,not_known_and_other,3399000000
+800,oljka,Olea europaea L. v. europaea,Olive,olive_plantations,3303050000
+801,p?enica (ozimna),Triticum L.,Common wheat winter,winter_common_soft_wheat,3301010101
+802,r? (ozimna),Secale cereale L.,Rye winter,winter_rye,3301010301
+803,pira (ozimna),Triticum spelta L.,Spelt winter,winter_spelt,3301011001
+804,krmna repica (ozimna),Brassica rapa L. var. silvestris (Lam.) Briggs,Turnip rape winter,winter_rapeseed_rape,3301060401
+807,tritikala (ozimna),X Triticosecale Wittmack (Triticum x Secale),Triticale winter,winter_triticale,3301010801
+808,oves (ozimni),Avena sativa L.,Winter oats,winter_oats,3301010501
+809,je?men (ozimni),Hordeum L.,Barley winter,winter_barley,3301010401
+811,me?anice ?it (ozimna),Mixture of cereals (winter),Mixture of cereals (winter),winter_unspecified_cereals,3301011501
+812,krmna ogr??ica (ozimna),Brassica napus L. var. napus f. biennis,Fodder rape (winter),winter_rapeseed_rape,3301060401
+814,oljna ogr??ica (ozimna),Brassica napus L. ssp. oleifera (Metzg.) Sinsk.,Winter rape,winter_rapeseed_rape,3301060401
+821,sor?ica (ozimna),Meslin (winter),Meslin (winter),winter_meslin,3301011101
+825,trda p?enica (ozimna),Triticum durum Desf.,Durum winter,winter_durum_hard_wheat,3301010201
+831,vrtni mak (ozimni),Papaver somniferum L. subsp. somniferum,Opium poppy winter,winter_poppy,3301060601
+833,krmni grah (ozimni),Pisum sativum L. (partim),Peas winter,peas,3301020600
+835,p?enica horasan (ozimna),Triticum turanicum Jakubz.,Khorasan wheat (winter),other_cereals,3301019900
+900,hmelj,Humulus lupulus L.,Common hop,hops,3301060200