feat: report empty required and recommended files - #2181
feat: report empty required and recommended files#2181mackenziereading19 wants to merge 2 commits into
Conversation
skalexch
left a comment
There was a problem hiding this comment.
I tried locally with https://mobilitydatabase.org/feeds/gtfs/mdb-340.
RequiredFileEmptyNotice works well. RecommendedFileEmptyNotice did not work for shapes.txt or when I only kept the header for feed_info.txt. I am not sure if we tag these files as recommended actually. We tag feed_info.txt as Conditionally Required.
By the way, a headerless shapes.txt seems to raise MissingRecommendedFile(shapes.txt) through the check shapeTable.getEntities().isEmpty() (here).
A few suggestions:
- Double check if those Recommended files have a "Recommended" tag (only two files I think: shapes and feed_info. We already have
MissingFeedInfoValidatorwhich tests for ashapes.txtwith a header only. We can do the same in - If tagging a file as Recommended is not easy, especially for
feed_info.txt, then remove RecommendedFileEmptyNotice from this PR and add the check inMissingFeedInfoValidator.
|
Thanks for testing this and for the suggestion. I’ve updated the PR accordingly in 8b74e24. The generic RecommendedFileEmptyNotice has been removed. RequiredFileEmptyNotice remains in CsvFileLoader for genuinely required files. For feed_info.txt, MissingFeedInfoValidator now treats a headers-only/empty table the same as a missing file, preserving its existing conditional semantics: no translations.txt → MissingRecommendedFileNotice(feed_info.txt) I left shapes.txt on its existing dedicated path, since MissingShapesFileValidator already treats an empty shapes table as missing and applies the DRT exceptions before emitting MissingRecommendedFileNotice. Focused verification after the revision: CsvTableLoaderTest: 8 passed |
skalexch
left a comment
There was a problem hiding this comment.
Good from spec side. Dev code review is required to approve the PR.
Summary
This PR addresses #1853 by distinguishing GTFS files that are present with valid headers but contain no data rows.
The loader already treats a completely empty/headerless CSV as
EmptyFileNotice. However, a headers-only file is considered parsable and previously produced no notice even when the file was required or recommended.This change adds:
RequiredFileEmptyNotice(ERROR) for required files with headers but no data rows;RecommendedFileEmptyNotice(WARNING) for recommended files with headers but no data rows;EmptyFileNoticedocumentation so it only describes genuinely headerless files.The parsed table remains
PARSABLE_HEADERS_AND_ROWSin all headers-only cases because the file structure and headers are valid; the new notices describe the file-content requirement rather than reclassifying the table as missing or unparsable.Tests
Added regression coverage for:
RequiredFileEmptyNotice;RecommendedFileEmptyNotice;Local verification:
:core:test: pass;NoticeReferenceTest: pass;:core:spotlessCheckand:main:spotlessCheck: pass;git diff --check: pass.Fixes #1853