Skip to content

feat: report empty required and recommended files - #2181

Open
mackenziereading19 wants to merge 2 commits into
MobilityData:masterfrom
mackenziereading19:1853-empty-required-recommended-files
Open

feat: report empty required and recommended files#2181
mackenziereading19 wants to merge 2 commits into
MobilityData:masterfrom
mackenziereading19:1853-empty-required-recommended-files

Conversation

@mackenziereading19

Copy link
Copy Markdown

Summary

This PR addresses #1853 by distinguishing GTFS files that are present with valid headers but contain no data rows.

The loader already treats a completely empty/headerless CSV as EmptyFileNotice. However, a headers-only file is considered parsable and previously produced no notice even when the file was required or recommended.

This change adds:

  • RequiredFileEmptyNotice (ERROR) for required files with headers but no data rows;
  • RecommendedFileEmptyNotice (WARNING) for recommended files with headers but no data rows;
  • no new notice for optional files with headers but no data rows;
  • updated EmptyFileNotice documentation so it only describes genuinely headerless files.

The parsed table remains PARSABLE_HEADERS_AND_ROWS in all headers-only cases because the file structure and headers are valid; the new notices describe the file-content requirement rather than reclassifying the table as missing or unparsable.

Tests

Added regression coverage for:

  • required headers-only file -> RequiredFileEmptyNotice;
  • recommended headers-only file -> RecommendedFileEmptyNotice;
  • optional headers-only file -> no empty-file notice.

Local verification:

  • full :core:test: pass;
  • targeted NoticeReferenceTest: pass;
  • :core:spotlessCheck and :main:spotlessCheck: pass;
  • git diff --check: pass.

Fixes #1853

@emmambd
emmambd requested a review from skalexch September 1, 2026 17:44

@skalexch skalexch left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried locally with https://mobilitydatabase.org/feeds/gtfs/mdb-340.

RequiredFileEmptyNotice works well. RecommendedFileEmptyNotice did not work for shapes.txt or when I only kept the header for feed_info.txt. I am not sure if we tag these files as recommended actually. We tag feed_info.txt as Conditionally Required.

By the way, a headerless shapes.txt seems to raise MissingRecommendedFile(shapes.txt) through the check shapeTable.getEntities().isEmpty() (here).

A few suggestions:

  • Double check if those Recommended files have a "Recommended" tag (only two files I think: shapes and feed_info. We already have MissingFeedInfoValidator which tests for a shapes.txt with a header only. We can do the same in
  • If tagging a file as Recommended is not easy, especially for feed_info.txt, then remove RecommendedFileEmptyNotice from this PR and add the check in MissingFeedInfoValidator.

@mackenziereading19

Copy link
Copy Markdown
Author

Thanks for testing this and for the suggestion. I’ve updated the PR accordingly in 8b74e24.

The generic RecommendedFileEmptyNotice has been removed. RequiredFileEmptyNotice remains in CsvFileLoader for genuinely required files.

For feed_info.txt, MissingFeedInfoValidator now treats a headers-only/empty table the same as a missing file, preserving its existing conditional semantics:

no translations.txt → MissingRecommendedFileNotice(feed_info.txt)
translations.txt present → MissingRequiredFileNotice(feed_info.txt)

I left shapes.txt on its existing dedicated path, since MissingShapesFileValidator already treats an empty shapes table as missing and applies the DRT exceptions before emitting MissingRecommendedFileNotice.

Focused verification after the revision:

CsvTableLoaderTest: 8 passed
MissingFeedInfoValidatorTest: 5 passed
MissingShapesFileValidatorTest: 3 passed
NoticeReferenceTest: passed
core/main Spotless checks: passed

@skalexch skalexch left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good from spec side. Dev code review is required to approve the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Required and recommended notices on files with no rows

2 participants