Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
69d4715
Fix for epa-airpollutantemission-level1
shourya116 Aug 10, 2026
f3a0069
Added Goldens
shourya116 Aug 11, 2026
2d5cedd
Goldens Added
shourya116 Aug 12, 2026
a7e8dcd
Apply suggestion from @gemini-code-assist[bot]
shourya116 Aug 12, 2026
cb21e4a
Removing goldens
shourya116 Sep 1, 2026
c540f89
Fix process_tribes.csv column regularization in EPA_AirPollutantEmiss…
shourya116 Sep 5, 2026
bd8a5b0
Add date freshness validation and remove dead code in EPA_AirPollutan…
shourya116 Sep 6, 2026
9856721
removed data freshness check
shourya116 Sep 6, 2026
15b2958
Address review comments: fix test imports, add regression tests, impr…
shourya116 Sep 7, 2026
60305db
Clean up exception handling, update validation description, and forma…
shourya116 Sep 8, 2026
242f228
Merge branch 'master' into fix_epa-airpollutantemission-level1
shourya116 Sep 8, 2026
acabcfb
Merge branch 'master' into fix_epa-airpollutantemission-level1
shourya116 Sep 9, 2026
349a9f4
Merge branch 'master' into fix_epa-airpollutantemission-level1
shourya116 Sep 10, 2026
dca305c
Eliminate redundant exception logging in worker and top-level handlers
shourya116 Sep 10, 2026
6b7432f
Merge branch 'master' into fix_epa-airpollutantemission-level1
shourya116 Sep 10, 2026
7bd2f04
Fix code review findings for epa-airpollutantemission-level1
shourya116 Sep 21, 2026
35c902f
Optimize national_emissions_inventory memory and disk footprint
shourya116 Sep 22, 2026
70a1926
fix(epa): remove erroneous total emissions rename in replacement_poin…
shourya116 Sep 22, 2026
008719e
Merge branch 'master' into fix_epa-airpollutantemission-level1
shourya116 Sep 22, 2026
8b890f9
fix(epa): address code review findings for national_emissions_inventory
shourya116 Sep 22, 2026
a1ca509
fix(epa): address code review findings for national_emissions_inventory
shourya116 Sep 22, 2026
23a89dc
Merge branch 'master' into fix_epa-airpollutantemission-level1
shourya116 Sep 23, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions scripts/us_epa/national_emissions_inventory/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ These are the attributes that we will use
| pollutant type(s) | The type of Gas generated which pollutes the Air. |

### Cleaned Data
Cleaned data will be inside [output/national_emissions.csv] as a CSV file with the following columns.
Cleaned data will be inside [gcs_output/output_files/national_emissions.csv] as a CSV file with the following columns.

- year
- geo_Id
Expand All @@ -48,8 +48,8 @@ Cleaned data will be inside [output/national_emissions.csv] as a CSV file with t


### MCFs and Template MCFs
- [output/national_emissions.mcf]
- [output/national_emissions.tmcf]
- [gcs_output/output_files/national_emissions.mcf]
- [gcs_output/output_files/national_emissions.tmcf]


### Running Tests
Expand Down
13 changes: 13 additions & 0 deletions scripts/us_epa/national_emissions_inventory/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
10 changes: 2 additions & 8 deletions scripts/us_epa/national_emissions_inventory/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,6 @@
'pollutant_code': 'pollutant code',
'total_emissions': 'total emissions',
'emissions_uom': 'emissions uom',
'total emissions': 'observation',
'pollutant_type': 'pollutant type(s)'
}

Expand All @@ -90,14 +89,10 @@
'fips state/county code': 'fips code',
'scc': 'scc',
'pollutant code': 'pollutant code',
'total emissions': 'observation',
'uom': 'unit'
'total emissions': 'total emissions',
'uom': 'emissions uom'
}

drop_tribes = [
'state', 'fips state code', 'data category', 'reporting period',
'emissions operating type', 'pollutant desc', 'data set'
]
drop_df = [
'scc', 'pollutant code', 'emissions type code', 'pollutant type(s)',
'fips code'
Expand Down Expand Up @@ -1500,7 +1495,6 @@
"1": "External Combustion",
"2": "Internal Combustion Engines",
"3": "Industrial Processes",
"4": "Chemical Evaporation",
"4": "Petroleum And Solvent Evaporation",
"5": "Waste Disposal",
"6": "MACT Source Categories",
Expand Down
9 changes: 5 additions & 4 deletions scripts/us_epa/national_emissions_inventory/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@
"process.py"
],
"source_files": [
"gcs_output/input_files/*/*.csv"
"gcs_output/input_files/*/*.csv",
"manifest.json",
"validation_config.json"
],
"import_inputs": [
{
Expand All @@ -21,13 +23,12 @@
}
],
"cron_schedule": "0 0 1 1-12/3 *",
"validation_config_file": "validation_config.json",
Comment thread
shourya116 marked this conversation as resolved.
"resource_limits": {
"cpu": 8,
"memory": 128,
"disk": 100
"disk": 300
}
}
]
}


Loading
Loading