APOG_public/acsmnode
0
0
Fork 0
mirror of https://gitea.psi.ch/APOG/acsmnode.git synced 2025-06-25 13:41:08 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request 'WIP.new_changes_leila' (#1) from new_changes_leila into main

Browse Source 
Reviewed-on: APOG/acsmnode#1
This commit is contained in:
florez_j
2025-06-04 09:35:31 +02:00
parent 8afa2f98d1 fc6f6708b8
commit c15aa4e828
2 changed files with 279 additions and 277 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
TODO.md
View File

@ -1,11 +1,13 @@
# TODO # TODO
* [from Leïla] Correct error when flags are loaded in the flagging app * [from Leïla] Data Flagging App. Correct error when flags are loaded in the app.
* [from Leïla] End the file at end of year (or filter only current year) * [from Leïla] Before Step 5. The file should contain only data of the current year.
* [from Leïla] Change "9999.999" to "9999999.9999" in header * [from Leïla] Ebas converter. Change "9999.999" to "9999999.9999" in header
* [from Leïla] Update Detection limit values in L2 header: take the ones (1h) from limits_of_detection.yaml * [from Leïla] Ebas converter. Update Detection limit values in L2 header: take the ones (1h) from limits_of_detection.yaml
* [from Leïla] For PAY, calculate error as 50% of concentration * [from Leïla] Ebas converter. Correct errors (uncertainties) that they can't be lower than 0.0001
* [from Leïla] Correct errors (uncertainties) that they can't be lower than 0.0001 * [from Leïla] Flag using validity threshold. Change flow rate values to 10% of flow rate ref.
* [from Leïla] Change flow rate values to 10% of flow rate ref * [from Leïla] In Step 1. Creation of a new collection should be an option and not automatic.
* [from Leïla] The data chain (except Step 5) should also include Org_mx and Ord_err. To discuss together.
* [from Leïla] Step 4.1. Add a step to verify ACSM data with external instruments (MPSS, eBC). To discuss together.
* [New] Create data flows to validate and replace existing data chain params. data/campaignData/param/ -> pipelines/params and campaignDescriptor.yaml -> acsm data converter. * [New] Create data flows to validate and replace existing data chain params. data/campaignData/param/ -> pipelines/params and campaignDescriptor.yaml -> acsm data converter.
* [New] DIMA. Add dictionaries to explain variables at different levels. * [New] DIMA. Add dictionaries to explain variables at different levels.
* [New] DIMA. Modify data integration pipeline to not add current datetime in filename when not specified. * [New] DIMA. Modify data integration pipeline to not add current datetime in filename when not specified.

4
pipelines/steps/utils.py
View File

@ -98,7 +98,7 @@ import numpy as np
def compute_uncertainty_estimate(x, x_err): def compute_uncertainty_estimate(x, x_err):
""" """
Computes uncertainty estimate: sqrt((0.5 * x_err)^2 + (0.5 * x)^2) Computes uncertainty estimate: sqrt((x_err)^2 + (0.5 * x)^2)
for scalar inputs. Prints errors if inputs are invalid. for scalar inputs. Prints errors if inputs are invalid.
""" """
try: try:
@ -109,7 +109,7 @@ def compute_uncertainty_estimate(x, x_err):
print(f"Warning: One or both inputs are NaN -> x: {x}, x_err: {x_err}") print(f"Warning: One or both inputs are NaN -> x: {x}, x_err: {x_err}")
return np.nan return np.nan
return math.sqrt((0.5 * x_err)**2 + (0.5 * x)**2) return math.sqrt((x_err)**2 + (0.5 * x)**2)
except (ValueError, TypeError) as e: except (ValueError, TypeError) as e:
print(f"Error computing uncertainty for x: {x}, x_err: {x_err} -> {e}") print(f"Error computing uncertainty for x: {x}, x_err: {x_err} -> {e}")