|
|
def67a0562
|
Added correct calibration parameters to calibration factors table
|
2025-03-03 14:23:45 +01:00 |
|
|
|
d3a491ea85
|
Renamed pipeline step join_tables.py to prepare_ebas_submission.py for accuracy. This step performs more than joining tables, it also does renaming using dictionaries and generates the ebas data submission package
|
2025-03-03 10:48:03 +01:00 |
|
|
|
877c0178d7
|
WIP: Added first version of join_tables and tested txt generation. We should probaly make this step encompasse name renaming for EBAS and rename step to be more generic. Change join_tables.py to generate_ebas_converter_input_files
|
2025-02-28 17:23:00 +01:00 |
|
|
|
a9533ae3e8
|
Refine function. Abstract out load parameters as function to simplify code.
|
2025-02-28 17:18:13 +01:00 |
|
|
|
6890377134
|
Remove dataset_name parameter from command line interface. Instead i now infer it assuming there is a consistent input filename in the hdf5 file that can be found using hard coded keywords.
|
2025-02-28 17:17:33 +01:00 |
|
|
|
499ba3d1c7
|
Update TODO.md
|
2025-02-27 09:14:19 +01:00 |
|
|
|
33f29a8080
|
WIP: Completed flag generation for species, command line interface is now clear but still does not consider potentially existing flags. Review TODO.md, for efficiency we need to simplify flag assignment and reconciliation for the species flag type.
|
2025-02-26 15:11:12 +01:00 |
|
|
|
600c60be36
|
Append variable specification for explicit definition of diagnostics and species to pipelines/params/calibration_params.yaml. This is to support flagging operations to particular variables
|
2025-02-26 15:07:58 +01:00 |
|
|
|
6d5d74e616
|
Append flag_ranking to app/flags/ebas_dict.yaml. This is to disambiguate/overide conflicting flags
|
2025-02-26 15:05:06 +01:00 |
|
|
|
e6868da3d4
|
Update dima to latest version
|
2025-02-25 13:34:52 +01:00 |
|
|
|
94cd536204
|
Updated dima to latest version
|
2025-02-22 18:08:14 +01:00 |
|
|
|
4edd6809e9
|
WIP: Redesigned command line functionality. options --species is half baked :). It runs without error but second part needs validation.
|
2025-02-22 16:00:27 +01:00 |
|
|
|
e24d451542
|
Updated thirdparty submodule to latest version
|
2025-02-22 15:02:17 +01:00 |
|
|
|
456f4c297f
|
Adjust code due to changes in flag definition location.
|
2025-02-20 17:28:43 +01:00 |
|
|
|
60e1c35745
|
Move flag definitions from ...utils.py to flags/ebas_flags.yaml
|
2025-02-20 17:26:50 +01:00 |
|
|
|
dc34e07f62
|
Add actris_acsm_converter as a third_party submodule
|
2025-02-20 16:07:47 +01:00 |
|
|
|
ae15333207
|
Update layout figure
|
2025-02-17 15:46:40 +01:00 |
|
|
|
de33977e71
|
Start restructuring of app to make it more modular, and improve layout a bit
|
2025-02-17 15:44:51 +01:00 |
|
|
|
0454b54b8c
|
Fix bug: When flag folder empty, it now returns [] (as opposed to None)
|
2025-02-17 15:42:57 +01:00 |
|
|
|
0911260f26
|
Save changes.
|
2025-02-15 18:20:58 +01:00 |
|
|
|
25f3ee12a4
|
Fix bug. Saved flags from apps were not associated with right parent variable.
|
2025-02-15 18:10:29 +01:00 |
|
|
|
46767996b8
|
Modify experimental campaign descriptor to point to yearly acsm_tofware data.
|
2025-02-15 18:08:14 +01:00 |
|
|
|
a516479036
|
Change data lineage format. Now, data lineage information is accessed using name of the file as key.
|
2025-02-15 17:57:50 +01:00 |
|
|
|
3123fec9da
|
Add new version of dash board graphical user interphase
|
2025-02-13 19:59:20 +01:00 |
|
|
|
10fdd1b8c7
|
Fix bug when saving flags to flagsFolder and subsequent visualization of existing flags.
|
2025-02-13 19:58:30 +01:00 |
|
|
|
e53d8dec8c
|
Simplify load_flags() function by removing redundant h5 verification step
|
2025-02-13 19:56:44 +01:00 |
|
|
|
11ecdebf1b
|
Move flagging app to app folder with initialized callbacks/ and components/ modules
|
2025-02-13 18:13:45 +01:00 |
|
|
|
479c11d0d4
|
Move app scripts to app folder. This is to improve modularity of the app
|
2025-02-13 18:03:41 +01:00 |
|
|
|
1a86dc2065
|
Implement instrument, file, and variable selection dropdown menus.
|
2025-02-13 17:56:08 +01:00 |
|
|
|
79f9badb1c
|
Update dima submodule to latest version.
|
2025-02-11 13:16:14 +01:00 |
|
|
|
f2d46101b1
|
Add demo jupyter notebook to visualize variables and their associated flags
|
2025-02-11 13:15:19 +01:00 |
|
|
|
ca08044569
|
Update keys of data_lineage_metadata.yaml. keys are now the filenames in the folder.
|
2025-02-11 13:14:14 +01:00 |
|
|
|
80fbe3af91
|
Update params/validity_thresholds.yaml with values from 2024 JFJ ACSM data
|
2025-02-11 13:12:00 +01:00 |
|
|
|
0623c2462b
|
Adjust command line interface of steps/apply_... and steps/compute_aut...
|
2025-02-11 13:07:29 +01:00 |
|
|
|
f2854d4ca0
|
Implement pipelines/steps/visualize_datatable_vars.py
|
2025-02-11 13:03:46 +01:00 |
|
|
|
e7ad4d2a36
|
Add new demo for acsm pipeline and clearout the other notebooks
|
2025-02-10 16:15:56 +01:00 |
|
|
|
3c482c1d21
|
Rename and move jupyter notebook demos to notebooks folder
|
2025-02-10 16:14:49 +01:00 |
|
|
|
b966151d9c
|
Update with template for compute_cpc_flags(). This still needs to be completed.
|
2025-02-10 10:54:18 +01:00 |
|
|
|
ca2b8b6787
|
Rename apply_diagnostic_flags.py to compute_automated_flags.py because the function also computes both cpc and diagnostic based flags
|
2025-02-10 10:45:55 +01:00 |
|
|
|
e2958b608f
|
Update dima to latest version
|
2025-02-08 14:52:27 +01:00 |
|
|
|
d93003f5c4
|
Add import statement and adjust dependencies.
|
2025-02-06 16:57:46 +01:00 |
|
|
|
ad95d11a0d
|
Implement utils module for reusable functions across steps in the pipelines
|
2025-02-06 16:54:36 +01:00 |
|
|
|
6ed9fe1eb1
|
Rename pipeline step to apply_diagnositc_flags, it perform a check based on specified thresholds and then assigns I, V, or M labels producing a labeling of a variable.
|
2025-02-06 16:25:40 +01:00 |
|
|
|
f8b900422b
|
Rename function
|
2025-02-06 16:15:19 +01:00 |
|
|
|
5e979b4e5e
|
Update dima submodule to latest version
|
2025-02-06 16:08:35 +01:00 |
|
|
|
2c2b154528
|
Update to account for yaml file attribute renamings.
|
2025-02-05 18:15:43 +01:00 |
|
|
|
e2b5aa9d69
|
Update file structure to depend on calibration_parameters.yaml and update thresholds for JFJ/ACSM_TOFWARE/2024 data.
|
2025-02-05 17:05:07 +01:00 |
|
|
|
afd6ef4d1b
|
Fix typo in denominators.
|
2025-02-05 16:49:36 +01:00 |
|
|
|
bc779bd583
|
Update calibration parameters for JFJ/ACSM_TOFWARE/2024 data.
|
2025-02-05 16:48:14 +01:00 |
|
|
|
48e2127fef
|
Reimplement apply_calibration_factors() in terms of load_calibration_file() and compute_calibration_factors() to reduce complexity and improve modularity.
|
2025-02-04 18:12:42 +01:00 |
|
