 florez_j
|
5a54a22ebd
|
Update after run with PAY dataset.
|
2025-04-11 11:31:35 +02:00 |
|
|
florez_j
|
0353c8492b
|
Added data chain step. This step updates the current parameters with new input folder params
|
2025-04-11 11:23:04 +02:00 |
|
|
florez_j
|
4408514439
|
Rename attribute station --> station_abbr
|
2025-04-11 11:20:48 +02:00 |
|
|
florez_j
|
e7863078aa
|
Add data chain step /update_datachain_params.py. This uses params specified in input data folder to update current data chain params
|
2025-04-11 11:07:52 +02:00 |
|
|
florez_j
|
cea83396d2
|
Add function to estimate project directory path
|
2025-04-11 11:05:41 +02:00 |
|
|
florez_j
|
68a7d5dc2b
|
Fix out of bound error in pipelines/steps/visualize_datatable_vars.py
|
2025-04-11 11:04:00 +02:00 |
|
|
florez_j
|
c9e25c2483
|
Add new placeholder for submitter name and remove metadata element 'station' conflicting with third party converter
|
2025-04-11 11:02:38 +02:00 |
|
|
florez_j
|
13bcd88a93
|
Replace station attribute with station_abbr, so it does not conflict with ebas database naming
|
2025-04-11 10:41:05 +02:00 |
|
|
florez_j
|
b0b3ccef66
|
Update campaignDescriptor.yaml with originator and submitter info
|
2025-04-08 21:27:51 +02:00 |
|
|
florez_j
|
4725d85e61
|
Update data chain notebook
|
2025-04-08 21:26:36 +02:00 |
|
|
florez_j
|
881209bf82
|
Update steps to allow filenames for PAYERN station
|
2025-04-08 21:26:01 +02:00 |
|
|
florez_j
|
3dfed2c5f3
|
Update pipelines/steps/prepare_ebas_submission.py. Rmoved hard coded paths and build output name using metadata from campaign descriptor. Also, we can now specify month ranges
|
2025-04-08 18:44:46 +02:00 |
|
|
florez_j
|
5dd280e88c
|
Update pipelines/steps/utils.py. Now we can load the campaign descriptor using load_project_yaml_files()
|
2025-04-08 18:43:01 +02:00 |
|
|
florez_j
|
bfaaeb4416
|
Add header templates for JFJ and PAY
|
2025-04-08 18:35:44 +02:00 |
|
|
florez_j
|
ca0366338a
|
Add pipeline step to update actris_header with submitter and originator
|
2025-04-08 18:34:51 +02:00 |
|
|
florez_j
|
0e625b558a
|
Update third party submodule
|
2025-04-08 15:46:18 +02:00 |
|
|
florez_j
|
00a85505c8
|
Fix bug when infering name of flags file from dataset name
|
2025-04-04 16:53:41 +02:00 |
|
|
florez_j
|
dcf6b8ec41
|
Clean up and document steps in notebooks/demo_visualize_diagnostic_flags_from_hdf5_file.ipynb
|
2025-04-04 16:07:07 +02:00 |
|
|
florez_j
|
d290daaa39
|
Update pipelines/steps/visualize_datatable_vars.py. Specify alternative path to flags in case they are not in the HDF5 file. We assume there is an append directory where they may be. Otherwise it throws and error.
|
2025-04-04 16:06:11 +02:00 |
|
|
florez_j
|
b3b5516aed
|
Fix typo in README.md
|
2025-04-04 09:46:36 +02:00 |
|
|
florez_j
|
17f497f1c5
|
Update README.md with instructions to set up network drive access credentials and update TODO.md
|
2025-04-02 14:40:55 +02:00 |
|
|
florez_j
|
9a23bf283a
|
Fix output directory in campaignDescripor.yaml
|
2025-04-02 14:17:45 +02:00 |
|
|
florez_j
|
73c0409207
|
Update README.md with instruction to build, run, and stop docker container on which the data chain will run
|
2025-04-02 14:10:52 +02:00 |
|
|
florez_j
|
73648ebe31
|
Update Dockerfile and docker-compose.yaml. By default, container can use cifs to map network drives so no need to install, and added bash command to docker compose file to run both jupyterlab and dash board simultanously
|
2025-04-02 14:09:44 +02:00 |
|
|
florez_j
|
c6b010702c
|
Remove run_container.sh. This is now done by running the docker-compose.yaml file using docker
|
2025-04-02 11:27:42 +02:00 |
|
|
florez_j
|
643a305782
|
Update ACSM data chain workflow with markdown descriptions
|
2025-04-02 11:26:03 +02:00 |
|
|
florez_j
|
f0da41e914
|
Add docker-compose.yaml to replace run_container.sh
|
2025-04-02 11:17:20 +02:00 |
|
|
florez_j
|
1e7f92d6e8
|
Update ignore files
|
2025-04-02 11:15:36 +02:00 |
|
|
florez_j
|
14cb24fb4b
|
Add config files .sh .yaml and .md to dockerignore
|
2025-03-18 09:21:26 +01:00 |
|
|
florez_j
|
aac6015986
|
Update README with Docker container workflow
|
2025-03-18 08:57:51 +01:00 |
|
|
florez_j
|
b7cbf7dd14
|
Add bash script to run container
|
2025-03-18 08:57:06 +01:00 |
|
|
florez_j
|
41506519d3
|
WIP: Update Dockerfile with requirements.txt that downgrade versions so that ebas converter can run. We need to rethink the Dockerfile container workflow using docker compose
|
2025-03-18 08:55:52 +01:00 |
|
|
florez_j
|
862a2c46e6
|
Clean up notebooks
|
2025-03-18 08:53:51 +01:00 |
|
|
florez_j
|
b0d68747e9
|
Update code to rearrange table columns based on column_order from acsm_to_ebas.yaml. This can be removed in the future.
|
2025-03-18 07:23:54 +01:00 |
|
|
florez_j
|
261c8b8154
|
Add column order attributes, storing the order in which variables need to be displayed to adhere to Nora's initial implementation.
|
2025-03-18 07:22:06 +01:00 |
|
|
florez_j
|
f3010bff47
|
Fix import problem. The app didn't run from a notebooks/<>.ipynb. This however may not be needed any longer. Is it good practice to run dashboards from a jupyter notebook?
|
2025-03-18 07:14:44 +01:00 |
|
|
florez_j
|
395b734cd6
|
Fix bug reported in TODO.md. FLAGGING APP didnt record the parent channel correctly, that is the channel that originated a particular flag. An additional problem was detected and corrected where _metadata.json was confused as flag.
|
2025-03-17 14:21:26 +01:00 |
|
|
florez_j
|
895a44cc8c
|
Update notebooks/demo_acsm_pipeline.ipynb by introducing global append and input file variables.
|
2025-03-14 15:26:37 +01:00 |
|
|
florez_j
|
d5a89b2fbd
|
Add more items to TODO.md
|
2025-03-14 15:25:47 +01:00 |
|
|
florez_j
|
f90708b043
|
Update todo.md
|
2025-03-14 15:17:31 +01:00 |
|
|
florez_j
|
9504576fc0
|
Refactor steps using main functions from each step instead of subprocess with command line interface
|
2025-03-14 13:44:03 +01:00 |
|
|
florez_j
|
b93cb7517b
|
Add comment to indicate where params are defined in original code.
|
2025-03-14 13:40:22 +01:00 |
|
|
florez_j
|
7d277e3e3b
|
Clean up print statements.
|
2025-03-14 13:39:12 +01:00 |
|
|
florez_j
|
39a9ab07a2
|
Refactor command line interface to run a main function on the parsed arguments.
|
2025-03-14 13:36:52 +01:00 |
|
|
florez_j
|
8cdd8a0771
|
Completed command line interface for pipelines/steps/prepare_ebas_submission.py. This finishes WIP associated with commit 2eb88e4.
|
2025-03-14 13:09:09 +01:00 |
|
|
florez_j
|
bfc9f0ab82
|
Cross out item. Requirements of data input to converter are defined in pipelines/dictionaries/acsm_to_ebas.yaml
|
2025-03-14 13:05:29 +01:00 |
|
|
florez_j
|
f3612267c1
|
Update dima to latest version.
|
2025-03-14 10:18:30 +01:00 |
|
|
florez_j
|
b643eb2d26
|
Refactored command line interface, generate_species_flags, and created functions to improve performance and code clarity.
|
2025-03-13 14:17:46 +01:00 |
|
|
florez_j
|
97f253074b
|
Add in try-except block a finally statement to close hdf5 file for secure file closing. This was done before in the try case which is not good practice
|
2025-03-13 14:06:01 +01:00 |
|
|
florez_j
|
0fe2b564b7
|
WIP: Change to copy command as COPY . /acsmdc.
|
2025-03-12 16:22:39 +01:00 |
|