mirror of
https://gitea.psi.ch/APOG/acsm-fairifier.git
synced 2025-07-10 18:01:50 +02:00
Prepare v1.0.0 release as acsm-fairifier
This commit is contained in:
20
CHANGELOG.md
Normal file
20
CHANGELOG.md
Normal file
@ -0,0 +1,20 @@
|
|||||||
|
# Changelog
|
||||||
|
|
||||||
|
All notable changes to this project will be documented in this file, which is a **cumulative record**.
|
||||||
|
|
||||||
|
Each version entry follows a consistent structure with the following optional sections:
|
||||||
|
|
||||||
|
- **Added** – New features
|
||||||
|
- **Changed** – Modifications to existing functionality
|
||||||
|
- **Deprecated** – Features marked for future removal
|
||||||
|
- **Removed** – Features removed in this version
|
||||||
|
- **Fixed** – Bug fixes
|
||||||
|
- **Security** – Vulnerability fixes
|
||||||
|
|
||||||
|
Format based on [Keep a Changelog](https://keepachangelog.com) and [Semantic Versioning](https://semver.org).
|
||||||
|
|
||||||
|
|
||||||
|
## [v1.0.0] - 2025-07-04
|
||||||
|
### Added
|
||||||
|
- Initial release of the ACSM FAIRifier
|
||||||
|
|
14
README.md
14
README.md
@ -113,16 +113,16 @@ Open **Git Bash** and run:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd Gitea
|
cd Gitea
|
||||||
git clone --recurse-submodules https://gitea.psi.ch/apog/acsmnode.git
|
git clone --recurse-submodules https://gitea.psi.ch/apog/acsm-fairifier.git
|
||||||
cd acsmnode
|
cd acsm-fairifier
|
||||||
```
|
```
|
||||||
|
|
||||||
## Run the ACSM FAIRifier Toolkit
|
## Run the ACSM FAIRifier Toolkit
|
||||||
|
|
||||||
This toolkit includes a containerized JupyterLab environment for executing the data processing pipeline, plus an optional dashboard for manual flagging.
|
This toolkit includes a containerized JupyterLab environment for executing the data processing pipeline, plus an optional dashboard for manual flagging.
|
||||||
|
|
||||||
1. Open **PowerShell as Administrator** and navigate to the `acsmnode` repository.
|
1. Open **PowerShell as Administrator** and navigate to the `acsm-fairifier` repository.
|
||||||
2. Create a `.env` file in the root of `acsmnode/`.
|
2. Create a `.env` file in the root of `acsm-fairifier/`.
|
||||||
3. **Securely store your network drive access credentials** in the `.env` file by adding the following lines:
|
3. **Securely store your network drive access credentials** in the `.env` file by adding the following lines:
|
||||||
```plaintext
|
```plaintext
|
||||||
CIFS_USER=<your-username>
|
CIFS_USER=<your-username>
|
||||||
@ -175,13 +175,9 @@ We recommend using Miniforge to manage your conda environments. Miniforge ensure
|
|||||||
2. Create the Environment from `environment.yml`
|
2. Create the Environment from `environment.yml`
|
||||||
After installing Miniforge, open **Miniforge Prompt** or a terminal with access to conda and run:
|
After installing Miniforge, open **Miniforge Prompt** or a terminal with access to conda and run:
|
||||||
```bash
|
```bash
|
||||||
cd path/to/Gitea/acsmnode
|
cd path/to/Gitea/acsm-fairifier
|
||||||
conda env create --file environment.yml
|
conda env create --file environment.yml
|
||||||
```
|
```
|
||||||
3. Activate the Environment
|
|
||||||
```bash
|
|
||||||
conda activate acsmnode_env
|
|
||||||
```
|
|
||||||
|
|
||||||
### Working with Jupyter Notebooks
|
### Working with Jupyter Notebooks
|
||||||
We now make the previously installed Python environment `acsmnode_env` selectable as a kernel in Jupyter's interface.
|
We now make the previously installed Python environment `acsmnode_env` selectable as a kernel in Jupyter's interface.
|
||||||
|
@ -20,9 +20,36 @@
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "code",
|
"cell_type": "code",
|
||||||
"execution_count": null,
|
"execution_count": 3,
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"outputs": [],
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"c:\\Users\\juanf\\OneDrive\\Documents\\Gitea\\acsmnode\\dima\n",
|
||||||
|
"c:\\Users\\juanf\\OneDrive\\Documents\\Gitea\\acsmnode\n",
|
||||||
|
"c:\\Users\\juanf\\OneDrive\\Documents\\Gitea\\acsmnode\\dima\n",
|
||||||
|
"c:\\Users\\juanf\\OneDrive\\Documents\\Gitea\\acsmnode\n",
|
||||||
|
"c:\\Users\\juanf\\OneDrive\\Documents\\Gitea\\acsmnode\\dima\n",
|
||||||
|
"c:\\Users\\juanf\\envs\\acsmnode_env\\python311.zip\n",
|
||||||
|
"c:\\Users\\juanf\\envs\\acsmnode_env\\DLLs\n",
|
||||||
|
"c:\\Users\\juanf\\envs\\acsmnode_env\\Lib\n",
|
||||||
|
"c:\\Users\\juanf\\envs\\acsmnode_env\n",
|
||||||
|
"\n",
|
||||||
|
"c:\\Users\\juanf\\envs\\acsmnode_env\\Lib\\site-packages\n",
|
||||||
|
"c:\\Users\\juanf\\envs\\acsmnode_env\\Lib\\site-packages\\win32\n",
|
||||||
|
"c:\\Users\\juanf\\envs\\acsmnode_env\\Lib\\site-packages\\win32\\lib\n",
|
||||||
|
"c:\\Users\\juanf\\envs\\acsmnode_env\\Lib\\site-packages\\Pythonwin\n",
|
||||||
|
"c:\\Users\\juanf\\OneDrive\\Documents\\Gitea\\acsmnode\n",
|
||||||
|
"c:\\Users\\juanf\\OneDrive\\Documents\\Gitea\\acsmnode\\third_party\\acsmProcessingSoftware\\src\n",
|
||||||
|
"c:\\Users\\juanf\\OneDrive\\Documents\\Gitea\\acsmnode\\third_party\\acsmProcessingSoftware\\src\\db\n",
|
||||||
|
"c:\\Users\\juanf\\OneDrive\\Documents\\Gitea\\acsmnode\\third_party\\acsmProcessingSoftware\\src\\iointerface\n",
|
||||||
|
"c:\\Users\\juanf\\OneDrive\\Documents\\Gitea\\acsmnode\\third_party\\acsmProcessingSoftware\n",
|
||||||
|
"workflow_acsm_data_JFJ_2024\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
"source": [
|
"source": [
|
||||||
"import sys\n",
|
"import sys\n",
|
||||||
"import os\n",
|
"import os\n",
|
||||||
@ -81,9 +108,58 @@
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "code",
|
"cell_type": "code",
|
||||||
"execution_count": null,
|
"execution_count": 5,
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"outputs": [],
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Manually loaded .env from: C:\\Users\\juanf\\OneDrive\\Documents\\Gitea\\acsmnode\\.env\n",
|
||||||
|
"Success : True\n",
|
||||||
|
"[Notice] All files already exist at ../data/collection_1_JFJ_2024_2025-07-04/, skipping copy.\n",
|
||||||
|
"\n",
|
||||||
|
"[Start] Data integration :\n",
|
||||||
|
"Source: ..\\data\\collection_1_JFJ_2024_2025-07-04\n",
|
||||||
|
"Destination: ..\\data\\collection_1_JFJ_2024_2025-07-04.h5\n",
|
||||||
|
"\n",
|
||||||
|
"Starting data transfer from instFolder: /ACSM_TOFWARE/2024\n",
|
||||||
|
"Completed transfer for //ACSM_TOFWARE/2024/calibration_params.yaml\n",
|
||||||
|
"Completed transfer for //ACSM_TOFWARE/2024/limits_of_detection.yaml\n",
|
||||||
|
"Completed transfer for //ACSM_TOFWARE/2024/validity_thresholds.yaml\n",
|
||||||
|
"[==================================================--------------------------------------------------] 50.0% ...\n",
|
||||||
|
"Completed data transfer for instFolder: /ACSM_TOFWARE/2024\n",
|
||||||
|
"Completed transfer for //ACSM_TOFWARE/2024/ACSM_JFJ_2024_meta.txt\n",
|
||||||
|
"Completed transfer for //ACSM_TOFWARE/2024/ACSM_JFJ_2024_timeseries.txt\n",
|
||||||
|
"Startdate 20240101000000\n",
|
||||||
|
"\n",
|
||||||
|
"Completed transfer for //ACSM_TOFWARE/2024/CH0001G.20240101000000.20250430141458.cpc.particle_number_concentration.aerosol.1y.1mn.CH02L_TSI_3772_JFJ.CH02L_CPC.lev1.nas\n",
|
||||||
|
"Completed transfer for //ACSM_TOFWARE/2024/Org_data_valid.csv\n",
|
||||||
|
"Completed transfer for //ACSM_TOFWARE/2024/Org_err_valid.csv\n",
|
||||||
|
"Completed transfer for //ACSM_TOFWARE/2024/Org_mz_valid.csv\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "stderr",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"c:\\Users\\juanf\\OneDrive\\Documents\\Gitea\\acsmnode\\dima\\instruments\\readers\\acsm_tofware_reader.py:112: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators (separators > 1 char and different from '\\s+' are interpreted as regex); you can avoid this warning by specifying engine='python'.\n",
|
||||||
|
" df = pd.read_csv(tmp_filename,\n",
|
||||||
|
"c:\\Users\\juanf\\OneDrive\\Documents\\Gitea\\acsmnode\\dima\\instruments\\readers\\acsm_tofware_reader.py:112: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators (separators > 1 char and different from '\\s+' are interpreted as regex); you can avoid this warning by specifying engine='python'.\n",
|
||||||
|
" df = pd.read_csv(tmp_filename,\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Completed transfer for //ACSM_TOFWARE/2024/Org_time_valid.csv\n",
|
||||||
|
"[====================================================================================================] 100.0% ...\n",
|
||||||
|
"Completed data transfer for instFolder: /ACSM_TOFWARE/2024\n",
|
||||||
|
"[End] Data integration\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
"source": [
|
"source": [
|
||||||
"path_to_config_file = '../campaignDescriptor.yaml'\n",
|
"path_to_config_file = '../campaignDescriptor.yaml'\n",
|
||||||
"paths_to_hdf5_files = get_campaign_data(path_to_config_file)\n",
|
"paths_to_hdf5_files = get_campaign_data(path_to_config_file)\n",
|
||||||
@ -353,7 +429,7 @@
|
|||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3",
|
"display_name": "acsmnode_env",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
@ -367,7 +443,7 @@
|
|||||||
"name": "python",
|
"name": "python",
|
||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.11.10"
|
"version": "3.11.13"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
|
Reference in New Issue
Block a user