diff --git a/README.md b/README.md index f345f5e..6f2a18e 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,30 @@ -# Not so fast!!! +# Not so fast! Read this! Hello! This is the read-me file with information for the code project. Please take some time to read it before going on, as it contains detailed description of code and methods used. So don't be hasty, and have a good look! -# Code Documentation -## Code Structure: +# Setup + +Make sure the following things are installed on your system: +1. `git` +2. `python` and the package manager `conda`, available through [miniforge](https://github.com/conda-forge/miniforge). +3. [`VSCode`](https://code.visualstudio.com/) (in principal, this is not required, but it makes things much easier...) + +Then, open a terminal and `cd` into the desired directory and run +``` +git clone git@gitea.psi.ch:tschan_d/Master_Thesis.git +``` +Then `cd` into the created directory via `cd Master_Thesis` and create the `conda` environment from the provided `environment.yml` file: + +``` +conda create --name MA --file environment.yml +``` +Lastly, make sure that `VSCode` is selecting the created interpreter by configuring the `settings.json`. The important line is currently set to `"python.defaultInterpreterPath": "C:/Users/tschan_d/AppData/Local/miniforge3/envs/MA/python"`. Adapt this setting so that the path points to the `conda` environment you just created. + +>Importantly, it may be necessary to add the current folder to the `PYTHONPATH` system variable in order for the code to run. For this, add the current directory to the `PYTHONPATH` system variable (there are several options to do this, one of the is shown [here](https://www.geeksforgeeks.org/python/pythonpath-environment-variable-in-python/)). + +Detailed descriptions of how to use `conda` and `git` are available below. + +# Code Structure and Documentation This code project is structured as follows: ``` Master_Thesis @@ -14,73 +35,105 @@ Master_Thesis | └───launch.json ├───defs | ├───__init__.py -| ├───computing.py -| └───visualising.py -├───docs +| ├───BTOF.py +| ├───LabDevices.py +| └───OFR.py +├───doc | └───defs -| ├───computing.html | ├───index.html -| └───visualising.html +| ├───BTOF.html +| ├───LabDevices.html +| └───OFR.html ├───run | ├───__init__.py -| ├───runfile_01.py -| └───... +| └───OFR_Analysis +| ├───20260612_SET02_EXP01 +| | └───202607023_OFR_Analysis_01.ipynb +| └───20260729_SET02_EXP02 ├───.env ├───.gitignore +├───.mplstyle ├───environment.yml +├───get_doc.bat ├───main.py └───README.md ``` -You are currently reading `README.md`, where the structure of this oroject is explained. You're in the right spot to start, then! +You are currently reading `README.md`, where the structure of this oroject is explained. Here is an explanation of what each folder contains: -### Auxilliary directories `.git` and `.vscode` -1. `.git` contains all files related to `git`. These files are usually not interesting, unless you need to go back to old commits or want to know more details. Best leave this alone. For a detailed description of how `git` (and with it the `.git` directory) works, see [`git` description](##Git). -2. `.vscode` contains all files relevant for environment setup to run this project in [Visual Studio Code](https://code.visualstudio.com/). These files are: - 1. `settings.json`: the setup file to set up VS Code in the desired configuration. Feel free to change these as you like. For example, the setting `"python.defaultInterpreterPath": "C:/Users/tschan_d/AppData/Local/miniforge3/envs/MA/python"` needs to be changed to point to the path of the Python interpreter you select. - 2. `launch.json`: the launch configuration file for the project, in which it is specified how VSCode should behave when you launch a job: that is, when you want to run a file or debug it. Again, feel free to customise these settings. -3. `defs` contains source code for data analysis: - 1. `__init__.py` is a file to structure the project like a package. This file is empty, but it is needed such that relative imports work properly. - 2. `defs.py` is the file defining classed and methods used to analyse data. -4. `vis` contains source code for data visualisation. +### Auxilliary directories `./.git` and `./.vscode` +1. `.git` contains all files related to version control via `git`. These files are usually not interesting, unless you need to go back to old commits or want to know more details. Best leave this alone. +2. `.vscode` contains all files relevant for environment setup to run this project in `VSCode`. These files are: + 1. `settings.json`: the setup file to set up VS Code in the desired configuration. Feel free to change these as you like. + 2. `launch.json`: the launch configuration file for the project, in which it is specified how VSCode should behave when you launch a job: that is, when you want to run a file or debug it. Adapt them as needed. -# Coding help -## Conda +### Directories `./defs` and `./doc` +1. `./defs` contains source code for data analysis. This includes: + 1. `__init__.py` is a file to structure the project like a package. This file contains just a [docstring](https://peps.python.org/pep-0257/) and is empty otherwise. The docstring contains the description of the module in the code documentation (see below). This file is needed so that relative imports work properly. + 2. `BTOF.py` contains all classes and methods necessary to analyse the BTOF output data. These methods are only for fully calibrated and processed BTOF data that result from IGOR/Tofwerk workflows. + 2. `LabDevices.py` contains the classes and methods required to analyse the data from devices in the lab. Those devices are the Teledyne CO2 monitor as well as the RH/T sensor. + 3. `OFR.py` contains the classes and methods required to analyse data from the OFR. + +>**Note**: proper code documentation for this "package" (it is not a package, but it is structured like one) is available under `./doc/defs/index.html`. Double-click on that file to open a static webpage in your browser that displays technical code documentation. + +### Directory `run` and its subdirectories +1. `__init__.py` is an empty file required to import functions from this module (**to be checked**). +2. `./run/OFRAnalysis` contains subdirectories such as `./run/OFRAnalysis/20260612_SET02_EXP01/` that in turn contain pyhton notebookes such as `./run/OFRAnalysis/20260612_SET02_EXP01/202607023_SET02_EXP01.ipynb` where the data analysis is executed and visualised. + +### Other files +1. `.env` is a setup file used by `VSCode` to adjust the `PYTHONPATH` variable (**to be checked**). +2. `.gitignore` is a file helping to maintain cleanliness in the `git` remote repository on the server. This file excludes certain file types, such as `.pdf` or `.svg` files (that are created when compiling figures), from being tracked by `git`. +3. `.mplstyle` is a style sheet for [`matplotlib`](https://matplotlib.org/) to ensure that all figures look alike. + +>**Note**: the `.mplstyle` file uses `LaTeX` to render text in figures. If this is not available on your machine, do not use the `.mplstyle` style. It is used by including the line of code `plt.style.use(.mplstyle)` in your code when you have imported `import matplotlib.pyplot as plt` in yout `python` code. If not using this style sheet, ensure that axis labels, legend entries, text and titles to not include any `TeX`! + +4. `environment.yml` the file containing all information on the `python` virtual environment used in this project, see also [creating environments from `.yml` files](https://docs.conda.io/projects/conda/en/stable/user-guide/tasks/creating-projects.html)' +5. `get_doc.bat` is a Windos batch file to that can be run in a terminal through `Start-Process get_dic.bat`. For this, first `cd` into the project home directory, and then execute the batch script. This batch job calls `pdoc` to compile code documentation from provided docstrings. The resulting `.html` files can be found in the directory `.doc/defs/`. If you compiled the project python environment from the provided `environment.yml` file, `pdoc` is automatically installed with it. +6. `main.py` is a file used to debug code or to create a script that may be used for analysis. It is included to maintain the "package"-like structure of the code, but it may be used for whatever purpose you need it for! +7. `README.md` - this file. Iy you're reading this right now, I guess there's no need to explain this further... + +# Overview over `conda` Extensive documentation is available on [docs.conda.io](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html). -1. Get info on conda environments: `conda info --envs` or `conda env list`; -2. Create new conda environments: `conda create --name ` where `` is replaced with the name; Additional flags include `python=3.0.0` indicating the python version; -3. Creating an environment from a `.yaml` file: `conda create --name --file path/to/file/environment.yaml`, or navigate directly to the good location via `cd path/to/file`; -4. Exporting a python environment to a `.yml` file: `conda env export --name --format=environment-yaml --file=environment.yaml`; -5. Updating an environment: `conda env update --file environment.yaml --prune`, where `--prune` removes any unused dependencies; -6. Activating a conda environment: `conda activate `; -7. Deactivate an environment: `conda deactivate`; -8. List packages in environment: if the environment is not activated, run `conda list -n `, else (if it is activated) run `conda list`; -9. Add a channel: `conda config --prepend channels new_channel` to add a channel to top of the channel list, or run `conda config --append channels new_channel` to append a channel to the bottom of the channel list; -10. Installing packages: `conda install --name numpy scipy ...` to install packages `numpy`, `scipy` etc. into the environment ``; Package versions may be specified like this `numpy=3.4.2`; -11. Update packages: run `conda update --name package` to update `package` in ``; updating python is `conda update python` and updating conda itself is `conda update conda`; -12. Pinning a package: add a file named `pinned` to the `conda-meta` directory of the enviornment, and add the following lines to the file `package 1.7.*` to restrict the package to remain in the `1.7` series, or `package==1.5.4` to force the package to remain at exactly the specified version; The pinning may be overrun by `conda update package --no-pin`. -13. Remove package: `conda remove -n package` to remove `package` from ``; -## Git -This ensures that there is a connection to the psi network, use the vpn domain `vpn.psi.ch`. -1. Add the private key to the ssh agent via the following snippet on mac -``` -eval "$(ssh-agent -s)" -ssh-add ~/.ssh/ -ssh -T git@gitea.psi.ch -``` -or windows +|Command |Description | +| :-------------------------------------------- | :----------------------------------------------- | +| `conda info --envs` or `conda env list`| Get info on your conda environments | +| `conda list --name my_env` | List all the installed packages in the environment `my_env` | +| ` conda activate my_env` | Activate environment `my_env` | +| `conda install --name my_env package1 package2` | Install packages `package1` and `package2` into the environment `my_env` (this works for arbitrarily many packages). If environment is activated, there is no need to add `--name my_env`. **Note**: when there are activation problems with the environment, packages will be installed into `base` and your interpreter may *not* see them. Package versions may be specified like this `numpy=3.4.2` | +| `conda update --name package` | Update package `package` in environment `my_env` | +| `conda update python` | Update `python` itself | +| `conda update conda` | Update conda itself | +| `conda deactivate` | Deactivate environment | +| `conda remove --name my_env package` | Remove package `package` from `my_env` | +| `conda create --name my_env` | Create new `conda` environment called `my_env`. Adding the flag `package=1.2.3` directly installs the desired packages into the environment. This also works to specify the python version used in this environment via `python=3.1.3` for instance | +| `conda create --name my_env --file env.yml` | Create conda environment named `my_env` from `.yml` file `env.yml` | +| `conda env export --name my_env --file=eny.yml` | Export environment `my_env` into a `.yml` file | +| `conda update --name my_env` | Update the conda environment `my_env` | +| `conda env update --file environment.yaml --prune` | Update the environment associated with the `.yml` file, where `--prune` removes any unused dependencies | +| `conda config --prepend channels new_channel` | Add channel `new_channel` to channel list. **Note**: for licensing reasons, only use `conda-forge` channel. This is specified in the `environment.yml` file | +| | Pinning a package: add a file named `pinned` to the `conda-meta` directory of the environment. Adding the line `package 1.7.*` restricts `package` to remain in the `1.7` series when updating. Adding the line `package==1.5.4` forces the package to remain at exactly the specified version. The pinning may be overrun by `conda update package --no-pin` | -``` -eval "$(ssh-agent -s)" -ssh-add ~/.ssh/ -ssh -T git@gitea.psi.ch -``` -2. To create a local repository, clone the remote one via `git clone git@gitea.psi.ch:tschan_d/Master_Thesis.git`. This command creates a new local folder named `repo`, initialises a git repo within it, a remote named `origin` is created, pointing to the connected server, all of the files and and commits are downloaded and the default branch is checked out. -3. Use `git fetch git@gitea.psi.ch:tschan_d/Master_Thesis.git` to retrieve new work on the remote, without merging everything into the local branches. -4. use `git merge REMOTE-NAME/BRANCH-NAME` to merge a remote branch with the local branch. -5. Steps 3 and 4 may be executed more succinctly via `git pull REMOTE-NAME BRANCH-NAME` to grab online updates and merge them into the local branches. -6. Local work may be added in a snapshot using either `git add .` to add all files under the current directory to the index, or using specific files `git add file1 file2`. It can be checked what is to be committed using the `git diff --cached`. If `--cached` is omitted, `git diff` will show any changes made but not yet added to the index. -7. Run `git commit`, which will prompt for a message describing the made changes, and then record a new version of the project. -8. To combine steps 6 and 7, run `git commit -a` to notice any modified (but not new) files, add them to the index, and commit. -9. To get an overview over the history, run `git log` or `git log -p`. It may be useful to run `git log --stat --summary`. \ No newline at end of file +# Overview over `git` +Extensive documentation is available at [git-scm.dev](https://git-scm.dev/doc). + +### Using `git`: + +**Prerequisites**: + +1. Ensure that there is a connection to the psi network, either by *being* on the PSI network, or by using the `vpn` with domain `vpn.psi.ch`. +2. Ensure there is a valid `ssh` key under `C:/Users/username/.ssh/`. If there is no valid key, follow the instructions to create a `ssh` key on the PSI Intranet. +3. Ensure that the private key is added to your `ssh-agent` and that the oublic key is pointing to the remote repository. You can check the connection via `ssh -T git@gitea.psi.ch` + +Then git operates as follows: + +| Command | Description | +| ------- | ----------- | +| `git clone git@gitea.psi.ch:username/repo` | initialises a `git` repo in your current directory (else `cd` into the deisred one), creates a remote `origin` and checks out the default branch | +| `git fetch` | Fetch changes from remote | +| `git merge` | Merge changes from remote on top of local checked-out branch | +| `git pull` | Execute `fetch` and `merge` in a single command | +| `git add file.ext` | Stage changes in `file.ext` if it has been modified | +| `git add .` | Stage changes of all modified files in `git` repo | +| `git commit -m "Message"` | Commit the staged changes with the pertaining commit message "Message" | +| `git push -u origin main` | Push changes to remote on branch `main`. Normally it suffices to just run `git push`, but specifying the branches is required when there are several of them... | \ No newline at end of file diff --git a/run/runfile_01.py b/run/runfile_01.py deleted file mode 100644 index e69de29..0000000