From bae86371ec4a4e6b4cef27e6080956080e18ffc3 Mon Sep 17 00:00:00 2001 From: Sven Augustin Date: Sat, 12 Apr 2025 12:07:51 +0200 Subject: [PATCH] updated current state; added some checkboxes --- autodeploy/TODO.md | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/autodeploy/TODO.md b/autodeploy/TODO.md index 0712c09..55b67d6 100644 --- a/autodeploy/TODO.md +++ b/autodeploy/TODO.md @@ -1,6 +1,6 @@ # TODO -## Make envs available in jupytera +- [ ] ## Make envs available in jupytera There's [pixi-kernel](https://github.com/renan-r-santos/pixi-kernel) but it is specifically "per directory", which does not fit our use case. @@ -16,21 +16,25 @@ The only "problem" is in how jupyter shows the env names: Note: if the `envs` folder is called differently, no prefix is shown at all... This might be an alternative idea. -## Shared features +- [ ] ## Shared features Pixi supports "features" (i.e., sets of packages that allow the user to do XYZ). This would be very useful for, e.g., a feature like "be able to plot on jupytera", which gives the correct versions of ipympl, matplotlib, etc. We would need this to be a central file that is somehow made available and/or merged into everyone's individual files. -Pixi does not seem to have this option (yet? opened a [ticket](https://github.com/prefix-dev/pixi/issues/3524)). TOML does not allow to link in further files out of the box. However, there is [tomlincl](https://github.com/qdongxu/tomlincl). +Pixi does not seem to have this option (yet? see below). TOML does not allow to link in further files out of the box. However, there is [tomlincl](https://github.com/qdongxu/tomlincl). -## Running the setup script +Opened a [ticket](https://github.com/prefix-dev/pixi/issues/3524) and the devs proposed a solution. Need to test! + +- [x] ## Running the setup script Currently, there runs a deploy script as part of the deployment. We do NOT want to replace that script, but run after it. Can this be changed such that the deploy script runs our script at the end? Where would our script live? -## Timeout +**Solution**: Simon calls `/sf/daq/code/snek/autodeploy/setup.sh` at the end of the deploy script with `/sf/alvra/config/python` as working dir. + +- [x] ## Timeout Currently the timeout of the autodeployer is rather short as the expectation is that there are only "filesystem copy tasks" performed. @@ -38,7 +42,9 @@ Conda env creation can take a few minutes even with Pixi. Can the timeout be made that long without negative implications? Can this, for instance, block other tasks? -## Trigger only on change of specific file +**Solution**: Simon set up a separate docker container with increased time out of 5 mins (to be confirmed this is enough). + +- [ ] ## Trigger only on change of specific file In case of changes to the env, the lock file (`pixi.lock`) should be committed at the end of the setup script to have a history and reproducibility. @@ -46,7 +52,13 @@ Currently, this will trigger an additional autodeployer run which does nothing. Can we instead trigger the autodeployer only if `pixi.yml` changes? -## Mounts +- [ ] ## Commit from the Autodeployer + +In case of changes to the env, the lock file (`pixi.lock`) should be committed at the end of the setup script to have a history and reproducibility. + +We need to allow the autodeployer to make commits. + +- [x] ## Mounts ### Problem @@ -84,3 +96,4 @@ sudo umount /sf/alvra/config/python sudo rmdir -p /sf/alvra/config/python ``` +**Solution**: bind mounts do not work in unprivileged containers. Simon now mounts the console path when starting the docker container.