Document merlin_archive script
All checks were successful
Lint markdown files / run-markdownlint (pull_request) Successful in 3s
Build and deploy documentation / build-docs (pull_request) Successful in 5s
Build and deploy documentation / deploy-docs (pull_request) Has been skipped
Build and deploy documentation / build-docs (push) Successful in 5s
Build and deploy documentation / deploy-docs (push) Successful in 7s

Also adds a few fixes:
- Warn about QT discontinuation
- Update contact emails
This commit was merged in pull request #6.
This commit is contained in:
Spencer Bliven
2025-10-08 16:32:18 +02:00
parent f5886f91f0
commit 417ec607d0

View File

@@ -547,6 +547,10 @@ achieve the same via drag and drop operations
## Ingest, Archive and Retrieve with QT desktop application SciCat
!!! warning
The QT desktop application has been discontinued at PSI, as it is not compatible
with Redhat 8. The instructions below are retained for redhat 7 users.
### Important Update since April 14th 2022
You currently first need to get a token before you can use SciCat: the
@@ -686,7 +690,7 @@ called "Landing Pages", which are hosted on <https://doi.psi.ch> .
The file data itself data becomes available via the normal data export
System of the Ra cluster, which requires however a PSI account. If you
want to make the file data anonymously available you need to send a
corresponding request to <stephan.egli@psi.ch> for now. This process is
corresponding request to <scicat-help@lists.psi.ch> for now. This process is
planned to be automated in future.
For now all publication are triggered by a scientist explicitly,
@@ -772,6 +776,28 @@ script with the sourceFolder as an argunent
This script must be run by a person who has the rights to modify the
access rights in AFS.
### Repairing ACLs on Merlin7
Sometimes Merlin7 users may get an error "UserError (missing files) while archiving job"
during archiving, despite files existing at the sourceFolder location. This section
describes how to fix this.
Archiving data from merlin7 is normally done without copying data (`--nocopy`). This
requires file permissions be set correctly. This should be correct by default, but some
operations (such as `cp -p`) might override the default permissions (given as an NFSv4
ACL).
The merlin admins provide a script to repair permissions:
```sh
merlin_archive -d sourceFolder
```
This should automatically repair permissions for all files in `sourceFolder`.
If problems persist, please contact the Data Catalog admins at
<scicat-help@lists.psi.ch> and/or the merlin admins at <merlin-admins@lists.psi.ch>.
## Appendix
### Installation of Tools
@@ -1254,14 +1280,14 @@ chosen for the same quantity:
For each beamline this automation is done together with the experts
from the data catalog group and potentially with the help from the
controls /detector-integration groups. Please contact
<scicatarchivemanager@psi.ch> to get in touch.
<scicat-help@lists.psi.ch> to get in touch.
The recommended method is to define preparation scripts, which
automatically produce the files metadata.json and optionally
filelisting.txt or folderlisting.txt (for multiple datasets) as you
would do in the manual case described in the previous section.
Example of such scripts can be provided by the data catalog team,
please contact <scicatingestor@psi.ch> for further help. The effort to
please contact <scicat-help@lists.psi.ch> for further help. The effort to
implement such a system depends very much on the availability of the
meta data as well as on the effort to convert the existing metadata to
the data catalog format inside the converter processes. If the meta
@@ -1325,7 +1351,7 @@ chosen for the same quantity:
such a case the meta data extraction must be done as part of the part
beamline preparation scripts. Example of such HDF5 extraction scripts
exist which can the basis of a beamline specific solution, again
please contact <scicatingestor@psi.ch>. These scripts will mostly need
please contact <scicat-help@lists.psi.ch>. These scripts will mostly need
minimal adjustments for each beamline, mainly specifying the filter
conditions defining which of the meta data in the HDF5 file are to be
considered meta data for the data catalog.
@@ -1504,7 +1530,7 @@ possibilities are that the catalog system is not available,
e.g. during scheduled maintenance periods. All outputs describing the
reason for the failure are written to STDERR. Please have a look at
these outputs to understand what the reason for the failure was. If
you need help please contact <scicatingestor@psi.ch>
you need help please contact <scicat-help@lists.psi.ch>
Please note: it is the task of the wrapper scripts to test
for the return code and to repeat the command once all conditions for