Document merlin_archive script
All checks were successful
Lint markdown files / run-markdownlint (pull_request) Successful in 3s
Build and deploy documentation / build-docs (pull_request) Successful in 5s
Build and deploy documentation / deploy-docs (pull_request) Has been skipped
Build and deploy documentation / build-docs (push) Successful in 5s
Build and deploy documentation / deploy-docs (push) Successful in 7s
All checks were successful
Lint markdown files / run-markdownlint (pull_request) Successful in 3s
Build and deploy documentation / build-docs (pull_request) Successful in 5s
Build and deploy documentation / deploy-docs (pull_request) Has been skipped
Build and deploy documentation / build-docs (push) Successful in 5s
Build and deploy documentation / deploy-docs (push) Successful in 7s
Also adds a few fixes: - Warn about QT discontinuation - Update contact emails
This commit was merged in pull request #6.
This commit is contained in:
@@ -547,6 +547,10 @@ achieve the same via drag and drop operations
|
||||
|
||||
## Ingest, Archive and Retrieve with QT desktop application SciCat
|
||||
|
||||
!!! warning
|
||||
The QT desktop application has been discontinued at PSI, as it is not compatible
|
||||
with Redhat 8. The instructions below are retained for redhat 7 users.
|
||||
|
||||
### Important Update since April 14th 2022
|
||||
|
||||
You currently first need to get a token before you can use SciCat: the
|
||||
@@ -686,7 +690,7 @@ called "Landing Pages", which are hosted on <https://doi.psi.ch> .
|
||||
The file data itself data becomes available via the normal data export
|
||||
System of the Ra cluster, which requires however a PSI account. If you
|
||||
want to make the file data anonymously available you need to send a
|
||||
corresponding request to <stephan.egli@psi.ch> for now. This process is
|
||||
corresponding request to <scicat-help@lists.psi.ch> for now. This process is
|
||||
planned to be automated in future.
|
||||
|
||||
For now all publication are triggered by a scientist explicitly,
|
||||
@@ -772,6 +776,28 @@ script with the sourceFolder as an argunent
|
||||
This script must be run by a person who has the rights to modify the
|
||||
access rights in AFS.
|
||||
|
||||
### Repairing ACLs on Merlin7
|
||||
|
||||
Sometimes Merlin7 users may get an error "UserError (missing files) while archiving job"
|
||||
during archiving, despite files existing at the sourceFolder location. This section
|
||||
describes how to fix this.
|
||||
|
||||
Archiving data from merlin7 is normally done without copying data (`--nocopy`). This
|
||||
requires file permissions be set correctly. This should be correct by default, but some
|
||||
operations (such as `cp -p`) might override the default permissions (given as an NFSv4
|
||||
ACL).
|
||||
|
||||
The merlin admins provide a script to repair permissions:
|
||||
|
||||
```sh
|
||||
merlin_archive -d sourceFolder
|
||||
```
|
||||
|
||||
This should automatically repair permissions for all files in `sourceFolder`.
|
||||
|
||||
If problems persist, please contact the Data Catalog admins at
|
||||
<scicat-help@lists.psi.ch> and/or the merlin admins at <merlin-admins@lists.psi.ch>.
|
||||
|
||||
## Appendix
|
||||
|
||||
### Installation of Tools
|
||||
@@ -1254,14 +1280,14 @@ chosen for the same quantity:
|
||||
For each beamline this automation is done together with the experts
|
||||
from the data catalog group and potentially with the help from the
|
||||
controls /detector-integration groups. Please contact
|
||||
<scicatarchivemanager@psi.ch> to get in touch.
|
||||
<scicat-help@lists.psi.ch> to get in touch.
|
||||
|
||||
The recommended method is to define preparation scripts, which
|
||||
automatically produce the files metadata.json and optionally
|
||||
filelisting.txt or folderlisting.txt (for multiple datasets) as you
|
||||
would do in the manual case described in the previous section.
|
||||
Example of such scripts can be provided by the data catalog team,
|
||||
please contact <scicatingestor@psi.ch> for further help. The effort to
|
||||
please contact <scicat-help@lists.psi.ch> for further help. The effort to
|
||||
implement such a system depends very much on the availability of the
|
||||
meta data as well as on the effort to convert the existing metadata to
|
||||
the data catalog format inside the converter processes. If the meta
|
||||
@@ -1325,7 +1351,7 @@ chosen for the same quantity:
|
||||
such a case the meta data extraction must be done as part of the part
|
||||
beamline preparation scripts. Example of such HDF5 extraction scripts
|
||||
exist which can the basis of a beamline specific solution, again
|
||||
please contact <scicatingestor@psi.ch>. These scripts will mostly need
|
||||
please contact <scicat-help@lists.psi.ch>. These scripts will mostly need
|
||||
minimal adjustments for each beamline, mainly specifying the filter
|
||||
conditions defining which of the meta data in the HDF5 file are to be
|
||||
considered meta data for the data catalog.
|
||||
@@ -1504,7 +1530,7 @@ possibilities are that the catalog system is not available,
|
||||
e.g. during scheduled maintenance periods. All outputs describing the
|
||||
reason for the failure are written to STDERR. Please have a look at
|
||||
these outputs to understand what the reason for the failure was. If
|
||||
you need help please contact <scicatingestor@psi.ch>
|
||||
you need help please contact <scicat-help@lists.psi.ch>
|
||||
|
||||
Please note: it is the task of the wrapper scripts to test
|
||||
for the return code and to repeat the command once all conditions for
|
||||
|
||||
Reference in New Issue
Block a user