rugnux: report what --model found, and attribute GEMMI's bundled headers
Two things left open by the previous commit. A model that was validated left nothing in the results report. R-work, R-free, the bulk-solvent parameters and the map filenames went to the log and nowhere else, so a run given --model was, in the file anyone reads afterwards, indistinguishable from one that was not - and the WARNING line added last commit was the only trace of a model that failed. Section 10. MODEL VALIDATION now carries it: MODEL_FILE, the space group the model is in, R_WORK / R_FREE with their reflection counts, the bulk-solvent k_sol / b_sol and overall scale, the mean 2Fo-Fc density at the atom centres, the hand and indexing operators the written reflections were brought into the model's frame with, and MAPS_PREFIX. A model that could not be used gets MODEL_VALIDATION= NOT_PERFORMED and the reason instead, in the shape the merging section already uses. A run that asked for no model has no section at all. That moves WARNINGS from 10 to 11, which is what REPORT_VERSION is for: it is now 4. No existing key was renamed and no table column moved, so a consumer that greps for what it needs is unaffected; one that keys on the section number of WARNINGS is not, hence the bump. The four third-party headers GEMMI has always bundled - fast_float, half, pocketfft and tinydir - were never listed in THIRD_PARTY_NOTICES.md, riding instead under GEMMI's single MPL-2.0 row. They are other authors' code under other licences (Apache-2.0 OR MIT OR BSL-1.0, MIT, BSD-3-Clause, BSD-2-Clause), and MPL-2.0 does not speak for them. Each now has its own row and its own licence text, alongside the PEGTL and sajson entries added with the CIF parser, and a note records why the six are listed separately from GEMMI. Their terms live in the headers rather than in LICENSE files, so five of the six texts are kept by hand; COLLECT.sh says so and copies the one - PEGTL - that ships a file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016WmryXe8ASbNi632sUMfsa
This commit is contained in:
Claude Opus 5
@@ -50,6 +50,10 @@ These are copied into the source tree (see the path) rather than fetched.
|
||||
| [GEMMI](https://github.com/project-gemmi/gemmi) | `gemmi_gph/` | Global Phasing Ltd. | MPL-2.0 | [gemmi.txt](licenses/gemmi.txt) |
|
||||
| [PEGTL](https://github.com/taocpp/PEGTL) | `gemmi_gph/gemmi/third_party/tao/` | Dr. Colin Hirsch and Daniel Frey | MIT | [pegtl.txt](licenses/pegtl.txt) |
|
||||
| [sajson](https://github.com/chadaustin/sajson) | `gemmi_gph/gemmi/third_party/sajson.h` | Chad Austin | MIT | [sajson.txt](licenses/sajson.txt) |
|
||||
| [fast_float](https://github.com/fastfloat/fast_float) | `gemmi_gph/gemmi/third_party/fast_float.h` | The fast_float authors (Daniel Lemire et al.) | Apache-2.0 OR MIT OR BSL-1.0 | [fast-float.txt](licenses/fast-float.txt) |
|
||||
| [half](https://half.sourceforge.net/) | `gemmi_gph/gemmi/third_party/half.hpp` | Christian Rau | MIT | [half.txt](licenses/half.txt) |
|
||||
| [pocketfft](https://github.com/mreineck/pocketfft) | `gemmi_gph/gemmi/third_party/pocketfft_hdronly.h` | Max-Planck-Society; Peter Bell; MIT (FFTW-derived parts) | BSD-3-Clause | [pocketfft.txt](licenses/pocketfft.txt) |
|
||||
| [tinydir](https://github.com/cxong/tinydir) | `gemmi_gph/gemmi/third_party/tinydir.h` | Cong Xu, Lautis Sun, Baudouin Feildel, Andargor | BSD-2-Clause | [tinydir.txt](licenses/tinydir.txt) |
|
||||
| [traccc (ACTS)](https://github.com/acts-project/traccc) | `image_analysis/spot_finding/StrongPixelSet.cpp`, `SpotExtractorGPU.cu` | CERN, for the benefit of the ACTS project | MPL-2.0 | [traccc.txt](licenses/traccc.txt) |
|
||||
| [xbflash.qspi](https://github.com/Xilinx/XRT) | `tools/xbflash.qspi/` | Xilinx / AMD | Apache-2.0 | [xbflash-qspi.txt](licenses/xbflash-qspi.txt) |
|
||||
| [wingetopt](https://github.com/alex85k/wingetopt) | `tools/wingetopt/` | Todd C. Miller; The NetBSD Foundation | ISC AND BSD-2-Clause | [wingetopt.txt](licenses/wingetopt.txt) |
|
||||
@@ -80,6 +84,12 @@ served frontend, so the shipped web UI carries its own attribution.
|
||||
* **MPL-2.0** (Eigen, GEMMI, libzmq, traccc): file-level copyleft. GEMMI is vendored in `gemmi_gph/`
|
||||
in trimmed form; libzmq is fetched at build time; Eigen is provided externally (header-only). The
|
||||
corresponding source is available from each project upstream.
|
||||
* **GEMMI's own bundled third-party headers** — PEGTL, sajson, fast_float, half, pocketfft and
|
||||
tinydir, all under `gemmi_gph/gemmi/third_party/` — are listed separately above rather than being
|
||||
absorbed into GEMMI's row: they are other authors' code under other licences (MIT, BSD and
|
||||
Apache/MIT/BSL), and GEMMI's MPL-2.0 does not speak for them. Their terms are carried in the
|
||||
headers themselves rather than in `LICENSE` files, so the `licenses/*.txt` copies are kept by hand;
|
||||
only PEGTL ships a `LICENSE`, which `COLLECT.sh` copies.
|
||||
* **traccc** is the one entry that is not a vendored directory. Its sparse connected-component
|
||||
labelling enters two otherwise first-party files: `StrongPixelSet.cpp` adapts the SparseCCL source,
|
||||
and `SpotExtractorGPU.cu` follows the design of its GPU counterpart. MPL-2.0 is file-level, so both
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.
|
||||
|
||||
* The rugnux results report records how the run was invoked, what it cost and what it ran on: `COMMAND_LINE=` is the command line as one shell-ready line, `WALL_TIME=` the whole invocation in seconds, and `GPU_COUNT=` / `GPU=` how many GPUs were visible and which ones. The total wall time is also printed on stdout, next to the processing time it is slightly larger than.
|
||||
* The rugnux results report has a `10. MODEL VALIDATION` section when `--model` was given: R-work / R-free with their reflection counts, the bulk-solvent and scale parameters, the mean 2Fo-Fc density at the atom centres, the reindexing operators and where the maps were written - or `MODEL_VALIDATION= NOT_PERFORMED` and the reason, when the model could not be used. `REPORT_VERSION` is 4; `WARNINGS` moves from section 10 to 11 and no existing key changed.
|
||||
* `rugnux --model` reads an mmCIF coordinate file as well as a PDB one, gzipped or not, taking the format from the file's content rather than its name. A `.cif` model was rejected before, and the run then finished successfully with no R-free and no maps.
|
||||
* A model `rugnux --model` cannot use - unreadable, or without atoms, a unit cell or a usable space group - is reported as a `WARNING:` line in the results report instead of only in the log, so a run that was asked to validate and could not is no longer indistinguishable from one that was never asked.
|
||||
* rugnux says which GPUs it can see before it starts processing, so a machine that turns out to have none - a driver mismatch, a leftover `CUDA_VISIBLE_DEVICES` - is noticed while there is still time to stop rather than after a run that took far longer than it should have.
|
||||
|
||||
+10
-1
@@ -553,7 +553,9 @@ Rates, per-image costs and progress remain on stdout only.
|
||||
|
||||
Sections, in order: `1. DATA SET`, `2. INDEXING`, `3. GEOMETRY POST-REFINEMENT` (rotation only),
|
||||
`4. SPACE GROUP DETERMINATION`, `5. SCALING AND MERGING`, `6. TWINNING`, `7. RADIATION DAMAGE`,
|
||||
`8. SWEEP QUALITY`, `9. DIFFRACTION ANISOTROPY`, `10. WARNINGS`.
|
||||
`8. SWEEP QUALITY`, `9. DIFFRACTION ANISOTROPY`, `10. MODEL VALIDATION` (only with `--model`),
|
||||
`11. WARNINGS`. Numbers are fixed: a section that does not apply is left out and the ones after it
|
||||
keep their numbers.
|
||||
|
||||
**`SPOT_RESOLUTION_ESTIMATE=`** in section 1 is how far the merged data are expected to reach, read
|
||||
off the found spots alone — no lattice, no integration, no merge — so it is there on a run that never
|
||||
@@ -775,6 +777,13 @@ cell or no usable space group, does not fail the run: it is logged, and the resu
|
||||
`WARNING: Model validation did not run: …` line, so a run that silently produced no R-free and no
|
||||
maps cannot be mistaken for one that was never given `--model`.
|
||||
|
||||
Either way the results report carries a **`10. MODEL VALIDATION`** section: `R_WORK=` / `R_FREE=`
|
||||
with their reflection counts, the bulk-solvent and overall scale parameters, the mean 2Fo-Fc density
|
||||
at the atom centres, the reindexing operators the written reflections were brought into the model's
|
||||
frame with, and `MAPS_PREFIX=`; or `MODEL_VALIDATION= NOT_PERFORMED` with
|
||||
`MODEL_VALIDATION_REASON=` when the model could not be used. A run given no `--model` has no such
|
||||
section at all.
|
||||
|
||||
It is a *data-quality lens*, independent of the internal statistics: R-free measures the merged
|
||||
intensities against external truth, where CC1/2 and R<sub>meas</sub> only measure them against
|
||||
themselves. It also settles the two things merged intensities alone cannot: the enantiomorph (data
|
||||
|
||||
+5
-3
@@ -69,9 +69,11 @@ copy_license bitshuffle-hperf.txt compression/bitshuffle_hperf/LICENSE-APACHE
|
||||
copy_license lz4.txt compression/lz4/LICENSE
|
||||
copy_license xilinx-hls-headers.txt fpga/include/LICENSE.HLS_Headers
|
||||
copy_license gemmi.txt gemmi_gph/LICENSE.txt
|
||||
# GEMMI bundles PEGTL under its own include tree and under its own (MIT) licence; the CIF parser
|
||||
# rugnux reads mmCIF models with is built on it. sajson has no LICENSE file of its own - its terms
|
||||
# are the comment block at the top of the header - so licenses/sajson.txt is kept by hand.
|
||||
# GEMMI bundles six third-party headers under its own include tree, each under its own licence
|
||||
# (see THIRD_PARTY_NOTICES.md). Only PEGTL ships a LICENSE file; sajson, fast_float, half, pocketfft
|
||||
# and tinydir carry their terms in a comment at the top of the header, so licenses/sajson.txt,
|
||||
# fast-float.txt, half.txt, pocketfft.txt and tinydir.txt are kept by hand - re-extract them from the
|
||||
# headers when GEMMI is updated.
|
||||
copy_license pegtl.txt gemmi_gph/gemmi/third_party/tao/LICENSE
|
||||
copy_license xbflash-qspi.txt tools/xbflash.qspi/LICENSE
|
||||
copy_license wingetopt.txt tools/wingetopt/LICENSE
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
fast_float by Daniel Lemire
|
||||
fast_float by João Paulo Magalhaes
|
||||
|
||||
|
||||
with contributions from Eugene Golushkov
|
||||
with contributions from Maksim Kita
|
||||
with contributions from Marcin Wojdyr
|
||||
with contributions from Neal Richardson
|
||||
with contributions from Tim Paine
|
||||
with contributions from Fabio Pellacini
|
||||
with contributions from Lénárd Szolnoki
|
||||
with contributions from Jan Pharago
|
||||
with contributions from Maya Warrier
|
||||
with contributions from Taha Khokhar
|
||||
with contributions from Anders Dalvander
|
||||
|
||||
|
||||
Licensed under the Apache License, Version 2.0, or the
|
||||
MIT License or the Boost License. This file may not be copied,
|
||||
modified, or distributed except according to those terms.
|
||||
|
||||
MIT License Notice
|
||||
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2021 The fast_float authors
|
||||
|
||||
Permission is hereby granted, free of charge, to any
|
||||
person obtaining a copy of this software and associated
|
||||
documentation files (the "Software"), to deal in the
|
||||
Software without restriction, including without
|
||||
limitation the rights to use, copy, modify, merge,
|
||||
publish, distribute, sublicense, and/or sell copies of
|
||||
the Software, and to permit persons to whom the Software
|
||||
is furnished to do so, subject to the following
|
||||
conditions:
|
||||
|
||||
The above copyright notice and this permission notice
|
||||
shall be included in all copies or substantial portions
|
||||
of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
|
||||
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
|
||||
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
|
||||
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
|
||||
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
||||
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
||||
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
|
||||
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
|
||||
DEALINGS IN THE SOFTWARE.
|
||||
|
||||
Apache License (Version 2.0) Notice
|
||||
|
||||
Copyright 2021 The fast_float authors
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
|
||||
BOOST License Notice
|
||||
|
||||
Boost Software License - Version 1.0 - August 17th, 2003
|
||||
|
||||
Permission is hereby granted, free of charge, to any person or organization
|
||||
obtaining a copy of the software and accompanying documentation covered by
|
||||
this license (the "Software") to use, reproduce, display, distribute,
|
||||
execute, and transmit the Software, and to prepare derivative works of the
|
||||
Software, and to permit third-parties to whom the Software is furnished to
|
||||
do so, all subject to the following:
|
||||
|
||||
The copyright notices in the Software and this entire statement, including
|
||||
the above license grant, this restriction and the following disclaimer,
|
||||
must be included in all copies of the Software, in whole or in part, and
|
||||
all derivative works of the Software, unless such copies or derivative
|
||||
works are solely in the form of machine-executable object code generated by
|
||||
a source language processor.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
|
||||
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
|
||||
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
|
||||
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
|
||||
DEALINGS IN THE SOFTWARE.
|
||||
@@ -0,0 +1,15 @@
|
||||
half - IEEE 754-based half-precision floating-point library.
|
||||
|
||||
Copyright (c) 2012-2025 Christian Rau <rauy@users.sourceforge.net>
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation
|
||||
files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy,
|
||||
modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the
|
||||
Software is furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
|
||||
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
|
||||
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
@@ -0,0 +1,41 @@
|
||||
This file is part of pocketfft.
|
||||
|
||||
Copyright (C) 2010-2024 Max-Planck-Society
|
||||
Copyright (C) 2019-2020 Peter Bell
|
||||
|
||||
For the odd-sized DCT-IV transforms:
|
||||
Copyright (C) 2003, 2007-14 Matteo Frigo
|
||||
Copyright (C) 2003, 2007-14 Massachusetts Institute of Technology
|
||||
|
||||
For the prev_good_size search:
|
||||
Copyright (C) 2024 Tan Ping Liang, Peter Bell
|
||||
|
||||
For the safeguards against integer overflow in good_size search:
|
||||
Copyright (C) 2024 Cris Luengo
|
||||
|
||||
Authors: Martin Reinecke, Peter Bell
|
||||
|
||||
All rights reserved.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without modification,
|
||||
are permitted provided that the following conditions are met:
|
||||
|
||||
* Redistributions of source code must retain the above copyright notice, this
|
||||
list of conditions and the following disclaimer.
|
||||
* Redistributions in binary form must reproduce the above copyright notice, this
|
||||
list of conditions and the following disclaimer in the documentation and/or
|
||||
other materials provided with the distribution.
|
||||
* Neither the name of the copyright holder nor the names of its contributors may
|
||||
be used to endorse or promote products derived from this software without
|
||||
specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
|
||||
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
|
||||
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
||||
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
|
||||
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
|
||||
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
|
||||
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
|
||||
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
|
||||
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
@@ -0,0 +1,26 @@
|
||||
Copyright (c) 2013-2021, tinydir authors:
|
||||
- Cong Xu
|
||||
- Lautis Sun
|
||||
- Baudouin Feildel
|
||||
- Andargor <andargor@yahoo.com>
|
||||
All rights reserved.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are met:
|
||||
|
||||
1. Redistributions of source code must retain the above copyright notice, this
|
||||
list of conditions and the following disclaimer.
|
||||
2. Redistributions in binary form must reproduce the above copyright notice,
|
||||
this list of conditions and the following disclaimer in the documentation
|
||||
and/or other materials provided with the distribution.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
|
||||
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
|
||||
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
||||
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
|
||||
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
|
||||
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
|
||||
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
|
||||
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
|
||||
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
@@ -56,6 +56,7 @@ ModelValidationResult ValidateAgainstModel(const std::vector<MergedReflection> &
|
||||
std::optional<int> data_space_group_number,
|
||||
bool probe_indexing_ambiguity) {
|
||||
ModelValidationResult result;
|
||||
result.model_path = model_path;
|
||||
|
||||
// --- read the atomic model ---
|
||||
gemmi::Structure st;
|
||||
|
||||
@@ -17,6 +17,7 @@ class Logger;
|
||||
// Result of validating merged intensities against an atomic model.
|
||||
struct ModelValidationResult {
|
||||
bool ok = false;
|
||||
std::string model_path; // the file that was asked for, so the report can name it
|
||||
// Why it did not run, when ok is false: the file could not be read, had no atoms, no cell or no
|
||||
// usable space group. A caller that asked for a model must be able to say so rather than finish
|
||||
// quietly with no R-free and no maps, which reads exactly like a run that was never asked.
|
||||
|
||||
+41
-3
@@ -19,7 +19,7 @@
|
||||
namespace {
|
||||
// The version of this file format. Bumped when a key is renamed or removed, a table column moves,
|
||||
// or a reason code changes meaning - a consumer can gate on it.
|
||||
constexpr int REPORT_VERSION = 3;
|
||||
constexpr int REPORT_VERSION = 4;
|
||||
|
||||
const char *BANNER = " ******************************************************************************";
|
||||
|
||||
@@ -344,11 +344,49 @@ std::string RenderResultReport(const std::string &output_prefix,
|
||||
*std::min_element(an.d_min_axis, an.d_min_axis + 3)));
|
||||
}
|
||||
|
||||
// --------------------------------------------------------------- 10. WARNINGS
|
||||
// ------------------------------------------------------- 10. MODEL VALIDATION
|
||||
if (result.model_validation.has_value()) {
|
||||
const auto &mv = *result.model_validation;
|
||||
Section(os, "10. MODEL VALIDATION");
|
||||
Key(os, "MODEL_FILE", mv.model_path);
|
||||
if (!mv.ok) {
|
||||
Key(os, "MODEL_VALIDATION", "NOT_PERFORMED");
|
||||
Key(os, "MODEL_VALIDATION_REASON", mv.failure_reason);
|
||||
os << "\n A model was given but could not be used, so there are no R-factors and no maps.\n"
|
||||
<< " Everything else in this report is unaffected: the merge does not depend on the model.\n";
|
||||
} else {
|
||||
Key(os, "MODEL_SPACE_GROUP_NUMBER", mv.model_space_group_number);
|
||||
Key(os, "R_WORK", fmt::format("{:.4f}", mv.r_work));
|
||||
Key(os, "R_FREE", fmt::format("{:.4f}", mv.r_free));
|
||||
Key(os, "R_WORK_REFLECTIONS", mv.n_work);
|
||||
Key(os, "R_FREE_REFLECTIONS", mv.n_free);
|
||||
Key(os, "BULK_SOLVENT_K_SOL", fmt::format("{:.3f}", mv.k_sol));
|
||||
Key(os, "BULK_SOLVENT_B_SOL", fmt::format("{:.1f}", mv.b_sol));
|
||||
Key(os, "SCALE_OVERALL", fmt::format("{:.4f}", mv.k_overall));
|
||||
Key(os, "MEAN_ATOM_DENSITY_SIGMA", fmt::format("{:.2f}", mv.mean_atom_density_sigma));
|
||||
// The relabelings that were applied to the written reflections, so a reader can tell
|
||||
// whether the file is in the indexing it was merged in or in the model's.
|
||||
Key(os, "MODEL_HAND_OPERATOR", mv.hand_op.triplet());
|
||||
Key(os, "MODEL_INDEXING_OPERATOR", mv.indexing_op.triplet());
|
||||
if (!mv.maps_prefix.empty())
|
||||
Key(os, "MAPS_PREFIX", mv.maps_prefix);
|
||||
os << "\n R-free here measures the merged intensities against an external structure, which is\n"
|
||||
<< " what CC1/2 and R_meas cannot do - they only measure the data against themselves. The\n"
|
||||
<< " model is not refined: it is scaled to the data with a flat bulk solvent and an overall\n"
|
||||
<< " anisotropic B, so these R-factors are higher than a refined structure's and are a\n"
|
||||
<< " data-quality reading, not a refinement result.\n";
|
||||
if (!(mv.hand_op == gemmi::Op::identity()) || !(mv.indexing_op == gemmi::Op::identity()))
|
||||
os << "\n The written reflections were reindexed into the model's frame - the operators above\n"
|
||||
<< " say how - so the reflection files, the R-factors and the maps all describe one\n"
|
||||
<< " indexing.\n";
|
||||
}
|
||||
}
|
||||
|
||||
// --------------------------------------------------------------- 11. WARNINGS
|
||||
if (result.cancelled)
|
||||
warnings.emplace_back(fmt::format("Processing was cancelled after {} images - this report "
|
||||
"describes an incomplete run", result.images_processed));
|
||||
Section(os, "10. WARNINGS");
|
||||
Section(os, "11. WARNINGS");
|
||||
os << " Everything that needs a person's attention, one line each, marked so a script can find\n"
|
||||
<< " them with a single grep for \"WARNING:\".\n\n";
|
||||
Key(os, "WARNING_COUNT", warnings.size());
|
||||
|
||||
@@ -3295,6 +3295,7 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
|
||||
// A model that was asked for and could not be used has to say so where anyone will see
|
||||
// it. Without this the run ends successfully with no R-free, no maps and nothing in the
|
||||
// report - indistinguishable from a run that was never given --model at all.
|
||||
result.model_validation = validation;
|
||||
if (!validation.failure_reason.empty())
|
||||
result.warnings.push_back("Model validation did not run: " + validation.failure_reason);
|
||||
if (data_sg.has_value()) {
|
||||
|
||||
@@ -18,6 +18,7 @@
|
||||
#include "../image_analysis/spot_finding/SpotFindingSettings.h"
|
||||
#include "../image_analysis/scale_merge/Merge.h" // MergeStatistics
|
||||
#include "../image_analysis/scale_merge/TwinningAnalysis.h" // TwinningAnalysisResult
|
||||
#include "ModelValidation.h" // ModelValidationResult
|
||||
#include "../image_analysis/scale_merge/SearchSpaceGroup.h" // SearchSpaceGroupResult
|
||||
#include "../image_analysis/geom_refinement/PostRefine.h" // PostRefineResult
|
||||
#include "../image_analysis/rotation_indexer/RotationIndexer.h" // RotationIndexerResult
|
||||
@@ -199,6 +200,11 @@ struct ProcessResult {
|
||||
// Twinning analysis of the final merged intensities (l_test_pairs == 0 when not computed).
|
||||
TwinningAnalysisResult twinning;
|
||||
|
||||
// What --model found, when one was given. Set even when the validation could not run - its
|
||||
// failure_reason then says why - so the report can report the failure rather than leave a run
|
||||
// that was asked to validate looking like one that was not. Unset when no model was asked for.
|
||||
std::optional<ModelValidationResult> model_validation;
|
||||
|
||||
// Space group used for the final re-scale/merge and written to the master file: determined by the
|
||||
// search when the user did not fix one, otherwise the fixed group.
|
||||
std::optional<int64_t> space_group_number;
|
||||
|
||||
@@ -1669,6 +1669,7 @@ static int RunRugnux(int argc, char **argv) {
|
||||
// A model that was asked for and could not be used has to reach the report, or the run ends
|
||||
// looking exactly like one that was never given --model.
|
||||
std::string model_validation_failure;
|
||||
std::optional<ModelValidationResult> model_validation;
|
||||
if (!output_prefix.empty() && !model_pdb.empty()) {
|
||||
const auto data_sg = experiment.GetSpaceGroupNumber();
|
||||
// With a reference MTZ the merohedral indexing was already resolved (stills per-image
|
||||
@@ -1678,6 +1679,7 @@ static int RunRugnux(int argc, char **argv) {
|
||||
output_prefix, logger,
|
||||
data_sg ? std::optional<int>(static_cast<int>(*data_sg)) : std::nullopt,
|
||||
/*probe_indexing_ambiguity=*/reference_data.empty());
|
||||
model_validation = validation;
|
||||
if (!validation.failure_reason.empty())
|
||||
model_validation_failure = validation.failure_reason;
|
||||
if (data_sg.has_value()) {
|
||||
@@ -1742,6 +1744,7 @@ static int RunRugnux(int argc, char **argv) {
|
||||
scale_result.error_model_b = error_model_b;
|
||||
scale_result.has_reference = !reference_data.empty();
|
||||
scale_result.twinning = twinning;
|
||||
scale_result.model_validation = model_validation;
|
||||
if (!model_validation_failure.empty())
|
||||
scale_result.warnings.push_back("Model validation did not run: " + model_validation_failure);
|
||||
provenance.wall_time_s = std::chrono::duration<double>(
|
||||
|
||||
@@ -69,7 +69,7 @@ TEST_CASE("ResultReport_Render", "[Diagnostics]") {
|
||||
const auto text = RenderResultReport("prefix", "in.h5", x, result);
|
||||
|
||||
// The stable keys a consumer greps for.
|
||||
CHECK(text.find("\nREPORT_VERSION= 3\n") != std::string::npos);
|
||||
CHECK(text.find("\nREPORT_VERSION= 4\n") != std::string::npos);
|
||||
CHECK(text.find("\nOUTPUT_PREFIX= prefix\n") != std::string::npos);
|
||||
CHECK(text.find("\nINDEXING_RATE= 0.8700\n") != std::string::npos);
|
||||
CHECK(text.find("\nSPACE_GROUP_NUMBER= 96\n") != std::string::npos);
|
||||
@@ -224,6 +224,48 @@ TEST_CASE("ResultReport_ProvenanceKeys", "[Diagnostics]") {
|
||||
CHECK(without.find("GPU_COUNT=") == std::string::npos);
|
||||
}
|
||||
|
||||
// A model that was asked for gets a section either way: the R-factors when it worked, and why it did
|
||||
// not when it did not. A run that never asked for one has no section at all.
|
||||
TEST_CASE("ResultReport_ModelValidationSection", "[Diagnostics]") {
|
||||
DiffractionExperiment x(DetJF(1));
|
||||
x.ImagesPerTrigger(600);
|
||||
|
||||
ProcessResult result;
|
||||
result.images_processed = 600;
|
||||
result.consensus_cell = UnitCell{.a = 79.0f, .b = 79.0f, .c = 38.0f,
|
||||
.alpha = 90.0f, .beta = 90.0f, .gamma = 90.0f};
|
||||
|
||||
CHECK(RenderResultReport("p", "in.h5", x, result).find("MODEL VALIDATION") == std::string::npos);
|
||||
|
||||
ModelValidationResult good;
|
||||
good.ok = true;
|
||||
good.model_path = "model.cif";
|
||||
good.model_space_group_number = 96;
|
||||
good.r_work = 0.1823;
|
||||
good.r_free = 0.2145;
|
||||
good.n_work = 16682;
|
||||
good.n_free = 928;
|
||||
good.mean_atom_density_sigma = 1.42;
|
||||
result.model_validation = good;
|
||||
|
||||
const auto text = RenderResultReport("p", "in.h5", x, result);
|
||||
CHECK(text.find("10. MODEL VALIDATION") != std::string::npos);
|
||||
CHECK(text.find("\nMODEL_FILE= model.cif\n") != std::string::npos);
|
||||
CHECK(text.find("\nR_FREE= 0.2145\n") != std::string::npos);
|
||||
CHECK(text.find("\nR_FREE_REFLECTIONS= 928\n") != std::string::npos);
|
||||
CHECK(text.find("MODEL_VALIDATION= NOT_PERFORMED") == std::string::npos);
|
||||
|
||||
ModelValidationResult failed;
|
||||
failed.model_path = "broken.pdb";
|
||||
failed.failure_reason = "model broken.pdb has no atoms or no unit cell";
|
||||
result.model_validation = failed;
|
||||
|
||||
const auto failed_text = RenderResultReport("p", "in.h5", x, result);
|
||||
CHECK(failed_text.find("\nMODEL_VALIDATION= NOT_PERFORMED\n") != std::string::npos);
|
||||
CHECK(failed_text.find("has no atoms") != std::string::npos);
|
||||
CHECK(failed_text.find("R_FREE=") == std::string::npos);
|
||||
}
|
||||
|
||||
// get_gpu_description collapses repeats, so a four-card machine reads as one line rather than the
|
||||
// same name four times. Build-independent: without CUDA there are no names and it is empty.
|
||||
TEST_CASE("ResultReport_GpuDescription", "[Diagnostics]") {
|
||||
|
||||
Reference in New Issue
Block a user