leonarski_fandClaude Opus 5.5
b0b217a795
With --model, model validation and the P1 cross-check merge ran one after the other at the end of the run. Neither reads anything the other writes, so the first ValidateAgainstModel now runs on its own thread while the main thread makes the P1 merge (rsm->Run in P1); the merge's result is kept and reported at the cross-check, where it used to be made, through a new `already_run` argument of scale_and_merge. Why it is exact: the validation reads the adopted merge, the cell, the group and the wavelength, all captured before the thread starts, and nothing else. The scaling engine merges its own copy of the observations under the group set on the experiment (P1 for the merge, restored after, as at the cross-check), holds its own copy of the cell, reads the outcomes' lattices and mosaicities only at Ingest, and writes back per-image G, CC and mosaicity - which nothing between the validation and the cross-check reads (the unmerged MTZ build reads the mosaicity but replaces it after the merge, as before). Every relabelling the validation decides reaches the P1 merge afterwards through merge_to_written, exactly as when it was made later. The result fields scale_and_merge sets are still set at the cross-check, so the report and the finalist ledger read the same values. Without --model, or without a P1 cross-check, nothing changes. The validation logs into a Logger::Buffered() and the lines are replayed as one block when it is done, so the two do not interleave; the P1 merge's log lines now come before the validation's. Checked against rc173 (rugnux_r4int_77710e1ce, output-identical to it) on 8tyy, 9hnc, 8sa8, 6oel, 7ph1 with --model and on myob: p.hkl, p.mtz, p.cif, p_P1.mtz, p_unmerged.mtz, the maps and the placed model are byte-identical, and so is p_report.txt apart from date/command/timing lines. A last-digit flip of one image's bkg in p_plot.txt (8sa8, one of two runs) is image-loop nondeterminism the baseline shows too (8tyy, base vs base). Tail wall (sweep-quality line to unmerged MTZ written), paired back-to-back runs on a loaded box: 8tyy 110.3 -> 87.3 s, 6oel 77.8 -> 63.3 s, 8sa8 171.4 -> 154.6 s; unpaired 9hnc 368.8 -> 337.0 s, 7ph1 21.1 -> 15.7 s. The saving is the P1 merge's time; the validation still sets the floor. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D1G8gJVAy6gp1K5Dz3NE5C
72 lines
2.4 KiB
C++
72 lines
2.4 KiB
C++
// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#include "spdlog/sinks/daily_file_sink.h"
|
|
#include "spdlog/sinks/stdout_color_sinks.h"
|
|
#include "spdlog/sinks/base_sink.h"
|
|
#include "spdlog/fmt/bin_to_hex.h"
|
|
|
|
#include "Logger.h"
|
|
#include "GitInfo.h"
|
|
|
|
Logger::Logger(const std::string &service_name, const std::string &file_name) {
|
|
std::string variant = "";
|
|
#ifdef JFJOCH_USE_CUDA
|
|
variant += "C";
|
|
#endif
|
|
#ifdef JFJOCH_USE_FFTW
|
|
variant += "F";
|
|
#endif
|
|
std::vector<spdlog::sink_ptr> sinks;
|
|
|
|
sinks.push_back(std::make_shared<spdlog::sinks::stdout_color_sink_mt>());
|
|
if (!file_name.empty())
|
|
sinks.push_back(std::make_shared<spdlog::sinks::daily_file_sink_mt>(file_name, 23, 59));
|
|
|
|
spdlog_logger = std::make_shared<spdlog::logger>(service_name, std::begin(sinks), std::end(sinks));
|
|
if (jfjoch_git_sha1().empty())
|
|
spdlog_logger->info("Version {} {}", jfjoch_version(), variant);
|
|
else
|
|
spdlog_logger->info("Version {} (git {} {}) {}",
|
|
jfjoch_version(),
|
|
jfjoch_git_sha1().substr(0, 6),
|
|
jfjoch_git_date(),
|
|
variant);
|
|
}
|
|
|
|
class LogBufferSink : public spdlog::sinks::base_sink<std::mutex> {
|
|
public:
|
|
std::vector<std::pair<spdlog::level::level_enum, std::string>> lines;
|
|
protected:
|
|
void sink_it_(const spdlog::details::log_msg &msg) override {
|
|
lines.emplace_back(msg.level, std::string(msg.payload.data(), msg.payload.size()));
|
|
}
|
|
void flush_() override {}
|
|
};
|
|
|
|
Logger::Logger(std::shared_ptr<LogBufferSink> sink) : buffer(std::move(sink)) {
|
|
spdlog_logger = std::make_shared<spdlog::logger>("buffer", buffer);
|
|
spdlog_logger->set_level(spdlog::level::trace); // the logger it is replayed into filters
|
|
}
|
|
|
|
Logger Logger::Buffered() {
|
|
return Logger(std::make_shared<LogBufferSink>());
|
|
}
|
|
|
|
// Called once the work that logged is finished, so nothing is still writing to the buffer.
|
|
void Logger::ReplayInto(Logger &to) const {
|
|
for (const auto &[level, line] : buffer->lines)
|
|
to.spdlog_logger->log(level, line);
|
|
}
|
|
|
|
void Logger::ErrorException(const std::exception &e) {
|
|
spdlog_logger->error(e.what());
|
|
}
|
|
|
|
Logger &Logger::Verbose(bool input) {
|
|
if (input)
|
|
spdlog_logger->set_level(spdlog::level::debug);
|
|
else
|
|
spdlog_logger->set_level(spdlog::level::info);
|
|
return *this;
|
|
} |