The rc178 branch continued after #88 was squash-merged; this is its content fromb9ada7914(the tree #88 squashed) to094eee31a, as one change on main. The individual commits stay on the rc178 branch: - still battery: --cold evicts each set's input from the page cache; Alvra pixel masks; manifest fixes - AGIPD: converter to one rigid panel per module, int16 photons, mask folded in; still battery reads the converted sets - DetectorModules: panel look-up by id, plane -> panel, plane bounds and a per-pixel walk; mask size checked - tests: [PanelLayout] equivalence harness, stacked vs assembled panel layout - Dense per-pixel maps through the panel table (panel layout stage 1) - Panel id on spots and the panel table in the start message - Viewer and preview show a panel detector as it lies on the detector plane - Panel layout stage 3: per-panel spot finding (CPU+GPU, CCL) and raster algorithms on the plane raster - Panel layout stage 2: prediction and Bragg integration through the panel table - XTC reader: frames handed up as stored, streams indexed in parallel - XFEL reader: integer frames stay integer, one gain, placement by panel - EXTERNAL_TEST_DATA_STILLS.md: images used vs the entry for every set; drop "Not in the battery" - still battery: EuXFEL sets read the depositors' Cheetah CXI files directly; AGIPD converter removed - float32_to_int16.py: a non-finite pixel becomes -32768, the int16 error marker, not a zero count - CrystFEL geometry: read the pixel masks and bad regions instead of ignoring them - Panel layout stage 5: the XFEL reader hands up the stored frame and its panel table - Panel table persisted in NXmx; modulation grid and pre-scan memo key on the plane - still battery: Alvra sets take their pixel mask from the CrystFEL geometry, not --mask-good - still battery: take the deposited model as mmCIF where the RCSB has no PDB-format file - still battery: dimple gets PDB-format coordinates converted from an mmCIF-only deposition - stills fixed cost: geometry refinement and pre-scan do the same work in less wall time - stills geometry sample and trial spot finding run as many workers as the image pass - Stills integration: background ring reach measured on empty apertures, radial curvature from the whole frame - StillsPartialityRefine: the partiality width is the run's, measured on the crystals, not the prediction window's - MergeOnTheFly: capture term and multiplicative outlier band for the stills merge (both opt-in) - Stills given a cell: the written cell is the mean over the frames that are merged - Stills: the radial curvature correction decided per run on the same empty apertures - StillsPartialityRefine: measure the run width only on shells with signal - Stills pre-scan: report the paired z of the radial-correction comparison - module placement: the five concurrent fits each keep every thread - still battery: score R-free against the deposited data through the same dimple protocol - still battery: the R-free target is the lower of the deposition's dimple and published values - Stills: background clip follows the radial curve; signal disk from the lattice spacing - Stills keep an overloaded observation as a lower bound instead of dropping it - Stills: drop the lattice-spacing signal disk; the correction decision back on the plain clip - battery: R-free against the deposition without refinement only, one protocol for rotation and stills - still battery: EuXFEL sets run on a 2,000-image subset for now - stills geometry refinement: fit the detector distance and the cell together - stills geometry refinement: a distance given with --detector-distance is held in the joint fit - Metal integration engine: back to its panel-s5 kernels; Mac stills keep the old clip, no saturation count - Metal integration test: the CPU comparison runs on a rotation experiment - Stills: kept overloads go back to the drop where they are not concentrated at low resolution - stills distance search: an absolute 30-3000 mm scan, a sample found without d-limits, the fit iterated - battery results: XFEL still battery 20261022-1358 on rc1780cf62452c- 12 sets, cold, R_free unrefined - battery results: still summary gains the resolution of our merge and of the deposition - Pre-scan masks a pixel at the saturation limit on most frames, as one holding the error value - Stills keep an overload as a lower bound only where it has pixels at the limit; run-level gate removed - XFEL reader: a per-frame mask is read the way the frames are, its chunk inflated outside the HDF5 lock - stills distance search: every centre scanned with a wider acceptance, a larger sample where nothing indexes - rugnux usage: --beam-center-check describes the absolute 30-3000 mm stills distance scan, not the old 60-160 % grid - stills: detector tilt refined in the geometry pre-pass, kept where held-out frames agree - CLAUDE.md: the changelog is written once, just before the merge into main - condensed one-liners of what users get - broker: serve the placed model and its maps of the last rugnux live run (/rugnux/model.cif, /rugnux/map.ccp4) - frontend: Mol* viewer of the placed model and its density on the Rugnux live page - writer: _model.cif from rugnux live keeps the input model's metadata, as the served model.cif does - rugnux live maps: one unit cell centred on the model; the viewer opens on all of it - Anomalous difference map: ANOM/PHANOM in _maps.mtz, /rugnux/map.ccp4?type=anom, a magenta mesh in the viewer - Mol* viewer: open with the maps drawn within 10 A of the centre of view - Mol* viewer: aaredb's structure-viewer setup, on a black stage - rugnux: the gain of a stills file that states none is read off its single-photon peak where the background resolves photons - Mol* viewer: in line with AareDB's StructureView at its current head (8ece2c2) - ACKNOWLEDGEMENT: the web frontend's Mol* viewer follows AareDB's setup (G. Gotthard, PSI) - stills tilt decision: the modules stay where the run placed them - still battery: no distance pins and no per-set tuning flags; gain from the data except where a file's baseline hides it - battery results: rc178 6f6f69 - full still battery and full rotation battery with REFMAC - Bragg integration: a CCD rotation sweep's sigmas at the measured gain, intensities in photons - BackgroundGain: the gain of a frame that resolves photons is the spacing of its photon peaks, so a baseline left in the frame no longer hides it - ShadowFinder: measure the background's azimuthal modulation instead of predicting it from the polarization - BackgroundGain: one gain estimate for rotation and stills; the still battery no longer gives EuXFEL 152 its gain - ShadowFinder: measure the modulation robustly under a wide one-sided shadow - ShadowFinderTest: a one-sided shadow is not fitted as part of the background's modulation - ShadowFinder: back to the predicted polarization factor, in the plane the background shows - MXAnalysisAfterFPGA: the Bragg engine's edge mask in the image's layout, so raw / untransformed FPGA runs start again - battery results: still battery on rc178 e54f55 replaces the earlier rc178 still tables - XFELReader: per-frame mask and photon conversion without a library call or a branch per pixel - stills geometry trial: the hypotheses a search asks are indexed side by side - stills tilt decision: the fits start beside the pre-scan and the geometry trial - XFELReader: a deflated frame is inflated and converted a piece at a time, while it is in the cache - XFELReader: a byte-shuffled frame is unshuffled a piece at a time, a whole pixel per step - GeometryRefiner: residuals carry only the blocks a fit moves; beam and distance one block - GeometryRefiner: a spot's residual differentiates its observed and predicted halves apart - docs: EXTERNAL_TEST_DATA lists every open-arm battery set, with corrected values and counts - battery: doc_check.py cross-checks the manifests against the dataset pages - stills module placement: the placement fit gets half the threads, each held-out fit an eighth - still battery: the EuXFEL sets run on all their frames, as the published still runs did - battery results: still battery on rc178 a0ce6c, cold on a quiet machine, replaces the earlier rc178 table - macOS build: explicit size_t in the XFEL reader's row bounds; parentheses in the CPU preprocessor's select Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
202 lines
7.2 KiB
CMake
202 lines
7.2 KiB
CMake
# git header
|
|
# the commit's SHA1, and whether the building workspace was dirty or not
|
|
|
|
FIND_PACKAGE(Git)
|
|
|
|
SET(GITINFO_SOURCE_DIR "${CMAKE_SOURCE_DIR}")
|
|
SET(GITINFO_TEMPLATE "${CMAKE_CURRENT_SOURCE_DIR}/GitInfo.cpp.in")
|
|
SET(GITINFO_OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/GitInfo.cpp")
|
|
SET(JFJOCH_BUILD_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
|
|
# The compiler, and the CUDA toolkit where the build uses one, for the line each program logs at start.
|
|
SET(JFJOCH_BUILD_COMPILER "${CMAKE_CXX_COMPILER_ID} ${CMAKE_CXX_COMPILER_VERSION}")
|
|
IF (JFJOCH_CUDA_AVAILABLE)
|
|
SET(JFJOCH_BUILD_CUDA "${CMAKE_CUDA_COMPILER_VERSION}")
|
|
ELSE()
|
|
SET(JFJOCH_BUILD_CUDA "")
|
|
ENDIF()
|
|
INCLUDE("${CMAKE_CURRENT_SOURCE_DIR}/GitInfoStamp.cmake")
|
|
|
|
set (THREADS_PREFER_PTHREAD_FLAG ON)
|
|
find_package (Threads REQUIRED)
|
|
|
|
MESSAGE(STATUS "Jungfraujoch git SHA1: ${GIT_SHA1}")
|
|
MESSAGE(STATUS "Jungfraujoch git date: ${GIT_DATE}")
|
|
MESSAGE(STATUS "Jungfraujoch version: ${JFJOCH_VERSION}")
|
|
|
|
# Re-stamped on every build, so RUGNUX_GIT in the rugnux report (and every other consumer of
|
|
# jfjoch_git_sha1) names the commit that was actually compiled.
|
|
ADD_CUSTOM_TARGET(JFJochGitInfo
|
|
COMMAND "${CMAKE_COMMAND}"
|
|
"-DGIT_EXECUTABLE=${GIT_EXECUTABLE}"
|
|
"-DGITINFO_SOURCE_DIR=${CMAKE_SOURCE_DIR}"
|
|
"-DGITINFO_TEMPLATE=${CMAKE_CURRENT_SOURCE_DIR}/GitInfo.cpp.in"
|
|
"-DGITINFO_OUTPUT=${CMAKE_CURRENT_BINARY_DIR}/GitInfo.cpp"
|
|
"-DJFJOCH_VERSION=${JFJOCH_VERSION}"
|
|
"-DJFJOCH_BUILD_CXX_FLAGS=${CMAKE_CXX_FLAGS}"
|
|
"-DJFJOCH_BUILD_COMPILER=${JFJOCH_BUILD_COMPILER}"
|
|
"-DJFJOCH_BUILD_CUDA=${JFJOCH_BUILD_CUDA}"
|
|
-P "${CMAKE_CURRENT_SOURCE_DIR}/GitInfoStamp.cmake"
|
|
BYPRODUCTS "${CMAKE_CURRENT_BINARY_DIR}/GitInfo.cpp"
|
|
COMMENT "Stamping GitInfo.cpp from git HEAD"
|
|
VERBATIM)
|
|
|
|
ADD_LIBRARY(JFJochVersion STATIC
|
|
${CMAKE_CURRENT_BINARY_DIR}/GitInfo.cpp GitInfo.h)
|
|
ADD_DEPENDENCIES(JFJochVersion JFJochGitInfo)
|
|
set_target_properties(JFJochVersion PROPERTIES
|
|
POSITION_INDEPENDENT_CODE ON
|
|
)
|
|
|
|
ADD_LIBRARY(JFJochLogger STATIC
|
|
Logger.cpp Logger.h
|
|
)
|
|
|
|
TARGET_LINK_LIBRARIES(JFJochLogger PUBLIC spdlog::spdlog JFJochVersion)
|
|
# NVTX v3 is header-only and architecture-neutral. CUDA::nvtx3 is no help here: it only points at
|
|
# CUDAToolkit_INCLUDE_DIRS, which in a cross build is the target tree (targets/sbsa-linux/include),
|
|
# and NVIDIA's cross-sbsa packages do not put nvtx3 there. So look in the target tree first and then
|
|
# in the toolkit's own include dir next to nvcc; where neither has it, the ranges compile out.
|
|
IF (JFJOCH_CUDA_AVAILABLE)
|
|
FIND_PATH(JFJOCH_NVTX3_INCLUDE_DIR nvtx3/nvToolsExt.h
|
|
HINTS ${CUDAToolkit_INCLUDE_DIRS} "${CUDAToolkit_BIN_DIR}/../include"
|
|
NO_DEFAULT_PATH NO_CMAKE_FIND_ROOT_PATH)
|
|
IF (JFJOCH_NVTX3_INCLUDE_DIR)
|
|
TARGET_INCLUDE_DIRECTORIES(JFJochLogger SYSTEM PRIVATE "${JFJOCH_NVTX3_INCLUDE_DIR}")
|
|
TARGET_LINK_LIBRARIES(JFJochLogger PRIVATE ${CMAKE_DL_LIBS}) # NVTX loads its injection library with dlopen
|
|
TARGET_COMPILE_DEFINITIONS(JFJochLogger PRIVATE JFJOCH_USE_NVTX)
|
|
ELSE()
|
|
MESSAGE(STATUS "NVTX3 headers not found - NVTX ranges compiled out")
|
|
ENDIF()
|
|
ENDIF()
|
|
|
|
ADD_LIBRARY(JFJochZMQ STATIC ZMQWrappers.cpp ZMQWrappers.h)
|
|
|
|
ADD_LIBRARY(JFJochCommon STATIC
|
|
Coord.cpp Coord.h
|
|
DiffractionExperiment.cpp DiffractionExperiment.h
|
|
RawToConvertedGeometry.h
|
|
JFJochException.h
|
|
Definitions.h
|
|
ThreadSafeFIFO.h
|
|
DiffractionSpot.cpp DiffractionSpot.h
|
|
StatusVector.cpp StatusVector.h
|
|
SpotToSave.h
|
|
NetworkAddressConvert.h NetworkAddressConvert.cpp
|
|
DetectorGeometry.h
|
|
DetectorModuleGeometry.cpp DetectorModuleGeometry.h
|
|
DetectorSetup.h DetectorSetup.cpp ZeroCopyReturnValue.h Histogram.h DiffractionGeometry.h
|
|
CUDAWrapper.cpp CUDAWrapper.h
|
|
CudaJitCache.cpp CudaJitCache.h
|
|
HostMemory.cpp HostMemory.h
|
|
ThreadAffinity.cpp ThreadAffinity.h
|
|
ADUHistogram.cpp ADUHistogram.h
|
|
RawToConvertedGeometryCore.h
|
|
Plot.h
|
|
../fpga/pcie_driver/jfjoch_fpga.h
|
|
DatasetSettings.cpp DatasetSettings.h
|
|
ROIMap.cpp ROIMap.h
|
|
ROIElement.cpp ROIElement.h
|
|
ROICircle.cpp ROICircle.h
|
|
ROIBox.cpp ROIBox.h
|
|
ImageBuffer.cpp ImageBuffer.h
|
|
time_utc.h
|
|
PixelMask.cpp PixelMask.h
|
|
InstrumentMetadata.cpp InstrumentMetadata.h
|
|
ImageFormatSettings.cpp
|
|
ImageFormatSettings.h
|
|
DetectorSettings.cpp
|
|
DetectorSettings.h
|
|
AzimuthalIntegrationSettings.cpp
|
|
AzimuthalIntegrationSettings.h
|
|
AzimuthalIntegrationProfile.h
|
|
AzimuthalIntegrationProfile.cpp
|
|
AzimuthalIntegrationMapping.h
|
|
AzimuthalIntegrationMapping.cpp
|
|
CheckPath.h
|
|
AutoIncrVector.h
|
|
ModuleSummation.cpp
|
|
ModuleSummation.h
|
|
ROIDefinition.cpp
|
|
ROIDefinition.h
|
|
ZeroCopyReturnValue.cpp
|
|
ColorScale.cpp ColorScale.h
|
|
FileWriterSettings.cpp
|
|
FileWriterSettings.h
|
|
MovingAverage.cpp
|
|
MovingAverage.h
|
|
DiffractionGeometry.cpp
|
|
ROIAzimuthal.cpp
|
|
ROIAzimuthal.h
|
|
print_license.h
|
|
DetectorGeometryModular.cpp
|
|
DetectorGeometryModular.h
|
|
DetectorModules.cpp
|
|
DetectorModules.h
|
|
PanelRows.h
|
|
PanelBoxes.cpp
|
|
PanelBoxes.h
|
|
AssembleToPlane.cpp
|
|
AssembleToPlane.h
|
|
ModuleShift.h
|
|
DetectorGeometryFixed.cpp
|
|
DetectorGeometryFixed.h
|
|
GridScanSettings.cpp
|
|
GridScanSettings.h
|
|
JFJochMessages.h
|
|
GoniometerAxis.cpp
|
|
GoniometerAxis.h
|
|
DetectorTransformation.cpp
|
|
DetectorTransformation.h
|
|
DetectorOrientation.cpp
|
|
DetectorOrientation.h
|
|
CompressedImage.cpp
|
|
CompressedImage.h
|
|
Reflection.h
|
|
MultiLinePlot.cpp
|
|
MultiLinePlot.h
|
|
IndexingSettings.cpp
|
|
IndexingSettings.h
|
|
CrystalLattice.cpp
|
|
CrystalLattice.h
|
|
ScanResult.cpp
|
|
ScanResult.h
|
|
ScanResultGenerator.cpp
|
|
ScanResultGenerator.h
|
|
BraggIntegrationSettings.cpp
|
|
BraggIntegrationSettings.h
|
|
SpotToSave.cpp
|
|
XrayFluorescenceSpectrum.cpp
|
|
XrayFluorescenceSpectrum.h
|
|
ResolutionShells.cpp
|
|
ResolutionShells.h
|
|
DarkMaskSettings.cpp
|
|
DarkMaskSettings.h
|
|
TopPixels.cpp
|
|
TopPixels.h
|
|
hkl_key.h
|
|
JfjochTCP.h
|
|
BrokerStatus.h
|
|
ScalingSettings.cpp
|
|
ScalingSettings.h
|
|
UnitCell.cpp
|
|
UnitCell.h
|
|
MtzTable.h
|
|
CorrelationCoefficient.cpp
|
|
CorrelationCoefficient.h
|
|
JFJochReceiverPlots.cpp
|
|
JFJochReceiverPlots.h
|
|
)
|
|
|
|
# librt is Linux/glibc-only (older glibc keeps clock_gettime/timer_*/sem_* there, and some
|
|
# distros are picky about it for pthreads); it does not exist on macOS/Windows.
|
|
TARGET_LINK_LIBRARIES(JFJochCommon JFJochLogger Compression JFCalibration gemmi Threads::Threads
|
|
$<$<PLATFORM_ID:Linux>:rt>)
|
|
|
|
TARGET_LINK_LIBRARIES(JFJochZMQ "$<BUILD_INTERFACE:libzmq-static>")
|
|
|
|
IF (JFJOCH_CUDA_AVAILABLE)
|
|
TARGET_SOURCES(JFJochCommon PRIVATE CUDAWrapper.cu )
|
|
TARGET_LINK_LIBRARIES(JFJochCommon CUDA::cudart_static
|
|
${CMAKE_DL_LIBS} $<$<PLATFORM_ID:Linux>:rt>)
|
|
ENDIF()
|