Files
Jungfraujoch/docs/RUGNUX_CALIBRATION.md
T
leonarski_fandClaude Opus 5 ef5da29319 calibration: a fit that hands back the file's own tilt is not a calibration
A powder calibration is run because the file's geometry is in doubt, so a fit that
quietly returns part of that file has answered nothing - and it is indistinguishable
from one that worked, down to the residual and the sigmas arranged around it.

On one of four LaB6 exposures of one detector the tilt came out at 2.93x its own
sigma, a hundredth under the significance gate, so it was declined and pinned - at the
master's hardcoded rot1 -0.08, rot2 -0.22 deg. That is eight times the tilt just
refused, on no evidence, and worth 10 px of PONI at 190 mm. rugnux printed it to four
decimal places, wrote the .poni, and exited 0.

Judge the result on provenance instead of on any residual: a geometry is a measurement
only if every parameter in it came from this data. Two ways out of the fits do not
qualify - a covariance that never conditioned, so the fit cannot say what it
determined, and a declined tilt pinned at a non-zero value from the file. A declined
tilt over a file stating no tilt still qualifies, because reporting no tilt is then
exactly what was measured; so does --no-refine-tilt, because a hold that was asked for
is a stated choice and not a silent substitution.

No single number separates the four. rms is 2.465 px against 1.44-1.64; the
significance of all four lies between 2.93 and 4.47, so the gate is nearly a coin flip
at these distances and moving it would only recalibrate on one population; and the
failed fit has the TIGHTEST parameter sigmas of the set, because pinning the tilt
removes the tilt/centre correlation that inflates a good fit's. The spot cross-check
reads 13.5 px against 0.98-2.66, but 10.4 px of that is the pinned tilt moving the
PONI - the same defect one step downstream, not independent evidence.

On a failure rugnux says so, writes no .poni - a PONI file states where the detector is
and has no field in which to say it does not know - writes the JSON with converged
false and the reason beside it, and exits non-zero. The re-binning pass now prefers a
converged refit over a non-converged one whatever its residual, so a tilt an earlier
pass measured is not what a later one gets pinned at.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 09:37:46 +02:00

5.5 KiB
Raw Blame History

Detector calibration from powder rings (rugnux --mode calibration)

The calibration mode determines the detector geometry — PONI x/y, the two tilts rot1/rot2 and the distance — from the powder rings of a calibrant, and writes it as a pyFAI <prefix>.poni file and a machine-readable <prefix>.json, alongside a printed report of how far each parameter moved from the header. Bragg data pin the beam centre worst (it is gauge-coupled to the crystal orientation); a powder ring has no orientation to be coupled to, so this is the measurement that fixes it.

rugnux --mode calibration --calibrant lab6 -o det LaB6_master.h5

What it writes

<prefix>.poni is for pyFAI and the tools that read its format. <prefix>.json is for everything else, and has two members:

  • dataset_settings — the geometry under the property names dataset_settings gives them in jfjoch_broker's OpenAPI schema, and nothing else. It is a valid dataset_settings body as it stands, so it can be POSTed or merged into one without translating a field:

    curl -X POST -H 'Content-Type: application/json' \
         -d "$(jq -c .dataset_settings det.json)" http://broker:5232/start
    

    beam_x_pxl/beam_y_pxl is the PONI, as everywhere in this system — on a tilted detector it is not where the direct beam lands. The three poni_rot*_rad are written whenever any of them is non-zero, and left out when all are zero: a body that omits them states a flat detector, so they travel together or not at all.

  • calibration — what the run knows about that geometry: the residual, the fit's own sigmas and the correlation between the tilt and the beam centre, whether the tilt cleared its significance test or was declined and pinned, where the direct beam lands, and where the spots independently put the beam. A calibration that has gone wrong looks exactly like one that has not until those are read.

--calibrant takes lab6, agbh (silver behenate), ceo2, si or ice, case-insensitively. ice calibrates a real experiment against its own ice rings — no calibrant exposure needed — and is the reason a calibrant is a list of ring positions rather than a unit cell: hexagonal ice is P63/mmc, so rings enumerated from its cell would include systematically absent ones.

--calibration picks how the rings are measured, and both use the whole dataset-s/-e/-t select which images:

  • rings (default) sums the (q × azimuth) azimuthal profile over every processed image into one map and fits the ring arcs in it. A powder ring is an arc, not a set of spots, and the summed profile measures it at every azimuth with all the run's counts behind it. It needs the profile to be binned in azimuth, so this mode defaults --azim-phi-bins to 32.
  • spots pools the found spots of every processed image and fits those. It determines the centre from scratch (a Hough circle vote, which quantises it to a whole pixel) and then refines.

Both routes read the ring position out of a binned profile or a spot centroid, so the radial sampling matters: at a long detector distance the default 0.01 Å⁻¹ q bin is several pixels wide and quantises the rings route accordingly — pass a finer --azim-q-spacing there (the total q × azimuth bin count must stay under 65534).

The report prints the fitted geometry, the scatter of the ring points about the fitted rings and the standard error that implies on the centre. That error is formal: it measures the scatter of the points, not whether the rings themselves are trustworthy, so it stays small when a fit goes wrong for a structural reason — one visible ring, or ice that is textured rather than smooth.

A calibration is run because the header is in doubt, so a fit that quietly hands part of that header back has answered nothing — and it looks exactly like a fit that worked, down to the residual and the sigmas around it. calibration.converged in the JSON says which of the two a file is. It is false when the tilt was declined and pinned at a non-zero header value — the rings said they could not tell a tilt from a shift of the beam centre, and the angle written in its place has no more support than the one refused — or when the fit's covariance never conditioned, so it cannot say what it determined. In either case rugnux prints the reason, writes the .json with converged false and not_converged_reason beside it, exits non-zero, and writes no .poni: a PONI file states where the detector is and has no field in which to say that it does not know. A declined tilt over a header that states no tilt is not this — reporting no tilt is then exactly what the fit measured.

--no-refine-tilt holds rot1/rot2 where the header put them and fits only the centre and the distance. The tilt is real and worth measuring, but a program that has nowhere to put one — XDS takes a detector normal to the beam — is better given a geometry that was measured with the tilt pinned than one that was measured tilted and then flattened, because in the tilted fit the centre and the distance have already absorbed the tilt.

Both the PONI (the point of normal incidence, which is what a .poni file stores) and the direct beam (where the beam lands, which is what most other programs call the beam centre) are printed. They differ by distance × tan(rot) once the detector is tilted, which on a 0.3° tilt at 300 mm is several pixels — enough to look like a disagreement with another program when there is none.