Files
leonarski_f 4809c1622c
Build Packages / build:portable:linux-x86_64:nocuda (push) Successful in 16m43s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 28m46s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 32m5s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 39m44s
Build Packages / build:portable:macos-arm64:nocuda (push) Successful in 5m7s
Build Packages / Create release (push) Successful in 8s
Build Packages / Build documentation (push) Successful in 49s
Build Packages / build:portable:linux-x86_64:cuda (push) Successful in 22m30s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 31m54s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 11m29s
Build Packages / build:portable:windows-x86_64:nocuda (push) Successful in 23m51s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 26m7s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 29m38s
Build Packages / build:portable:windows-x86_64:cuda (push) Successful in 28m6s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 29m44s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 33m22s
Build Packages / Generate python client (push) Successful in 47s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 34m6s
Build Packages / Unit tests (push) Successful in 1h2m47s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 35m59s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 34m8s
Build Packages / Validation Battery (push) Successful in 38m31s
v1.0.0-rc.187 (#98)
* Rugnux: Refinement several times faster, one protocol on the CPU and the GPU, parts of it on the Apple GPU; its random draws are the same on every platform.
* Rugnux: Refinement follows common practice: the free set decides only the restraint weights and the displacement model (isotropic, TLS or anisotropic, by Hamilton's test), everything else is decided on the working set; lower R-free for placed and predicted models.
* Rugnux: Restraint dictionaries via `--restraints` (or among the inputs) and more built-in ligands; a model whose space group is a subgroup of the data's is refined in its own group; reflections plus a model select `--mode refine` on their own.
* Rugnux: `-C` fixes the crystal system on the axes as typed; `-S` alone fixes the space group only, the cell is still found and reduced; a space group or cell stored in the input file is ignored, with a line in the log.
* Rugnux: More reliable multi-sweep merging; `--mode scale` merges several stills runs as one; `_plot.txt` and the report name each input; a second crystal form is reported for stills given a cell.
* jfjoch_broker (experimental): Live Rugnux on FPGA detectors (JUNGFRAU) takes the same path as a DECTRIS stream, rotation and serial alike, model step included, with analysis threads per GPU; frames waiting for the analysis can be viewed; no host stage drops a frame - a source faster than the analysis is held at the card.
* jfjoch_broker: Summation of more than 256 frames per image is split between the card and the host (`summation_mode` is then `hybrid`), and live Rugnux runs on such collections; pedestal runs take a receiver of their own.
* jfjoch_broker (experimental): `stills_downsample` analyses 1 of every N serial images (chosen by a hash; the first 2000 all; the others written without analysis; reset to 1 by `/initialize`); `max_spot_count` left out means 250 for per-image indexing and up to 1000 for live stills; the live status gives the merge before the model work, the fitted resolution and the anomalous peak height of each unexplained blob.
* jfjoch_broker: Without a `pcie` entry the broker runs on a mock of the card instead of the HLS simulation; its spot, ROI and azimuthal results are zero then.
* Web frontend: Redesigned Rugnux live page; Vite 8, TypeScript 6, Mol* 5.13 and dependency security updates.
* Tools: `jfjoch_fpga_stills_test` measures live stills on the cards with a stored dataset; `jfjoch_fpga_test` loses the HLS-simulation option.
* Other: FFTW no longer fetched or linked (the CPU FFT indexer keeps the name `FFTW`); PCIe driver now GPL-2.0-or-later and third-party notices complete; several crash fixes.

Reviewed-on: #98
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-11-19 20:49:28 +01:00

193 lines
9.3 KiB
Python

# SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
# SPDX-License-Identifier: GPL-3.0-only
"""One document, written twice: Markdown and a self-contained HTML page.
The report is built as a list of blocks (heading, paragraph, bullet list, table) and each writer
turns the same list into its format, so the two can never say different things.
"""
import html
CSS = """
:root { --bg:#fff; --fg:#1d1d1f; --mute:#6e6e73; --line:#d9d9de; --head:#f3f3f5;
--pass:#1a7f37; --fail:#c62828; --warn:#9a6700; --passbg:#e8f5ec; --failbg:#fdecea;
--series1:#2a78d6; --good:#0ca30c; --critical:#d03b3b; --warning:#fab219;
--neutral:#898781; --grid:#e1e0d9; --axis:#c3c2b7; }
@media (prefers-color-scheme: dark) {
:root { --bg:#161618; --fg:#e8e8ea; --mute:#9a9aa0; --line:#3a3a3e; --head:#232326;
--pass:#5cc97a; --fail:#ff7b72; --warn:#e3b341; --passbg:#16301f; --failbg:#3a1a1a;
--series1:#3987e5; --grid:#2c2c2a; --axis:#383835; }
}
figure { margin:12px 0 20px; max-width:900px; } figcaption { color:var(--mute); font-size:13px; }
svg.plot { width:100%; height:auto; display:block; }
svg.plot text { fill:var(--mute); font:11px system-ui, sans-serif; }
svg.plot .grid { stroke:var(--grid); stroke-width:1; } svg.plot .ref { stroke:var(--axis); stroke-width:2; }
svg.plot circle:hover, svg.plot rect:hover { stroke:var(--fg); stroke-width:2; }
body { background:var(--bg); color:var(--fg); margin:0 auto; max-width:1400px; padding:16px;
font:14px/1.45 system-ui, -apple-system, "Segoe UI", sans-serif; }
h1 { font-size:22px; margin:8px 0 4px; } h2 { font-size:18px; margin:28px 0 8px; }
h3 { font-size:15px; margin:20px 0 6px; }
p, li { max-width:900px; } .mute { color:var(--mute); }
.wrap { overflow-x:auto; margin:8px 0 16px; }
table { border-collapse:collapse; font-variant-numeric:tabular-nums; }
th, td { border-bottom:1px solid var(--line); padding:4px 8px; text-align:left; white-space:nowrap; }
th { background:var(--head); position:sticky; top:0; }
td.num, th.num { text-align:right; }
td.pass { color:var(--pass); font-weight:600; } td.fail { color:var(--fail); font-weight:600; }
td.warn { color:var(--warn); font-weight:600; }
tr.fail td { background:var(--failbg); }
code { font:12px ui-monospace, Menlo, monospace; }
"""
class Doc:
def __init__(self, title):
self.title = title
self.blocks = []
def h(self, level, text):
self.blocks.append(("h", level, text))
def p(self, text):
self.blocks.append(("p", text))
def ul(self, items):
self.blocks.append(("ul", list(items)))
def svg(self, markup, caption):
"""An inline SVG plot (HTML only; the Markdown names it and points at the HTML)."""
self.blocks.append(("svg", markup, caption))
def table(self, head, rows, num=(), row_class=None, cell_class=None):
"""head: column titles; rows: lists of str; num: indices of right-aligned columns;
row_class(row) and cell_class(i, value) give optional CSS classes (HTML only)."""
self.blocks.append(("table", head, rows, set(num), row_class, cell_class))
def markdown(self):
out = [f"# {self.title}", ""]
for b in self.blocks:
if b[0] == "h":
out += ["#" * b[1] + " " + b[2], ""]
elif b[0] == "p":
out += [b[1], ""]
elif b[0] == "ul":
out += ["- " + x for x in b[1]] + [""]
elif b[0] == "svg":
out += [f"_Plot in report.html: {b[2]}_", ""]
else:
_, head, rows, num, _, _ = b
if not rows:
out += ["(none)", ""]
continue
out.append("| " + " | ".join(head) + " |")
out.append("|" + "|".join("--:" if i in num else ":--" for i in range(len(head))) + "|")
for r in rows:
out.append("| " + " | ".join(str(x).replace("|", "/") for x in r) + " |")
out.append("")
return "\n".join(out)
def html(self):
e = html.escape
out = ["<!DOCTYPE html>", '<html lang="en"><head><meta charset="utf-8">',
'<meta name="viewport" content="width=device-width, initial-scale=1">',
f"<title>{e(self.title)}</title><style>{CSS}</style></head><body>",
f"<h1>{e(self.title)}</h1>"]
for b in self.blocks:
if b[0] == "h":
out.append(f"<h{b[1]}>{e(b[2])}</h{b[1]}>")
elif b[0] == "p":
out.append(f"<p>{e(b[1])}</p>")
elif b[0] == "ul":
out.append("<ul>" + "".join(f"<li>{e(x)}</li>" for x in b[1]) + "</ul>")
elif b[0] == "svg":
out.append(f"<figure>{b[1]}<figcaption>{e(b[2])}</figcaption></figure>")
else:
_, head, rows, num, row_class, cell_class = b
if not rows:
out.append('<p class="mute">(none)</p>')
continue
t = ['<div class="wrap"><table><thead><tr>']
t += [f'<th class="num">{e(h)}</th>' if i in num else f"<th>{e(h)}</th>"
for i, h in enumerate(head)]
t.append("</tr></thead><tbody>")
for r in rows:
rc = row_class(r) if row_class else ""
t.append(f'<tr class="{rc}">' if rc else "<tr>")
for i, x in enumerate(r):
cls = ["num"] if i in num else []
if cell_class and cell_class(i, x):
cls.append(cell_class(i, x))
c = f' class="{" ".join(cls)}"' if cls else ""
t.append(f"<td{c}>{e(str(x))}</td>")
t.append("</tr>")
t.append("</tbody></table></div>")
out.append("".join(t))
out.append("</body></html>")
return "\n".join(out)
# ------------------------------------------------------------------ plots (inline SVG, no libraries)
STATUS = {"pass": "var(--good)", "accepted_alt": "var(--series1)", "fail": "var(--critical)",
"crash": "var(--fg)", "unscored": "var(--neutral)", "not_run": "var(--warning)"}
def verdict_bars(rows):
"""One horizontal bar per arm, split pass / accepted alt / fail / crash / unscored / not run.
rows: [(arm, {status: count})], where a status is a verdict, except that a pass carried by an
accepted alternative reference is counted on its own."""
e = html.escape
w, left, bar, gap = 900, 90, 22, 14
h = 30 + len(rows) * (bar + gap)
total = max(sum(c.values()) for _, c in rows) or 1
out = [f'<svg class="plot" viewBox="0 0 {w} {h}" role="img" aria-label="verdicts per arm">']
x = left
for v, col in STATUS.items(): # legend, above the bars
out.append(f'<rect x="{x}" y="4" width="10" height="10" rx="2" fill="{col}"/>'
f'<text x="{x + 14}" y="13">{v.replace("_", " ")}</text>')
x += 100
for i, (arm, cnt) in enumerate(rows):
y = 26 + i * (bar + gap)
out.append(f'<text x="0" y="{y + bar / 2 + 4}">{e(arm)}</text>')
x = left
for v, col in STATUS.items():
n = cnt.get(v, 0)
if not n:
continue
bw = (w - left - 10) * n / total
out.append(f'<rect x="{x:.1f}" y="{y}" width="{max(bw - 2, 1):.1f}" height="{bar}" rx="4" '
f'fill="{col}"><title>{e(arm)}: {n} {v.replace("_", " ")}</title></rect>')
if bw > 28:
ink = "var(--bg)" if v == "crash" else "#fff" # the crash segment is drawn in --fg
out.append(f'<text x="{x + 6:.1f}" y="{y + bar / 2 + 4}" style="fill:{ink}">{n}</text>')
x += bw
out.append("</svg>")
return "".join(out)
def ratio_dots(points, what):
"""Sets sorted by a ratio (ours / reference), on a log axis clipped to 0.5..2, with the line
at 1. points: [(set, ratio, verdict, tooltip)]. Failed sets are drawn in the fail colour."""
e = html.escape
import math
w, h, left, top, bottom = 900, 240, 44, 22, 18
lo, hi = math.log(0.5), math.log(2.0)
y = lambda r: top + (hi - math.log(min(max(r, 0.5), 2.0))) / (hi - lo) * (h - top - bottom)
pts = sorted(points, key=lambda p: p[1])
step = (w - left - 10) / max(len(pts), 1)
out = [f'<svg class="plot" viewBox="0 0 {w} {h}" role="img" aria-label="{e(what)}">']
for t in (0.5, 0.75, 1.0, 1.5, 2.0):
out.append(f'<line class="{"ref" if t == 1.0 else "grid"}" x1="{left}" x2="{w}" '
f'y1="{y(t):.1f}" y2="{y(t):.1f}"/><text x="0" y="{y(t) + 4:.1f}">{t:g}</text>')
out.append(f'<circle cx="{left + 6}" cy="10" r="4" fill="var(--series1)"/>'
f'<text x="{left + 14}" y="14">pass or unscored</text>'
f'<circle cx="{left + 136}" cy="10" r="4" fill="var(--critical)"/>'
f'<text x="{left + 144}" y="14">fail</text>'
f'<text x="{left + 190}" y="14">(clipped to 0.5-2; hover a point for the set)</text>')
for i, (name, r, verdict, tip) in enumerate(pts):
col = "var(--critical)" if verdict in ("fail", "crash") else "var(--series1)"
out.append(f'<circle cx="{left + step * (i + 0.5):.1f}" cy="{y(r):.1f}" r="4" fill="{col}">'
f'<title>{e(name)}: {r:.3f} - {e(tip)}</title></circle>')
out.append("</svg>")
return "".join(out)