rugnux: _plot.txt gets dataset and image columns for runs of several inputs

The per-image table numbered its rows by the run's processed-image ordinal
only, which is unusable for a run made of several files. It now carries
three numbering columns: ordinal (the run's row, as before), dataset (the
input, from 1 in the order given) and image (the image within that input).
The header describes each column before the legend line.

- Several stills masters read as one: rows are renumbered per master.
- Joint merge of sweeps: the table covered only as many rows as the
  reference sweep's file held, so the other sweeps' frames were missing;
  it now has one row per frame of every merged sweep, with each sweep's own
  angle and per-image analysis (bkg, resolution, spots) from its own run
  and the scaling columns from the joint merge.
- The angle is a per-row value now, so each sweep reads on its own goniometer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UqXYdJDrM65gxsCn2PrbfK
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-11-16 09:29:41 +01:00
1 parent 8bceabbde2
commit c4e4dc88c9
8 files changed
+207 -69

No files matched your search

+1 -1
View File
@@ -599,7 +599,7 @@ The vocabulary is **closed and stable**: a code is never renamed, and never reus
meaning. New codes are only ever added, and adding one moves `REPORT_VERSION` at the next release.
The columns are: `FIRST_IMAGE`/`LAST_IMAGE` — inclusive, in processed-image ordinals (the numbering
of `<prefix>_plot.txt` and of every other per-image array `rugnux` writes; with `-s`/`--stride` the
of the `ordinal` column of `<prefix>_plot.txt` and of every other per-image array `rugnux` writes; with `-s`/`--stride` the
source image is `start + ordinal * stride`); `ROTATION` — the width of the range in degrees;
`SEVERITY` — the fraction of the run's typical diffracting power missing over the range, 0 (as good
as the run) to 1 (nothing at all); `SCALE` and `CC` — the range's mean per-image scale and
+10 -5
View File
@@ -363,13 +363,15 @@ Absorption is corrected only by the empirical surface a rotation run fits and ke
JSON also by the broker's `/start`, merged into its dataset settings; see
[Module geometry](RUGNUX_FORMATS.md#module-geometry)).
- `<prefix>_plot.txt` — **the per-image table**, one row per processed image, whitespace-separated,
with a single `#` legend line over its own columns so gnuplot plots it as it stands
(`plot '<prefix>_plot.txt' using 1:4`). The columns are:
under a `#` header - one line per column saying what it is, then a legend line over the columns -
so gnuplot plots it as it stands (`plot '<prefix>_plot.txt' using 1:5`). The columns are:
| Column | Meaning |
|--------|---------|
| `image` | Processed-image ordinal, the numbering every per-image array `rugnux` writes uses; with `-s`/`--stride` the source image is `start + ordinal * stride`. |
| `angle_deg` | Spindle angle at mid-exposure. |
| `ordinal` | Processed-image ordinal: the row of the run, the numbering every per-image array `rugnux` writes uses (and the report's `FIRST_IMAGE`/`LAST_IMAGE`). |
| `dataset` | The input the image is from, counted from 1 in the order the inputs were given: the sweep of a joint merge of several (`--sweeps joint`), the file of several read as one stills run. 1 on a run of one input. |
| `image` | The image's number in that input, from 0, as `-s`/`-e` count it; with `-s`/`--stride` it is `start + ordinal * stride`. |
| `angle_deg` | Spindle angle at mid-exposure, on that input's own goniometer. |
| `bkg` | Background estimate from the azimuthal profile (the `bkgEstimate` of the HDF5 file). |
| `resolution_A` | How far the image's spots reach, from spot finding alone. |
| `spots` | Spots found. |
@@ -382,7 +384,10 @@ Absorption is corrected only by the empirical surface a rotation run fits and ke
A quantity nothing measured for an image is written `nan`, which gnuplot and numpy both skip; the
columns never shift and a row is never left out. A stills run writes the table too, with `nan` in
the columns a sweep would fill; `--no-merge` writes it with the scaling columns `nan`.
the columns a sweep would fill; `--no-merge` writes it with the scaling columns `nan`. A joint merge of
sweeps writes one row per frame of the sweeps it merged, sweep after sweep in input order:
each row's per-image analysis is its own sweep's, the scaling columns the joint
merge's; a sweep left out of the merge has no rows.
**Reading `sigma_M_deg`.** It is fitted per image — a maximum-likelihood Gaussian rocking width
over that image's own indexed spots — but it is then **smoothed in frame order** over the same
+35 -12
View File
@@ -109,16 +109,43 @@ std::string RenderDetectorDiagnostic(const DiffractionExperiment &experiment,
return WriteJPEGToMem(rgb_image, 85);
}
void FillJointPlotRows(PerImagePlot &joint, size_t first_row, int dataset, const PerImagePlot &own) {
for (auto *v : {&joint.angle_deg, &joint.bkg_estimate, &joint.resolution_A, &joint.spot_count})
v->resize(joint.image_count, NAN);
for (size_t f = 0; f < own.image_count; f++) {
const size_t row = first_row + f;
joint.dataset[row] = dataset;
joint.image[row] = own.image[f];
joint.angle_deg[row] = Element(own.angle_deg, f);
joint.bkg_estimate[row] = Element(own.bkg_estimate, f);
joint.resolution_A[row] = Element(own.resolution_A, f);
joint.spot_count[row] = Element(own.spot_count, f);
}
}
std::string PerImagePlotText(const PerImagePlot &plot) {
const std::optional<GoniometerAxis> &goniometer = plot.goniometer;
std::string file;
// One comment line, its names over their own columns, and nothing else a plotting program has
// to be told to skip. The '#' takes the first character of the image-number field.
file += "#" + fmt::format("{:>5} {:>10} {:>12} {:>12} {:>8} {:>12} {:>13} {:>12} {:>8}"
// What each column is, then one comment line with the names over their own columns - nothing a
// plotting program has to be told to skip, as gnuplot and numpy pass over '#' lines. The '#' takes
// the first character of the ordinal field.
file += "# ordinal: row of the run, the numbering of every per-image array rugnux writes\n"
"# dataset: input file or sweep, from 1 in the order given\n"
"# image: image of that input, from 0 (with -s/--stride: start + ordinal * stride)\n"
"# angle_deg: spindle angle at mid-exposure\n"
"# bkg: background estimate from the azimuthal profile\n"
"# resolution_A: how far the image's spots reach\n"
"# spots: spots found\n"
"# scale_G: per-image scale factor fitted by scaling\n"
"# sigma_M_deg: mosaicity, the rocking-curve standard deviation\n"
"# cc_to_merge: correlation of the image's observations with the merged data\n"
"# cc_n: observations that correlation was taken over\n"
"# merged: 1 where the image's observations are in the merged data, 0 where not\n"
"# disposition: 1 merged, 0.5 downgraded, 0 rejected\n";
file += "#" + fmt::format("{:>7} {:>7} {:>7} {:>10} {:>12} {:>12} {:>8} {:>12} {:>13} {:>12} {:>8}"
" {:>7} {:>11}\n",
"image", "angle_deg", "bkg", "resolution_A", "spots", "scale_G",
"sigma_M_deg", "cc_to_merge", "cc_n", "merged", "disposition");
"ordinal", "dataset", "image", "angle_deg", "bkg", "resolution_A", "spots",
"scale_G", "sigma_M_deg", "cc_to_merge", "cc_n", "merged", "disposition");
// Whether "merged" has anything to answer at all. With no merge behind the run - --no-merge, or
// a sweep nothing indexed on - a column of zeros would read as a sweep every frame of which was
@@ -128,10 +155,6 @@ std::string PerImagePlotText(const PerImagePlot &plot) {
[](float g) { return std::isfinite(g); });
for (size_t i = 0; i < plot.image_count; i++) {
const float angle = goniometer
? goniometer->GetAngle_deg(static_cast<float>(i))
+ goniometer->GetWedge_deg() / 2.0f
: NAN;
const float scale_g = Element(plot.scale_g, i);
// Did this image's observations reach the merged data? The sweep-quality diagnostic says so
// outright; where it did not run, an image with no fitted scale contributed nothing, which
@@ -154,9 +177,9 @@ std::string PerImagePlotText(const PerImagePlot &plot) {
} else if (merge_ran && std::isfinite(scale_g))
disposition = 1.0f;
file += fmt::format("{:>6} {:>10.4f} {:>12.4g} {:>12.4g} {:>8.0f} {:>12.5g} {:>13.4f}"
file += fmt::format("{:>8} {:>7} {:>7} {:>10.4f} {:>12.4g} {:>12.4g} {:>8.0f} {:>12.5g} {:>13.4f}"
" {:>12.4f} {:>8} {:>7} {:>11}\n",
i, angle, Element(plot.bkg_estimate, i), Element(plot.resolution_A, i),
i, plot.dataset[i], plot.image[i], Element(plot.angle_deg, i), Element(plot.bkg_estimate, i), Element(plot.resolution_A, i),
Element(plot.spot_count, i), scale_g, Element(plot.mosaicity_deg, i),
Element(plot.cc_to_merge, i), Element(plot.cc_n, i),
merge_ran ? std::to_string(merged) : "nan", disposition);
+13 -3
View File
@@ -29,9 +29,14 @@ std::string RenderDetectorDiagnostic(const DiffractionExperiment &experiment,
// The columns of <prefix>_plot.txt. Every vector is indexed by the processed-image ordinal - the
// numbering of every per-image array rugnux writes - and a value nothing measured is NAN.
// dataset and image say which input (from 1, in the order given) and which of its own images a row is,
// so a run made of several files - stills masters read as one, a joint merge of sweeps - plots per file.
// frame_disposition holds FrameDisposition (Merge.h) and is empty when the sweep-quality diagnostic
// did not run.
struct PerImagePlot {
std::vector<int> dataset;
std::vector<int64_t> image;
std::vector<float> angle_deg; // at mid-exposure; NAN without a goniometer
std::vector<float> bkg_estimate;
std::vector<float> resolution_A;
std::vector<float> spot_count;
@@ -40,10 +45,15 @@ struct PerImagePlot {
std::vector<float> cc_to_merge;
std::vector<int> cc_n;
std::vector<uint8_t> frame_disposition;
std::optional<GoniometerAxis> goniometer; // the angle column; none: nan
size_t image_count = 0; // the rows
};
// The text of <prefix>_plot.txt: one '#' legend line naming the columns, then one whitespace-separated
// row per image, so `plot '<f>' using 1:4` works with nothing in between.
// The rows a joint merge of sweeps (--sweeps joint) holds of one sweep, from `first_row` on: which input
// and image each is, its angle and its per-image analysis, all from that sweep's own run (`own`). The
// scaling columns stay the joint merge's.
void FillJointPlotRows(PerImagePlot &joint, size_t first_row, int dataset, const PerImagePlot &own);
// The text of <prefix>_plot.txt: '#' lines saying what each column is, the last of them naming the
// columns over their own fields, then one whitespace-separated row per image, so
// `plot '<f>' using 1:5` works with nothing in between.
std::string PerImagePlotText(const PerImagePlot &plot);
+13 -2
View File
@@ -1420,8 +1420,19 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
}
if (result.has_merge_statistics)
plot.frame_disposition = result.merge_statistics.sweep_quality.frame_disposition;
plot.goniometer = experiment_.GetGoniometer();
plot.image_count = static_cast<size_t>(images_to_process);
// One input as far as this run knows: the caller renumbers a run of several (rugnux_cli.cpp).
// The goniometer is already shifted to the processed range, so ordinal i is at its angle i.
const auto goniometer = experiment_.GetGoniometer();
// A joint merge's rows are the frames of all its sweeps, more than the reference sweep's file holds.
plot.image_count = joint ? indexer->GetIntegrationOutcome().size() : static_cast<size_t>(images_to_process);
plot.dataset.assign(plot.image_count, 1);
plot.image.resize(plot.image_count);
plot.angle_deg.assign(plot.image_count, NAN);
for (size_t i = 0; i < plot.image_count; i++) {
plot.image[i] = start_image + static_cast<int64_t>(i) * config_.stride;
if (goniometer)
plot.angle_deg[i] = goniometer->GetAngle_deg(static_cast<float>(i)) + goniometer->GetWedge_deg() / 2.0f;
}
result.outputs->plot = std::move(plot);
}
+2
View File
@@ -14,6 +14,7 @@
#include "../gemmi_gph/gemmi/symmetry.hpp"
#include "../image_analysis/IntegrationOutcome.h"
#include "../image_analysis/geom_refinement/CellEsd.h"
#include "DiagnosticOutput.h"
// Several sweeps of one crystal merged as one (--sweeps joint), in two levels: each sweep is processed on
// its own - indexed, integrated, its partials combined into fulls and scaled - and the fulls of all of them
@@ -31,6 +32,7 @@ struct JointMemberSweep {
std::vector<MergedReflection> merged;
std::vector<ScaledFull> fulls; // all its fulls, unmerged, on the same axes
std::vector<MtzTable::Batch> batches; // the batch headers of its own unmerged MTZ
PerImagePlot plot; // its own per-image table, for the joint one
double cc_half = NAN, completeness = NAN, d_min = NAN; // its merge on its own
double isa = NAN; // ... and its ISa, NaN where b was not resolved
double distance_mm = NAN; // the detector distance its run ended at
+19
View File
@@ -3974,6 +3974,18 @@ int RugnuxInputRun::RunPreparedInput() {
}
}
// The table of a run read from several inputs: each row in its own input's numbering.
if (multi_reader && result.outputs && result.outputs->plot) {
PerImagePlot &plot = *result.outputs->plot;
for (size_t i = 0; i < plot.image_count; i++) {
size_t k = multi_reader->GetNumberOfInputs() - 1;
while (plot.image[i] < multi_reader->FirstImage(k))
k--;
plot.dataset[i] = static_cast<int>(k) + 1;
plot.image[i] -= multi_reader->FirstImage(k);
}
}
// Every file the run made, from what it returned.
if (!output_prefix.empty()) {
try {
@@ -4322,6 +4334,8 @@ int RugnuxCli::RunJointSweeps(const RugnuxSweepPlan &plan) {
m.fulls = merged.scaled_fulls;
if (sweep.member_result->outputs->unmerged)
m.batches = sweep.member_result->outputs->unmerged->batches;
if (sweep.member_result->outputs->plot)
m.plot = *sweep.member_result->outputs->plot;
m.cc_half = merged.statistics.overall.cc_half;
m.completeness = merged.statistics.overall.possible_unique_reflections > 0
? 100.0 * merged.statistics.overall.unique_reflections
@@ -4543,6 +4557,11 @@ int RugnuxCli::RunJointSweeps(const RugnuxSweepPlan &plan) {
: relation.sweeps[k].why_left_out)});
result.joint_sweeps.push_back(std::move(w));
}
// The joint table's rows are the sweeps' frames end to end: each says which sweep and frame it is.
if (result.outputs && result.outputs->plot)
for (size_t seg = 0; seg < input.segments.size(); ++seg)
FillJointPlotRows(*result.outputs->plot, input.segments[seg].first_frame,
input.segment_sweep[seg] + 1, members[input.segment_sweep[seg]].plot);
if (result.outputs && result.outputs->merged && export_unmerged) {
const MergedOutput &merged = *result.outputs->merged;
std::vector<std::string> history{"From Rugnux " + jfjoch_version() + ", data reduction",
+114 -46
View File
@@ -25,6 +25,24 @@ namespace {
return ret;
}
// One input, its images in order: what a run of one file writes.
void OneDataset(PerImagePlot &plot, size_t n) {
plot.image_count = n;
plot.dataset.assign(n, 1);
plot.image.resize(n);
for (size_t i = 0; i < n; i++)
plot.image[i] = static_cast<int64_t>(i);
}
// The table's rows: the lines after its '#' header.
std::vector<std::string> Rows(const std::vector<std::string> &lines) {
std::vector<std::string> ret;
for (const auto &line : lines)
if (line[0] != '#')
ret.push_back(line);
return ret;
}
std::vector<std::string> ReadLines(const std::string &path) {
std::ifstream f(path);
std::vector<std::string> ret;
@@ -93,6 +111,9 @@ TEST_CASE("DiagnosticPlot_Format") {
const std::string prefix = "diagnostic_plot_format_test";
PerImagePlot plot;
OneDataset(plot, 4);
plot.image = {10, 12, 14, 16};
plot.angle_deg = {10.25f, 10.75f, 11.25f, 11.75f};
plot.bkg_estimate = {1.0f, 2.0f, 3.0f, 4.0f};
plot.resolution_A = {2.5f, 2.4f, NAN, 2.6f};
plot.spot_count = {100.0f, 120.0f, 0.0f, 90.0f};
@@ -104,57 +125,66 @@ TEST_CASE("DiagnosticPlot_Format") {
static_cast<uint8_t>(FrameDisposition::Downgraded),
static_cast<uint8_t>(FrameDisposition::Rejected),
static_cast<uint8_t>(FrameDisposition::Rejected)};
const GoniometerAxis goniometer("omega", 10.0f, 0.5f, {0, 1, 0}, {});
plot.goniometer = goniometer;
plot.image_count = 4;
WritePerImagePlot(prefix, plot);
const auto lines = ReadLines(prefix + "_plot.txt");
const auto all = ReadLines(prefix + "_plot.txt");
// The header first, all of it comments, the last line of it the legend; no '#' after it.
size_t header = 0;
while (header < all.size() && all[header][0] == '#')
header++;
REQUIRE(header > 1);
const auto lines = std::vector<std::string>(all.begin() + static_cast<long>(header) - 1, all.end());
REQUIRE(lines.size() == 5);
// Exactly one legend row, and it is a comment.
REQUIRE(lines[0][0] == '#');
for (size_t i = 1; i < lines.size(); i++)
CHECK(lines[i].find('#') == std::string::npos);
// Every column is described in the header.
const auto legend = Columns(lines[0]);
REQUIRE(legend.size() == 11);
CHECK(legend[0] == "#image");
CHECK(legend[1] == "angle_deg");
CHECK(legend[6] == "sigma_M_deg");
CHECK(legend[8] == "cc_n");
CHECK(legend[9] == "merged");
CHECK(legend[10] == "disposition");
REQUIRE(legend.size() == 13);
for (size_t c = 1; c < legend.size(); c++) {
bool described = false;
for (size_t l = 0; l + 1 < header; l++)
described |= all[l].starts_with("# " + legend[c] + ":");
CHECK(described);
}
CHECK(legend[0] == "#ordinal");
CHECK(legend[1] == "dataset");
CHECK(legend[2] == "image");
CHECK(legend[3] == "angle_deg");
CHECK(legend[8] == "sigma_M_deg");
CHECK(legend[10] == "cc_n");
CHECK(legend[11] == "merged");
CHECK(legend[12] == "disposition");
for (size_t i = 1; i < lines.size(); i++) {
const auto row = Columns(lines[i]);
REQUIRE(row.size() == legend.size());
CHECK(row[0] == std::to_string(i - 1));
CHECK(row[1] == "1");
CHECK(row[2] == std::to_string(10 + 2 * (i - 1)));
}
// Mid-exposure angle: start + ordinal * increment + increment / 2.
CHECK(std::stof(Columns(lines[1])[1]) == 10.25f);
CHECK(std::stof(Columns(lines[3])[1]) == 11.25f);
CHECK(std::stof(Columns(lines[1])[3]) == 10.25f);
CHECK(std::stof(Columns(lines[3])[3]) == 11.25f);
// Merged and downgraded frames are in the merged data; a rejected one is not.
CHECK(Columns(lines[1])[9] == "1");
CHECK(Columns(lines[2])[9] == "1");
CHECK(Columns(lines[3])[9] == "0");
CHECK(Columns(lines[4])[9] == "0");
CHECK(Columns(lines[1])[11] == "1");
CHECK(Columns(lines[2])[11] == "1");
CHECK(Columns(lines[3])[11] == "0");
CHECK(Columns(lines[4])[11] == "0");
// Graded: merged 1, downgraded 0.5, rejected 0 - and a rejected frame that never got a scale
// never reached the merge, so nothing is known of it.
CHECK(Columns(lines[1])[10] == "1");
CHECK(Columns(lines[2])[10] == "0.5");
CHECK(Columns(lines[3])[10] == "nan");
CHECK(Columns(lines[4])[10] == "0");
CHECK(Columns(lines[1])[12] == "1");
CHECK(Columns(lines[2])[12] == "0.5");
CHECK(Columns(lines[3])[12] == "nan");
CHECK(Columns(lines[4])[12] == "0");
// The count the correlation beside it was taken over.
CHECK(Columns(lines[1])[8] == "500");
CHECK(Columns(lines[1])[10] == "500");
// A frame nothing measured leaves its own columns unplottable, not the row missing.
CHECK(Columns(lines[3])[3] == "nan");
CHECK(Columns(lines[3])[5] == "nan");
}
TEST_CASE("DiagnosticPlot_NoDisposition") {
@@ -162,22 +192,22 @@ TEST_CASE("DiagnosticPlot_NoDisposition") {
// Without the sweep-quality diagnostic, a frame counts as merged when it was scaled at all.
PerImagePlot plot;
OneDataset(plot, 3);
plot.scale_g = {1.0f, NAN, 0.8f};
plot.image_count = 3;
WritePerImagePlot(prefix, plot);
const auto lines = ReadLines(prefix + "_plot.txt");
REQUIRE(lines.size() == 4);
CHECK(Columns(lines[1])[9] == "1");
CHECK(Columns(lines[2])[9] == "0");
CHECK(Columns(lines[3])[9] == "1");
CHECK(Columns(lines[1])[10] == "1");
CHECK(Columns(lines[2])[10] == "nan");
CHECK(Columns(lines[3])[10] == "1");
const auto lines = Rows(ReadLines(prefix + "_plot.txt"));
REQUIRE(lines.size() == 3);
CHECK(Columns(lines[0])[11] == "1");
CHECK(Columns(lines[1])[11] == "0");
CHECK(Columns(lines[2])[11] == "1");
CHECK(Columns(lines[0])[12] == "1");
CHECK(Columns(lines[1])[12] == "nan");
CHECK(Columns(lines[2])[12] == "1");
// No goniometer: the angle column is present and unplottable, so the columns never shift.
CHECK(Columns(lines[1]).size() == 11);
CHECK(Columns(lines[1])[1] == "nan");
CHECK(Columns(lines[0]).size() == 13);
CHECK(Columns(lines[0])[3] == "nan");
}
TEST_CASE("DiagnosticPlot_NoMerge") {
@@ -186,16 +216,54 @@ TEST_CASE("DiagnosticPlot_NoMerge") {
// Nothing was merged - --no-merge, or a sweep nothing indexed on - so "merged" has no answer to
// give and must not read as a sweep every frame of which was thrown out.
PerImagePlot plot;
OneDataset(plot, 2);
plot.bkg_estimate = {12.0f, 13.0f};
plot.scale_g = {NAN, NAN};
plot.image_count = 2;
WritePerImagePlot(prefix, plot);
const auto lines = ReadLines(prefix + "_plot.txt");
REQUIRE(lines.size() == 3);
CHECK(Columns(lines[1])[9] == "nan");
CHECK(Columns(lines[2])[9] == "nan");
CHECK(Columns(lines[1])[10] == "nan");
const auto lines = Rows(ReadLines(prefix + "_plot.txt"));
REQUIRE(lines.size() == 2);
CHECK(Columns(lines[0])[11] == "nan");
CHECK(Columns(lines[1])[11] == "nan");
CHECK(Columns(lines[0])[12] == "nan");
// The columns a run without a merge still fills are there.
CHECK(std::stof(Columns(lines[1])[2]) == 12.0f);
CHECK(std::stof(Columns(lines[0])[4]) == 12.0f);
}
TEST_CASE("DiagnosticPlot_JointSweeps") {
const std::string prefix = "diagnostic_plot_joint_test";
// Two sweeps of 2 and 3 frames, as their own runs measured them.
PerImagePlot a, b;
OneDataset(a, 2);
a.angle_deg = {0.05f, 0.15f};
a.bkg_estimate = {5.0f, 6.0f};
OneDataset(b, 3);
b.angle_deg = {90.05f, 90.15f, 90.25f};
b.spot_count = {7.0f, 8.0f, 9.0f};
// The joint merge's table: its rows are the sweeps' frames end to end, its scale the joint merge's.
PerImagePlot joint;
OneDataset(joint, 5);
joint.scale_g = {1.0f, 1.0f, 2.0f, 2.0f, 2.0f};
FillJointPlotRows(joint, 0, 1, a);
FillJointPlotRows(joint, 2, 2, b);
WritePerImagePlot(prefix, joint);
const auto lines = Rows(ReadLines(prefix + "_plot.txt"));
REQUIRE(lines.size() == 5);
const std::vector<std::string> dataset = {"1", "1", "2", "2", "2"};
const std::vector<std::string> image = {"0", "1", "0", "1", "2"};
for (size_t i = 0; i < lines.size(); i++) {
const auto row = Columns(lines[i]);
CHECK(row[0] == std::to_string(i));
CHECK(row[1] == dataset[i]);
CHECK(row[2] == image[i]);
}
CHECK(std::stof(Columns(lines[0])[3]) == 0.05f);
CHECK(std::stof(Columns(lines[4])[3]) == 90.25f);
CHECK(std::stof(Columns(lines[1])[4]) == 6.0f);
CHECK(Columns(lines[2])[4] == "nan");
CHECK(std::stof(Columns(lines[3])[6]) == 8.0f);
CHECK(std::stof(Columns(lines[2])[7]) == 2.0f);
}