Files
SP2XR/config/my_data_schema.yaml
2025-11-21 15:00:26 +01:00

252 lines
8.0 KiB
YAML

'# INSTRUCTIONS':
- This config file contains both schema definitions and column mappings.
- 1. The *_schema sections define the data types for your input files.
- 2. The *_column_mapping sections map your file columns to canonical names.
- 3. Update column mappings if your files use different column names.
- 4. If your file doesn't have a particular canonical column, set it to null or remove
the line.
- 5. The output parquet files will use the canonical column names for consistency.
pbp_schema:
Time (sec): float
Packet Time Stamp: float
Flag: float
Dropped Records: float
Record Count: float
Record Size: float
Particle Time Stamp: float
Particle Flags: float
Scatter relPeak: float
Scatter Transit Time: float
Scatter Peak Time: float
Scatter FWHM: float
Scatter Size (nm): float
Incand relPeak: float
Incand Transit Time: float
Incand Peak Time: float
Incand FWHM: float
Incand Delay: float
Incand Mass (fg): float
Reserved: float
hk_schema:
Time Stamp: string
Time (sec): float
Time Stamp (UTC sec): float
Elapsed Time: float
Error Code: float
Packet Time Stamp: float
Laser TEC Temp (C): float
Crystal TEC Temp (C): float
Inlet Air Temp (C): float
Computer Heatsink Temp (C): float
Laser Heatsink Temp (C): float
Outlet Air Temp (C): float
YAG Output Monitor (V): float
Cavity Pressure (hPa): float
Laser Driver Power Monitor (uA): float
Laser Driver Current Limit Monitor (A): float
Laser Driver Current Monitor (A): float
Laser TEC Sense: float
Laser Over Temp (On/Off): float
+5V Laser Rail (V): float
' +5V Rail (V)': float
+12V Rail (V): float
High Voltage (V): float
Battery Temp (C): float
UPS Output (V): float
12V Iso Rail (V): float
5V Iso Rail (V): float
3.3V Iso Rail (V): float
Spare 22: float
Spare 23: float
408 Board Spare 0: float
408 Board Spare 1: float
408 Board Spare 2: float
408 Board Spare 3: float
408 Board Spare 4: float
Purge Flow Monitor (sccm): float
System Input Voltage (V): float
Board Temperature (C): float
408 Board Spare 8: float
408 Board Spare 9: float
408 Board Spare 10: float
408 Board Spare 11: float
408 Board Spare 12: float
408 Board Spare 13: float
408 Board Spare 14: float
408 Board Spare 15: float
Sheath Flow Controller Read (vccm): float
Sheath Flow Controller Read (sccm): float
Sheath Flow Controller Pressure (psia): float
Sheath Flow Controller Temperature (C): float
Sample Flow Controller Read (vccm): float
Sample Flow Controller Read (sccm): float
Sample Flow Controller Pressure (psia): float
Sample Flow Controller Temperature (C): float
Fan 1 (RPM): float
Fan 2 (RPM): float
Laser Fan (RPM): float
Spare tach: float
Threshold Crossing Events: float
Dual Qualified Scatter and Incand Particles: float
Qualified Scatter Only Particles: float
Qualified Incand Only Particles: float
Disqualified Due to Scatter Saturation: float
Disqualified Due to Scatter Transit Time Min: float
Disqualified Due to Scatter Transit Time Max: float
Disqualified Due to Scatter FWHM Min: float
Disqualified Due to Scatter FWHM Max: float
Scatter Inter Part Period Min Violation: float
Disqualified Due to Incand Saturation: float
Disqualified Due to Incand Transit Time Min: float
Disqualified Due to Incand Transit Time Max: float
Disqualified Due to Incand FWHM Min: float
Disqualified Due to Incand FWHM Max: float
Incand Inter Part Period Min Violation: float
Baseline Sizer Lo: float
Baseline Sizer Hi: float
Baseline Incand Lo: float
Baseline Incand Hi: float
Bandwidth Sizer Hi: float
Bandwidth Sizer Lo: float
Bandwidth Incand Lo: float
Bandwidth Incand Hi: float
ABD-0408 HK ADCs min: float
ABD-0436 HK ADCs min: float
ABD-0408 HK ADCs max: float
ABD-0436 HK ADCs max: float
Incand Particle Conc (cts/ccm): float
Scattering Particle Conc (cts/ccm): float
Incand Mass Conc (fg/sccm): float
Scattering Mass Conc (fg/sccm): float
Sheath Flow Set Point: float
Sample Flow Set Point: float
Laser Temp Set Point: float
Laser Current Set Point: float
Spare 4 Set Point: float
Spare 5 Set Point: float
PMT HV Set Point: float
Particle Density (g/ccm): float
PbP Packet Time: float
Scatter Bin 1: float
Scatter Bin 2: float
Scatter Bin 3: float
Scatter Bin 4: float
Scatter Bin 5: float
Scatter Bin 6: float
Scatter Bin 7: float
Scatter Bin 8: float
Scatter Bin 9: float
Scatter Bin 10: float
Scatter Bin 11: float
Scatter Bin 12: float
Scatter Bin 13: float
Scatter Bin 14: float
Scatter Bin 15: float
Scatter Bin 16: float
Scatter Bin 17: float
Scatter Bin 18: float
Scatter Bin 19: float
Scatter Bin 20: float
Incand Bin 1: float
Incand Bin 2: float
Incand Bin 3: float
Incand Bin 4: float
Incand Bin 5: float
Incand Bin 6: float
Incand Bin 7: float
Incand Bin 8: float
Incand Bin 9: float
Incand Bin 10: float
Incand Bin 11: float
Incand Bin 12: float
Incand Bin 13: float
Incand Bin 14: float
Incand Bin 15: float
Incand Bin 16: float
Incand Bin 17: float
Incand Bin 18: float
Incand Bin 19: float
Incand Bin 20: float
pbp_canonical_schema:
Time (sec): float
Packet Time Stamp: float
Flag: float
Dropped Records: float
Record Count: float
Record Size: float
Particle Time Stamp: float
Particle Flags: float
Scatter relPeak: float
Scatter Transit Time: float
Scatter Peak Time: float
Scatter FWHM: float
Scatter Size (nm): float
Incand relPeak: float
Incand Transit Time: float
Incand Peak Time: float
Incand FWHM: float
Incand Delay: float
Incand Mass (fg): float
Reserved: float
hk_canonical_schema:
Time Stamp: datetime
Time (sec): float
Sample Flow Controller Read (sccm): float
Sample Flow Controller Read (vccm): float
Time Stamp (UTC sec): float
Elapsed Time: float
Error Code: float
Packet Time Stamp: float
Laser TEC Temp (C): float
Crystal TEC Temp (C): float
Inlet Air Temp (C): float
Computer Heatsink Temp (C): float
Laser Heatsink Temp (C): float
Outlet Air Temp (C): float
YAG Output Monitor (V): float
Cavity Pressure (hPa): float
Laser Driver Power Monitor (uA): float
Laser Driver Current Limit Monitor (A): float
Laser Driver Current Monitor (A): float
pbp_column_mapping:
Time (sec): Time (sec)
Packet Time Stamp: Packet Time Stamp
Flag: Flag
Dropped Records: Dropped Records
Record Count: Record Count
Record Size: Record Size
Particle Time Stamp: Particle Time Stamp
Particle Flags: Particle Flags
Scatter relPeak: Scatter relPeak
Scatter Transit Time: Scatter Transit Time
Scatter Peak Time: Scatter Peak Time
Scatter FWHM: Scatter FWHM
Scatter Size (nm): Scatter Size (nm)
Incand relPeak: Incand relPeak
Incand Transit Time: Incand Transit Time
Incand Peak Time: Incand Peak Time
Incand FWHM: Incand FWHM
Incand Delay: Incand Delay
Incand Mass (fg): Incand Mass (fg)
Reserved: Reserved
hk_column_mapping:
Time Stamp: Time Stamp
Time (sec): Time (sec)
Sample Flow Controller Read (sccm): Sample Flow Controller Read (sccm)
Sample Flow Controller Read (vccm): Sample Flow Controller Read (vccm)
Time Stamp (UTC sec): Time Stamp (UTC sec)
Elapsed Time: Elapsed Time
Error Code: Error Code
Packet Time Stamp: Packet Time Stamp
Laser TEC Temp (C): Laser TEC Temp (C)
Crystal TEC Temp (C): Crystal TEC Temp (C)
Inlet Air Temp (C): Inlet Air Temp (C)
Computer Heatsink Temp (C): Computer Heatsink Temp (C)
Laser Heatsink Temp (C): Laser Heatsink Temp (C)
Outlet Air Temp (C): Outlet Air Temp (C)
YAG Output Monitor (V): YAG Output Monitor (V)
Cavity Pressure (hPa): Cavity Pressure (hPa)
Laser Driver Power Monitor (uA): Laser Driver Power Monitor (uA)
Laser Driver Current Limit Monitor (A): Laser Driver Current Limit Monitor (A)
Laser Driver Current Monitor (A): Laser Driver Current Monitor (A)