FLASH HDF5 structure

The photon diagnostic, electron diagnostic and beamline information as well as the information about the pump-probe laser and the infrastructure offered for users (GHz/MHz ADCs) can be included in one HDF5 file which is organised according to train IDs. The general structure is:

Electron Diagnostic

Photon Diagnostics

Beamlines

Experiment

Timing

A detailled description of (most) channels can be found in the lower part of the hdf5 viewer:

/FL1/Photon Diagnostic/GMD/Pulse resolved energy/energy tunnelalways saved (PBD)DOOCS prop : TTF2.DAQ/PHFLUX/OUT34/VALDAQ channel: PBD.PHFLUX/TUNNEL.ENERGYPULSE.USERdesc :Energy per pulse Tunnel (from e-) - the values are set to "0" if there was no SASE beam in the FELunits : a.u. (more or less µJ but need to be calibrated with the "Average energy" for good precision) see here for help

/FL1/Photon Diagnostic/GMD/Pulse resolved energy/energy tunnel (raw)always saved (PBD)DOOCS prop : TTF2.DAQ/PHFLUX/OUT14/VALDAQ channel: PBD.PHFLUX/TUNNEL.ENERGYPULSE.FFdesc :Energy per pulse Tunnel (from e-) - uncorrected values. There are also values saved if there was no beam ... just background noise units : a.u. (more or less µJ but need to be calibrated with the "Average energy" for good precision) see here for help

/FL1/Photon Diagnostic/GMD/Pulse resolved energy/energy BDAalways saved (PBD)DOOCS prop : TTF2.DAQ/PHFLUX/OUT35/VALDAQ channel: PBD.PHFLUX/BDA.ENERGYPULSE.USERdesc :Energy per pulse BDA (from e-) - the values are set to "0" if there was no SASE beam in the FELunits : a.u. (more or less µJ but need to be calibrated with the "Average energy" for good precision) see here for help

/FL1/Photon Diagnostic/GMD/Pulse resolved energy/energy BDA (raw)always saved (PBD)DOOCS prop : TTF2.DAQ/PHFLUX/OUT15/VALDAQ channel: PBD.PHFLUX/BDA.ENERGYPULSE.FFdesc :Energy per pulse BDA (from e-) - uncorrected values. There are also values saved if there was no beam ... just background noise units : a.u. (more or less µJ but need to be calibrated with the "Average energy" for good precision)see here for help

arrival time

/FL1/Electron Diagnostic/BAM/4DBC3/electron bunch arrival time (low charge)always saved (PBD)DOOCS prop : FLASH.SDIAG/BAM/4DBC3/LOW_CHARGE_ARRIVAL_TIMEDAQ channel: FLASH.SDIAG/BAM.DAQ/4DBC3.LOW_CHARGE_ARRIVAL_TIMEdesc: Electron bunch arrival time measured with the BAM (more or less) before the undulator (pulse resolved data) units: ps (bigger numbers indicate later arrivaltime of the electrons) note: besides the arivaltime from FLASH1 there is also the FLASH2/3 electron arrival time saved. LINK to detailled infos from MSK (may only work inside DESY network and to a recent talk about the working principle of the BAM

set number of pulses

/FL1/Timing/set number of bunchesalways saved (PBD)DOOCS prop : FLASH.DIAG/TOROID.ML/3GUN/NUMBEROFBUNCHES.FLASH1DAQ channel: TTF2.UTIL/LASER.CONTROL/GUN/PULSE_NUMdesc: Number of pulses set at the gun (FLASH1) units:

actual number of pulses

/FL1/Timing/actual number of bunchesalways saved (PBD)DOOCS prop : FLASH.DIAG/TOROID.ML/12EXP/NUMBEROFBUNCHES.FLASH1DAQ channel: TTF2.DIAG/PBD.TOROID.ML/12EXPdesc: Number of bunches measured BEHIND the undulator. If pulses are used for diagnostic of the protection system of the accelerator limits the number of bunches to be accelerated this is the actual number that created XUV radiation.The number is calculated by the DAQ middle layer server, (FLASH1) units:

actual pulse pattern recorded after the undulator

/FL1/Timing/Bunch pattern/pattern after undulatoralways saved (PBD)DOOCS prop : TTF2.DIAG/PBD.TOROID.ML/12EXP/CHARGE.TDDAQ channel: TTF2.DIAG/PBD.TOROID.ML/12EXPdesc: The bunch pattern as function of time in a burst recorded by toroide diagnostic BEHIND the undulator. (FLASH1) units:

Train ID

/Timing/train IDalways saved (PBD)DOOCS prop : noneDAQ channel: nonedesc: Each 10 Hz burst has its unique train ID. For the HDF5 data set the ID is the same for all parameters with the same index (note camera images may be shifted by 1 ID - talk to the experts !) units:

Train time

/Timing/train timedesc:Local time as array of day, hour, minute, second, and centisecond. This data set is meant for visualization purposes only. For correlations use the train ID or the Unix time of the time stamp units: d h min s cs

/Timing/time stampdesc:first column: Local time in unix time. To get day, hour, minute, second you can use unix: e.g. date --date='@1553617729' or matlab, python etc second column: microseconds third column: Train ID of FLASH

/FL1/Experiment/Pump probe laser/delay line IK220.0/ENC.DELAYalways saved (PBD)DOOCS prop : TTF2.FEL/DELLINE.ENC/IK220.0/ENC.DELAYDAQ channel: TTF2.FEL/DELLINE.ENC/IK220.0:ENC.DELAYsubsystem: DELLINE.ENC desc : delay of the Pump probe laser - measured by an encoder. The position is read out with 10Hz train synchronized and should be used to determine the actual laser delay (the motor position is only read out about every second units : ps ( pos delay means IR comes later)

User Data (FLASH1)

The data saved specifically for detectors at an experiment will show up in /Experiment/ there is a large number of options for cameras or monitoring pslow properties (motor positons etc) for user experiments. For details please ask your local contact.

NOTE: If parameters for an experiment are included on short notice the correct naming in the HDF5 may not be in time and the data will show up in /uncategorized/ with the DOOCS names

The most common and permanently installed device used by experiment are our ADCs:

GHz ADCs

ADC traces of the (SPDevices 412) GHZ ADCs available for the users . More information about the ADCs can be foundheresaved on DEMAND in the user DAQ

DOOCS prop : FLASH.FEL/ADC.ADQ.PG/EXP1.CH00/CH00.TD or CH00.DAQ.TDhere the CH00.TD is the full ADC trace as it is sampled ( typically several 100.000 samples per pulse train) while the CH00.DAQ.TD trace only has the number of samples which are sent to the DAQ OR if grouping is activated the CH00.DAQ.TD conatins only the grouped spectra. To read the ADC trace with an online analysis program the CH00.DAQ.TD is used preferablly.DAQ channel: FLASH.FEL/ADC.ADQ.PG/EXP1.CH00

In addition there are also additional parameters saved like:

sample frequency: it shows the sample frequency in MHz (number of samples per µs). NOTE: the clock of the ADC is NOT synchronized to the FLASH timing system. Thus the number of samples between bunches in the bunch train may be not integer numbers which will be show up for long bunch trains.

number of samples: total number of samoles recorded for each 10 Hz trigger

sample frequency: it shows the sample frequency in MHz (number of samples per µs). NOTE: the clock of the ADC is NOT synchronized to the FLASH timing system. Thus the number of samples between bunches in the bunch train may be not integer numbers which will be show up for long bunch trains.

number of samples: total number of samoles recorded for each 10 Hz trigger

/FL2/Photon Diagnostic/GMD/Average energy/energy tunnel uncertaintyalways saved (PBD2)DOOCS prop : FLASH.FEL/XGM.PHOTONFLUX/FL2.TUNNEL/PHOTONFLUX.UJ.SIGMA DAQ channel: FLASH.FEL/XGM.PHOTONFLUX/FL2.TUNNEL/PHOTONFLUX.UJ.SIGMA desc : This parameter gives an indication of the error of the measurement of the average pulse energy. This takes signal to noise, detector resolution, uncertainties in crossection etc into account. (it is NOT the measurement of the statistical fluctuation of the SASE pulses)units : microJ

/FL2/Photon Diagnostic/GMD/Pulse resolved energy/energy tunnelalways saved (PBD2)DOOCS prop : FLASH.FEL/XGM.INTENSITY/FL2.TUNNEL/INTENSITY.TDDAQ channel: FLASH.FEL/XGM.INTENSITY/FL2.TUNNEL/INTENSITY.TDdesc : Energy per pulse measured in the Tunnel (in fromt of the gas attenuator and the apertures in the Hall)units : a.u. (more or less µJ but need to be calibrated with the "Average energy" for good precision)see here for help

/FL2/Photon Diagnostic/GMD/Pulse resolved energy/energy tunnel uncertainty (sigma)always saved (PBD2)DOOCS prop : FLASH.FEL/XGM.INTENSITY/FL2.TUNNEL/INTENSITY.SIGMA.TDDAQ channel: FLASH.FEL/XGM.INTENSITY/FL2.TUNNEL:4desc :This parameter gives an indication of the error of the measurement of the pulse energy. This takes signal to noise, detector resolution, uncertainties in crossection etc into account. (it is NOT the measurement of the statistical fluctuation of the SASE pulses)units : a.u. (more or less µJ - related to the value of the pulse energy )

All values for the GMD are also available for the HALL GMD which is located in the experimental hall down stream the gas attenuator. If the attenuator is on the ratio between Hall and Tunnel signal shows the attenuation. BUT NOTE that the filter unit and the Aperture 4 are downstream the GMD. So if filters and aperture are used this influence is NOT measured by the GMD hall !

Besides pulse energy the GMD also provides information about the beam position

/FL2/Photon Diagnostic/GMD/Pulse resolved beam position/position tunnel x always saved (PBD2)DOOCS prop : FLASH.FEL/XGM.BPM/FL2.HALL/X.TDDAQ channel: FLASH.FEL/XGM.BPM/FL2.HALL:2 desc: Besides the well calibrated averaged beam position information there is also the option to measure the beam position on a single bunch level. HOWEVER this methide needs a perfectly adjusted signal level (talk to your local contact !!) and also then the signal to noise is rather small and one needs some averaging ... BUT with this option one can determine if there was a spatial slope on a burst (say forst bunches were lower than the last ones or so ...)units : mm

/FL2/Photon Diagnostic/Wavelength/OPIS tunnel/Processed/number of analyzed bunch (in older version this can be found in OPIS tunnel/Expert stuff/General operation parameters/) saved opon request (PBD2)DOOCS prop : FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL060DAQ channel: FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL060desc : The bunch number of the bunch used for the wavelength calculationunits :

If Opis is running typically on the the averaged data is saved. For several experiments it may make sense to save the information for each single bunch. This is up to now done by savng the compleate ADC trace of the TOF setup. This is a huge amount of data and needs processing. This has to be performed after the beamtime in close contact to Markus Braune ( respobsible for OPIS)

In case OPIS was not operating there is still informaton about the set wavelength for the undulators (see below) which may differ by up to 5 % from the actual wavelength due to different settings in the FEL ...

undulator settings

/FL2/Electron Diagnostic/Undulator setting/set wavelengthalways saved (PBD2)DOOCS prop : TTF2.FEEDBACK/FL2.WAVELENGTHCONTROL/FLASH2/WAVELENGTHDAQ channel: TTF2.FEEDBACK/FL2.WAVELENGTHCONTROL/FLASH2/WAVELENGTHdesc: Set value for the anticipated wavelength . This parameter is used to set the undulator gap. It may however deviate from the actual wavelength by several % ...units: nm

/FL2/Electron Diagnostic/Undulator setting/SASE14 gapalways saved (PBD2)DOOCS prop : FLASH.UTIL/FL2.UND.MOTOR/FL2SASE14/GAPDAQ channel: FLASH.UTIL/FL2.UND.MOTOR/FL2SASE14/GAPdesc: gap value of the undulators. This can be used to follow up how many undulators were closed and if there was a taperunits: mm

The gap values are saved for all 12 undulators (Nr 3 to 14). Undulator 14 is the one closest to the experimental hall.

The BAM measures the arrivaltime of FLASH 1 and FLASH2 in the same data set (thus also sorted in at /FL1/ !). The first values are for FLASH1 bunches. After a gap of about 70 micros ( 70 colums) with as entry for the switching between FLASH 1 and 2 the values for the electrons used in FLASH2 start. The start time of FLASH2 is also recorded in the DAQ. FLASH1 start time is for historic reasons 700. thus if e.g. the start time of FLASH2 ( property name see below) is 1200 it means that FLASH starts at column 500 (1200-700) ...

Timing information, rep rate etc. (FLASH2)

start time of FLASH2

/FL2/Timing/start time flash2always saved (PBD2)DOOCS prop : FLASH.DIAG/TIMER/FLASHCPUTIME1.0/BUNCH_POSITION.2DAQ channel: FLASH.DIAG/TIMER/FLASHCPUTIME1.0/BUNCH_POSITION.2desc: The max 600 µs acceleration time of FLASH is devided between FLASH1 and FLASH2. This 600 µs window starts with FLASH1 (up to now) at a time"label" of 700 µs (for historic reasons). Thus the first bunch of FLASH 1 comes at "700" and the last possibel bunch comes at 1300 (700+600). After FLASH1 train is over ther is a about 70µs switching time with no bunches. Then comes the first FLASH2 bunch. Thus if e.g. the start time of FLASH2 is 1200 it means that FLASH2 starts at column 500 (1200-700) in the HDF5 files. (However for yet unknown reasons this may change by 2-3 colums ...)units: µs

actual number of pulses

/FL1/Timing/actual number of bunchesalways saved (PBD2)DOOCS prop : FLASH.DIAG/PBD2.TOROID.ML/3GUN/NUMBEROFBUNCHES.FLASH2DAQ channel: FLASH.DIAG/PBD2.TOROID.ML/3GUN/NUMBEROFBUNCHES.FLASH2desc: Number of bunches measured BEHIND the undulator. If pulses are used for diagnostic of the protection system of the accelerator limits the number of bunches to be accelerated this is the actual number that created XUV radiation.The number is calculated by the DAQ middle layer server, (FLASH2) units:

actual pulse pattern recorded after the undulator

/FL1/Timing/Bunch pattern/pattern after undulatoralways saved (PBD2)DOOCS prop : FLASH.DIAG/PBD2.TOROID.ML/9FL2BURN.PULSEPATTERNDAQ channel: FLASH.DIAG/PBD2.TOROID.ML/9FL2BURN.PULSEPATTERNdesc: The bunch pattern as function of time in a burst recorded by toroide diagnostic BEHIND the undulator. (FLASH2) units:

Train ID

/Timing/train IDalways saved (PBD2)DOOCS prop : noneDAQ channel: nonedesc: Each 10 Hz burst has its unique train ID. For the HDF5 dataset the ID is the same for all parameters with the same index (note camera images may be shifted by 1 ID - talk to the experts !) units:

Train time

always saved (PBD2)/Timing/train timedesc:Local time as array of day, hour, minute, second, and centisecond. This dataset is meant for visualisation purposes only. For correlations use the train ID or the Unix time of the time stamp units: d h min s cs

/Timing/time stamp

always saved (PBD2)desc: first column: Local time in unix time. To get day, hour, minute, second you can use unix: e.g. date --date='@1553617729' or matlab, python etc second column: microseconds third column: Train ID of FLASH

User Data (FLASH2)

The data saved specifically for detectors at an experiment will show up in /Experiment/ there is a large number of options for cameras or monitoring pslow properties (motor positons etc) for user experiments. For details please ask your local contact.

NOTE: If parameters for an experiment are included on short notice the correct naming in the HDF5 may not be in time and the data will show up in /uncategorized/ with the DOOCS names

The most common and permanently installed device used by experiment are our ADCs:

GHz ADCs

ADC traces of the (SPDevices 412) GHZ ADCs available for the users . More information about the ADCs can be foundheresaved on DEMAND in the user DAQ

DOOCS prop : FLASH.FEL/ADC.ADQ.FL2EXP1/FL2EXP1.CH00/CH00.TD or CH00.DAQ.TDhere the CH00.TD is the full ADC trace as it is sampled ( typically several 100.000 samples per pulse train) while the CH00.DAQ.TD trace only has the number of samples which are sent to the DAQ OR if grouping is activated the CH00.DAQ.TD conatins only the grouped spectra. To read the ADC trace with an online analysis program the CH00.DAQ.TD is used preferablly.DAQ channel: FLASH.FEL/ADC.ADQ.FL2EXP1/FL2EXP1.CH00

In addition there are also additional parameters saved like:

sample frequency: it shows the sample frequency in MHz (number of samples per µs). NOTE: the clock of the ADC is NOT synchronized to the FLASH timing system. Thus the number of samples between bunches in the bunch train may be not integer numbers which will be show up for long bunch trains.

number of samples: total number of samoles recorded for each 10 Hz trigger

error (ADC): 0 indicates that there was no error

/CH0-CH3/offset: To use the full dynamic range of the ADC one can shift the base line . This offset is saved here.

sample frequency: it shows the sample frequency in MHz (number of samples per µs). NOTE: the clock of the ADC is NOT synchronized to the FLASH timing system. Thus the number of samples between bunches in the bunch train may be not integer numbers which will be show up for long bunch trains.

number of samples: total number of samoles recorded for each 10 Hz trigger

/FL1/Experiment/Pump probe laser/laser delay readbackalways saved (PBD2)DOOCS prop : FLASH.SYNC/LASER.LOCK.EXP/FLASH2.PPL1.OSC1/FMC0.MD22.0.POSITION.RDDAQ channel: FLASH.SYNC/LASER.LOCK.EXP/FLASH2.PPL1.OSC1/FMC0.MD22.0.POSITION.RDdesc: delay of the Pump probe laser - measured by the read back position of the motor. There is also the set value available ( upto now these values are only updating every 1-2 seconds. There is no fast encoder property as on FLASH1 available)units : ps )

"/uncategorized/"

If parameters for an experiment are included on short notice the correct naming in the HDF5 may not be in time and the data will show up in /uncategorized/ with their DOOCS names

HDF5 structure revisions

Starting with Beamblock 4, August 2018, the hierarchy of the HDF group names have been adapted to reflect the new situation at FLASH. FLASH2 is operating for users quite some time now. Therefore, both accelerators appear equally in their respective HDF groups, namely "/FL1" and "/FL2". The root group of proper, by run organised HDF files have an attribute called "version". This version attribute has changed from "0.2.x" to "0.3.x". The changes in detail:

All FLASH1 related HDF groups moved to group "/FL1", i.e. a new prefix "/FL1" is added to their HDF path.

The ambigious term "pulse" has been replaced by "train" to refer to "pulse train". Most notably, the dataset "/Timing/pulse ID" has changed to "/Timing/train ID".

A number of inconsistent names have been streamlined. The relevant changes are listed in the following table.