Overview

The Payload System, a part of the DAQ-PROD project, has mainly two features.

• Dispatch data inside the DAQ system
• Serialization --- the serialized date will be sent to the offline-softwares written by C++

In this page, I would concentrate the second feature.

Data Structure

Serialized data is currently written by a derived class of DataOutputAdapter which implements java.io.DataOutput interface. That means, the outputs are essentially consist of boolean, byte(correspond to char in C++), int, float, etc. but not include any complicated objects.

Every data are packed in the proper structure, the Payload and its descendants. For example, bunch of data associated with an event should be enclosed in an EventPayload. An EventPayload can have a EventRecode to describe some informations about the event, and other (smaller) payloads, and its Envelope.

Payloads

There are five types of payloads. Every payload must have an envelope on the top of the payload. Following the envelope, a Record, a Payload, Data can be placed.

Here is some examples of each types.

• case 1) HitPayload...
• case 2) ReadoutRequestPayload...
• case 3) it can be...
• case 4) EventPayload_v2, TriggerRequestPayload, EngineeringFormatTriggerPayload...
• case 5) EventPayload_v2, TriggerRequestPayload, ReadoutDataPayload...

As you can see, case 5) has more complecated structure. This type would be a decsendant of AbstractCompositePayload. It must have an Composite Envelope and one or more payloads(it can be again a composite payload), but not have to have an Record.

Envelopes

There are only two types of envelopes.
One is general envelope named PayloadEnvelope which must be placed on the top of a payload, and recodes the length of payload (including the PayloadEnvelope itself), integer number of payload type, and UTCTime which the payload was created.
The other is CompositePayloadEnvelope which will appear only when the payload is a descendant of an AbstractCompositePayload, in order to describe the number of contained payload, composite type, and length of the contained payloads (including the length of CompositePayloadEnvelope itself).

The Envelope must be written by BIG_ENDIAN.

Records

Records have to be placed right after the Envelope. The first field is 4byte int value of Recode_Type, which should have the value of 0~255. Because of the field itself has 4byte, we can use the field to check Byte Order which the recode itself is obeying. If you find the Recode_Type is out of the range of 0~255, then you need to Byte Swapping to get correct values.

Data

Any data placed right after the Envelope or Record. Basically all data should be written in BIG_ENDIAN, however, there might be some exceptions. Perhaps we can use same trick with Recode.

Text output of the payload for debugging purpose

The payload system has a functionality to write down the payload in ascii format for debugging purpose.
Here you can get the example to trace the EventPayload structure. It would be great help to understand Payload Serialization even if the format is relatively old.

dump out of EventPayload_v1

dump_out of EventPayload_v2 (binary object-out file is here)

Class structure and hierarchy

Payload Interface (eps)

A Payload class (abstract) (eps)

Payload Tree (eps)

Record interfaces (eps)

How an EventPayload is created?

A EventPayload holds whole history of the Trigger and its corresponding ReadOut data. Because the EventPayload is a composite payload, it can hold one trigger information generated by GlobalTrigger and several readouts which contains raw waveforms.
To understand what EventPayload can provide, tracing how each components are created at individual devices or program-modules may be good help. Here I will try to follow the order of creation of each components in EventPayload. It may have serious misunderstandings so that your comments and corrections are appreciated.

This picture is a schematic view of the trigger/data flow. All solid single-lines indicates the flow of trigger informations wrapped by TriggerPayload. Dot lines are ReadoutRequestPayloads which are not contained in EventPayload. Partially-dot lines are ReadoutDataPayload which holds waveform data and may have n : 1 relationship with ReadoutRequestPayload.

TriggerPayload

There is a very nice document about Global Trigger System written by Seon-Hee Seo. See it for detailed informations.

EventPayload has only one TriggerRequestPayload which is generated by Global Trigger. However, because the TriggerRequestPayload is a composite payload, it can hold each history of triggers on lower level triggers: like InIce/IceTop/Amanda trigger etc. or more lower layer like String Processor and IceTop DataHandlers.

Each level of trigger module needs descendants of ITriggerPayload as inputs. For example, String Processor expects to get HitPayloads. Then, the trigger module makes one ReadoutRequestRecord to store all results of its calculation. The ReadoutRequestRecord has a vector to store ReadoutRequestElementRecords: which has a ReadoutType*, trigger time window , SourceID** , and DomID. Last two field may be -1 (which means invalid) depending on the ReadoutType.
* Currently there are six candidates : Global, InIce Global, IceTop Global, InIce String, InIce Module and the IceTop Module.
**Sometimes SourceID is a confusing field because it is not clear that what the "Source" indicates. In this case it indicates which module will have the waveform data required by the ReadoutRequestEventRecord. Actually, it will be one of the String Processors or IceTop Data Handlers.

This ReadoutRequestRecord is stored inside the TriggerRequestRecord together with the informations about the trigger itself. A trigger is specified with three parameters named SourceID, TriggerConfigID, and TriggerType. The SourceID indicates the trigger module itself while others represent trigger conditions: TriggerType indicates a specific trigger algorithm which is actually fired and the TriggerConfigID shows detailed trigger condition at the that time (some trigger algorithm cannot specified only one field of the TriggerType: e.g. TriggerType "TwoCoincidenceTrigger" needs more information, at least what were the coincided triggers).

After that, each trigger module generate one TriggerRequestPayload as its output. Created TriggerRequestRecord in the above process is stored right after the envelope of the TriggerRequestPayload. Then the trigger module stores all input ITreggerPayloads into the composite part of the TriggerRequestPayload. In this way one TriggerRequestPayload has both input and output informations of its creator(trigger)module, then we can trace all history of the lower triggers by looking the composite elements.This TriggerRequestPayload can be the input of more higher level trigger, therefore these processes are repeated each level of the trigger modules and finally the Global Trigger makes only one TriggerRequestPayload as an input of EventBuilder.

ReadoutRequestPayload / ReadoutDataPayload

Once the EventBuiler get the TriggerRequestPayload from Grobal Trigger, it generates ReadoutRequestPayloads using the information inside the TriggerRequestRecord. Since the ReadoutRequestPayload has one ReadoutRequestRecord(which stores vector of ReadoutRequestElementRecord), the EventBuilder decomposes a given ReadoutRequestRecord(inside the TriggerRequestPayload) into several pieces of the ReadoutRequestRecords, then deliver them to the String Processors(SP) or IceTop DataHandlers(ITDH).
Note that it doesn't use any information inside the history of triggers stored in composite vector. This decomposed ReadoutRequestRecords(wrapped by ReadoutRequestPayload) are not saved inside the EventPayload.

When the SP/ITDH gets ReadoutRequestPayload, it knows specific time window and DOMID(if it is specified) to extract waveform data by looking through each ReadoutRequestElementRecords. One ReadoutoutRequestElementRecord can order to get several waveform data from different DOMs: e.g. if it requires whole string data within a specific time window. Practically only one ReadoutRequestPayload is delivered to one SP/ITDH per event as long as the Global Trigger generates only one TriggerRequestPayload, even though PayloadSystem itself doesn't forbid to send several ReadoutRequestPayloads to one specific SP or ITDT within an event.

SP and ITDH have their buffer to hold waveforms during 32sec. If all the time window given by the ReadoutRequestElementRecords are already stored in this buffer, they immediately create one ReadoutDataPayload which stores all required waveforms and send it back to the EventBuilder. Sometimes, however, some of time windows can be out of the range of the stored 32sec. At that time, SP / ITDH creates first ReadoutDataPayload with existing waveform data and wait until remaining waveforms will be stored in the buffer.

ReadoutDataPayload has a field to store the serial number among whole ReadoutDataPayloads created with a same ReadoutRequestPayload. Note that the SP / ITDH cannot know how many ReadoutDataPayloads will be created from one ReadoutRequestPayload until the last ReadoutDataPayload is created. Therefore the ReadoutDataPayload has one flag to notify whether the ReadoutDataPayload is last of the sequence within the bunch of ReadoutDataPayloads or not. Only EventBuilder can know how many ReadoutDataPayloads are created from one ReadoutRequestPayload.

Currently, neither ReadoutRequestPayload nor ReadoutDataPayload send between EventBuilder and SP / ITDH. EventBuilder gets bunch of raw binary waveform data from SPs / ITDHs, then creates ReadoutRequestPayloads and ReadoutDataPayloads inside itself in order to comply the EventPayload format.

EventPayload

After collecting all ReadoutDataPayloads from SPs / ITDHs, EventBuilder generates one EventPayload as its output.
The structure of EventPayload is very simple: 1st composite payload must be the TriggerRequestPayload send from Global Trigger, and followings are collected ReadoutDataPayloads. The number of ReadoutDataPayloads can exceed the nunber of ReadoutRequestPayload for the reason mentioned above.

Following table summarize the order of creation of each components. It contains some future plans which will be mentioned in the next section.

Future Plan

Following picture shows our upgrade plan of Surface DAQ.
Payload System can support plan A completely and will adapts any upgrade after A+ with minimum modifications.
One of the new feature would be data complession like Delta-Complession(see this document), and corresponding new engineering format payload will be supported in near future.

Reference Docs

• Surface DAQ Event Detection System Status (March7,2005, Chuck & Dan's talk)
• Icecube Global Trigger Web Page
• Global Trigger System (draft V2R0)
• Unified Trigger Configuration (Version 1.0)
• Binary Representation of DOM Raw Engineering Event (Version 1.6)
• IceCube Delta Compressor data format and processes (Version 1.0)
• IceCube DOMAPP CPU FPGA Interface V1.11
• Kael's document for delta-compressed payload