E852 Documentation: The Event Dispatcher

Introduction

The E852 Event Dispatcher is a robust and versatile system for distributed data analysis.

Terminology

The basic units of data handled by the dispatcher are raw data events and analyzed (processed) data events. The Dispatcher receives raw data events from a data source (a tape reading program, a data acquisition program, a monte carlo generator, etc...), sends them to multiple identical analyzers (process clients), receives analyzed data back and sends it to various display clients.

Data flow graph

How to specify a Dispatcher connection

The Dispatcher name is hostname:port, where hostname is the host name of the machine running the Dispatcher and port is the Dispatcher TCP port.

Currently allocated TCP ports are:

10002 - E881 offline analysis
10099 - E852 offline analysis
10098 - E852 online system

The Dispatcher client library

The Dispatcher, as a state machine

The Dispatcher has a number of internal states:

• NoTape - no data is being processed
• RequestingTape - waiting for 'BEGINTAPE' from the data source.
• ProcessingTape - data is being processed.
• FinishingTape - the data source sent an 'ENDTAPE' but there is still buffered non-analyzed data.
• EndingTape - data processing is finished, the Dispatcher is waiting for the process clients to 'FINISH'.

The Dispatcher changes his states in responses to various commands sent to him. The figure below and a more verbose discussion of the Dispatcher-analyzer protocol after it document the behaviour of the Dispatcher.

The Dispatcher - analyzer protocol

Before data can be analysed, the Data source, Dispatcher and analysers have to agree on what they are doing. To acheive agreement, they send each other various messages (called commands) using the disIO_command() function.

The expected sequence of events follows:

1. the Dispatcher starts up and waits.
2. the analyzers start up, connect to the Dispatcher, send a REQUEST_DATA command and wait.
3. the Dispatcher receives the REQUEST_DATA messages and registers their senders as process clients
4. the data source starts up and sends the BEGINTAPE command.
5. the Dispatcher receives BEGINTAPE and sends BEGINTAPE:nnn commands to the process clients, where nnn is the 'client number'.
6. analyzers receive BEGINTAPE:nnn and perform any necessary initialization. The client number nnn allows (for example) one of the analysers to identify himself as a 'master analyser'.
7. after sending BEGINTAPE, the data source starts sending raw data.
8. the Dispatcher receives raw data and dispatches it to process clients that have outstanding requests for raw data.
9. when the data source has no more raw data to send (end of tape, end of file, etc...), it sends the ENDTAPE command.
10. when the Dispatcher receives the ENDTAPE, it waits until the raw data buffered inside the Dispatcher is analysed and sends ENDTAPE messages to all the process clients.
11. when the analyzers receive ENDTAPE commands from the Dispatcher, they perform any necessary cleanup/shutdown work and send back the FINISHED command.
12. the Dispatcher waits for all the process clients reply with FINISHED. After all the process clients reply, the Dispather sends them the QUIT command and disconnects them. At this point the data analysis cycle is finished. If more data is to be analysed, the analysers might reconnect to the Dispacther, a new data source can be started and everything starts anew.

The Dispatcher Commands

General commands (send by clients to the Dispatcher)

• BROADCAST:dest:command - the analyzer sends the command to clients specified by dest, which can be one of-...
• PWD - print current working directory (getcwd())
• DIRECTORY:itapedir set the itape directory (default is './itape')
• DIRECTORY print the value of the itape directory
• HOST:str set the value of the Dispatcher 'HostComputer' variable to 'str'
• DEBUG:0 turn debug mode 0 on.
• DEBUG:1 turn debug mode 1 on.
• NODEBUG:0 turn debug mode 0 off.
• NODEBUG:1 turn debug mode 1 off.
• CLOSE:DATA close the data source connection
• CLOSE:socket close the given socket
• DROP:socket close the given socket
• QUIT close the sender's socket
• QUIT_DISPLAY close the sender's socket
• DUMP print the Dispatcher's internal tables, states, queues, buffers, etc...
• NOP ignored
• STATISTICS print a number of accumulated statistics

Handshaking with process clients

• login - [A->DIS] the sender identifies himself as a process client
• BEGINTAPE:clientNo:runNo - [DIS->A] sent by the Dispatcher when he receives a 'BEGINTAPE' from the data source.
• REQUEST_DATA - [A->DIS] request raw data.
• ENDTAPE - [DIS->A] sent by the Dispatcher when he receives an 'ENDTAPE' from the data source.
• FINISHED - [A->DIS] sent to the Dispatcher in reply to the 'ENDTAPE' message
• QUIT - [DIS->A] sent by the Dispatcher when he wants the analyzer to terminate.

Handshaking with the data source

• BEGINTAPE[:runNo] - [R->DIS]
• ENDTAPE - [R->DIS]

Handshaking with display clients

• Display:triggerMask:latchMask - [D->DIS] the sender is registered as a display client interested in events with triggers specified by 'triggerMask' and latch value specified by 'latchMask'. If either mask is absent, it is taken to be '0xffffffff'.
• REQUEST_DISPLAY - [D->DIS] request analyzed data.

Special cases

The default behaviour of the Dispatcher is data neutral. This means that he does not try to interpret the data being dispatched.

When given the -e852 switch, some additional, experiment dependant processing is allowed. Currently, only special treatment of begin-of-spill (BOS) and end-of-spill (EOS) events is involved, by more can be added later.

Special treatment of BOS/EOS events

The structure of the E852 data is such that some vital information (such as beam line (Dibbuk) or MPS magnet probe data) is recorded only once per spill in the BOS and EOS events. Therefore these events are to be passed to every analyzer, unlike regular events, which sent to one and only one analyzer.

//CO 17 Apr 1994