Data Transfer

Data is transfered using a shared memory area between the sender and the receiver. DROPS memory regions are used to administrate this memory. In most cases the shared memory region is created on connection setup and exists until the connection is closed. In some cases it is necessary to use dynamic mapping to transfer the data. See the Discussion on the Pros and Cons of Dynamic Mapping. The memory backing the shared region can be of any type. This includes normal on-board memory, but also memory on adapter cards like a video card or a framegrabber. This allows a component to offer data from hardware devices without copying it in a transparent way to the clients of this component. This works also for the receiving direction, if the sender of the data respects the layout the receiving hardware device may impose.

Once the shared memory is established, it is in the responsibility of the sender to choose appropriate pieces of this region for data transmission. The sender selects a piece in the shared memory, stores the data in it and uses the DSI API to communicate the location of the data to the receiver.

DSI uses packets containing a scatter/gather-list to refer to the data in the shared memory. There are no restrictions on the size and alignment of the scatter/gather-elements by DSI. But hardware components may add these restrictions, and the corresponding components must signal this separately to the communication peers. Currently, DSI does not support this signalling of alignment or size restrictions.

The references to data in the scatter/gather-lists are stored relative to the beginning to the data region. This way the shared data region can be mapped at different addresses into the senders and the receivers address space.

Todo:

Add signalling of alignment and size restrictions to DSI.

To get the address and size of the shared memory region (or the region intended for data transfer in the case of dynamic mapping) use dsi_socket_get_data_area(). To add a piece of memory to the scatter/gather-list of a packet use dsi_packet_add_data(). The receiver uses dsi_packet_get_data() to get the data fragments from the scatter/gather-list.

Information about getting and sending packets can be found in the Synchronisation section.

The following discussion presents the pros and cons of dynamic mapping as a technique to enforce correct synchronisation in nonblocking mode.

In nonblocking mode, there are no IPCs used as synchronisation points, the components must synchronise using timers and variables. One suggestion was to use dynamic mapping and unmapping in the data region to enforce correct synchronisation. The idea behind is, that the sender component builds a window of valid data by mapping only the parts, that it likes to be accessible at that time. If the receiver tries to access data which is out of this window, it would receive a signal generated by a pagefault-ipc. The advantage of this proposal is the implicit signalling mechanism, not disturbing in normal operation.

A problem of the solution is that it does not ensure to detect malfunction. The memory region is limited in size, a wraparound is needed. If the difference between sender and receiver is large enough, it wont be detected. This means, other synchronisation mechanisms are still needed, nothinging the advantage of easy signalling.

The overhead of the mapping ipcs related to every chunk of data is a real downside. Every data transport will require a mapping IPC together with a relatively costly unmapping syscall.

An want-to-be argument is that the sender can manage its ressources more flexible with this architecture. It is weak for the following reasons: The ammount of memory the sender needs is precalculated by jitter and bandwidth and does not change. So the sender has no reason to reclaim memory during operation to lower the amount of memory for the connection. If a sender wants to exchange the physical pages in the memory region, dynamic mapping is needed. But this can be done without producing holes in the shared area and has nothing to do with synchronisation.

To sum it up, the dynamic mapping as the principal technique for signalling has no advantages, but introduces some overhead. Thatswhy it is not used in DSI. Mapping of particular pages offers additional functionality for shuffling physical pages at the sender and is therefore included in DSI.