Achieving true zero-copy with Iceoryx (Shared Memory)

[fabiannagel]

Hi everyone, I've been trying to use this library with CycloneDDS and Iceoryx. I built both libraries from source and can verify everything works from examples & benchmarks.

However, I'm struggling to achieve true zero-copy for large messages (raw images; ~5 MiB/image, 24 Hz). This is my sample IDL:

module TestMessage {
    struct CameraImage {
        unsigned long width;
        unsigned long height;
        sequence<octet> image_data;
    };
};

Given a numpy array, I can use arr.to_bytes() to publish the message. However, it takes ~350 ms to do so:

msg_instance = CameraImage_type(
     width=arr.shape[1],
     height=arr.shape[0],
     image_data=arr.to_bytes()
)
self.writer.write(msg_instance)

Using np.ascontiguousarray() and memoryview, I can get it down to ~0.1 ms:

contiguous_arr = np.ascontiguousarray(image_np)
c_buffer = memoryview(contiguous_arr.data)

msg_instance = CameraImage_type(
    width=contiguous_arr.shape[1],
    height=contiguous_arr.shape[0],
    image_data=c_buffer
)
self.writer.write(msg_instance)

BUT it quickly fails (~30 calls) with this error:

Failed to encode member image_data, value is <memory at 0x72118cf35990>

I was guessing that Python garbage collection is not in tune with CycloneDDS & Iceoryx. So I tried keeping explicit references to the last 1000 references to both contiguous_arr and c_buffer in a collection.deque() - without success.

Is there another way of making this work?

Best
Fabian

[eboasson]

There are two obstacles to doing zero-copy that come to mind. The first is that there is quite a bit of stuff between Python and the shared memory. It is probably best to look at how it works when done in C because that is a simpler case:

1. the writing application allocating shared memory for the data, using dds_request_loan
2. then it constructs the sample in the shared memory
3. it passes the shared memory address to dds_write, which recognises it as a loan, and publishes it in Iceoryx without serializing it (it still has to serialize it if it (also) has to go onto the network)
4. the reader application gets a pointer to the same shared memory from Iceoryx, stores a sample in the reader history cache that points to the shared memory
5. when dds_takeing the sample in "loaning" mode, the pointer to the shared memory is handed to the application
6. the receiving application does whatever with the data
7. ... and calls dds_return_loan to release it

I do think you could do the same thing with Python and memory views, but it will take some work to extend the Python binding for this:

• the API functions need to be mapped
• you'd need to represent the content of the samples in memory in a fixed layout (e.g., C struct layout) and do that via a memory view or something
• because those things haven't been done, it didn't seem worth the bother to implement support for this in Python type support code

I don't think it'd be very difficult — the only bit that I really know my way around is the third one, but if you know how to do the Python side we could probably work it out.

Then there's another obstacle. Your type:

module TestMessage {
    struct CameraImage {
        unsigned long width;
        unsigned long height;
        sequence<octet> image_data;
    };
};

Contains a sequence, but a sequence is not a fixed memory layout. You would have to map it to a sufficiently large array of octets instead.

If the Python type support code were to be extended to support shared memory, then you still wouldn't have zero-copy because the type doesn't allow it to be placed in shared memory. It would allow you to avoid the network stack, and that does help quite a bit.