Memory Mapped Buffers

[tjablin]

Consider a batch one inference system where input activations are mapped in host DRAM are mapped into the device's address space on demand using an ioctl. The device driver is optimized for the case where a single buffer is reused for many inferences. In order to avoid unnecessary ioctl's, the device driver checks if the buffer is already mapped, skipping the ioctl for the common case of buffer reuse. Is 'caching' pointers for the purpose of avoiding ioctls allowed in MLPerf?

I think the answer is 'Yes.' My reasoning is as follows:

1. In a real application, most users would copy activations into a re-usable buffer and avoid the ioctl.
2. The data still starts in host DRAM.
3. If the shared memory were managed by the user instead, the user could achieve the same affect by loading the dataset into a manually allocated a shared buffer. The driver's 'caching' just accomplishes automatically, what the user could have done manually.

[DilipSequeira]

I agree. This class of system programming optimizations apply to generic (broader than ML) accelerators, and it would be counterproductive to exclude them.