fi_rxm - The RxM (RDM over MSG) Utility Provider
The RxM provider (ofi_rxm) is an utility provider that supports RDM endpoint
emulated over MSG endpoint of a core provider.
RxM provider requires the core provider to support the following features:
- MSG endpoints (FI_EP_MSG)
- RMA read/write (FI_RMA)
- FI_OPT_CM_DATA_SIZE of at least 24 bytes
The RxM provider currently supports FI_MSG
: The provider supports only FI_EP_RDM
: The following data transfer interface is
: The RxM provider supports FI_PROGRESS_AUTO
: FI_SOCKADDR, FI_SOCKADDR_IN
: FI_MR_VIRT_ADDR, FI_MR_ALLOCATED, FI_MR_PROV_KEY MR mode
bits would be required from the app in case the core provider requires it.
When using RxM provider, some limitations from the underlying MSG provider could
also show up. Please refer to the corresponding MSG provider man pages to find
about those limitations.
RxM provider does not support the following features:
- op_flags: FI_FENCE.
- Scalable endpoints
- Shared contexts
- Authorization keys
- Application error data buffers
- FI_ADDR_STR, FI_SYNC_ERR
- Reporting unknown source addr data as part of completions
- Triggered operations
When sending large messages, an app doing an sread or waiting on the CQ file
descriptor may not get a completion when reading the CQ after being woken up
from the wait. The app has to do sread or wait on the file descriptor again.
The ofi_rxm provider checks for the following environment variables.
: Defines the transmit buffer size / inject size.
Messages of size less than this would be transmitted via an eager protocol and
those above would be transmitted via a rendezvous protocol. Transmit data
would be copied up to this size (default: ~16k).
: Defines the maximum number of MSG provider
CQ entries (default: 1) that would be read per progress (RxM CQ read).
fabric(7), fi_provider(7), fi_getinfo(3)