ADM-XRC SDK 2.8.1 User Guide (Linux)
© Copyright 2001-2009 Alpha Data


Caveats of DMA transfers

This section details a few practices regarding DMA that should be avoided or used with care.

DMA write to host memory may not update application buffer until complete

Unaligned DMA transfer to/from non-memorylike local bus region, assuming 32 bit local data bus

Unaligned DMA transfer to/from non-memorylike local bus region, assuming 64 bit local data bus

DMA write to host memory may not update application buffer until complete

When a DMA transfer writes data to host memory, ie. the direction of the transfer is from the local bus to the PCI bus, applications must not rely on being able to see the data as it is written by the PCI device byte-by-byte. This is for two reasons:

  1. On platforms that use bounce-buffers, the PCI device may in fact be targetting bounce buffers rather than the application's buffer.
  2. On some platforms, CPU cache coherency is not maintained during DMA transfers. The CPU's caches may be made coherent at the end of the DMA transfer, but not during the DMA transfer.

In short, an application's buffer is guaranteed to contain valid data only after the DMA transfer has completed (ie. the call to ADMXRC2_DoDMA or ADMXRC2_DoDMAImmediate has returned).

Unaligned DMA transfer to/from non-memorylike local bus region, assuming 32 bit local data bus

A memorylike region on the local bus is defined to be a range of the local bus address space in which reads and writes have no side-effects. The only effect of performing a write within such a range is to update zero or more byte locations (depending on the value of the byte enables, LBE#) with new data.

A non-memorylike region on the local bus is defined to be a range of the local bus address space where reads and writes have "side-effects". For example, an FPGA design may implement a FIFO whose read and/or write ports are mapped to a particular local bus addresses. Reading or writing these ports causes the FIFO's internal state to change, which is considered to be a side-effect.

When performing DMA transfers to non-memorylike regions, unaligned DMA transfers should be used with great care. An unaligned DMA transfer is one where the host memory buffer for the DMA transfer does not begin at an aligned address. If a 32 bit wide local bus is being used, then an aligned address is one whose lower two bits are zero. If a 64-bit wide local bus is being used, then an aligned address is one whose lower three bits are zero.

First, consider the following DWORD-aligned DMA transfer, assuming that the local bus has 32 bit wide data:

In this case, the data will be transferred correctly; The 7 DWORDs in the user-space buffer resulted in 7 DWORDs written into the FIFO.

Now consider the following unaligned DMA transfer, again assuming that the local bus has 32 bit wide data:

The required 28 bytes are written into the FIFO, but 8 rather than 7 DWORDs in total are written into the FIFO. Two of those DWORDs have partial byte enables, and this may represent a problem. There are a couple of ways in which to address this issue:

  1. Ensure that DMA transfers are performed using buffers that are aligned to a DWORD (4 byte) boundary. This may not be possible; for instrance, if the application does not control how memory is allocated.
  2. A better solution, assuming that the length of block of data in a DMA transfer is always a multiple of 4 bytes, is to use LBE#[3] (see the LBE# local bus signal) to qualify actually committing the data to the FIFO. When a word is written to the FPGA with LBE#[3] deasserted, the FPGA latches the data for those byte enables that are asserted, but does not yet commit the data to the FIFO. Eventually, the DMA transfer will pick up where it left off and assert LBE#[3] along with the data when it begins the next block in the linked-list DMA transfer. At this point, the FPGA commits the completed word to the FIFO. This does not result in any restrictions on the alignment of buffers in host memory.

Unaligned DMA transfer to/from non-memorylike local bus region, assuming 64 bit local data bus

If the local bus has 64 bit wide data, then an aligned host memory buffer is one that begins at an address whose lower 3 bits are zero. By a similar process of reasoning to the 32 bit case above, the issues related to unaligned DMA transfers can be addressed in the following ways:

  1. Ensure that DMA transfers are performed using buffers that are aligned to a QWORD (8 byte) boundary. This may not be possible; for instance if the application does not control how memory is allocated.
  2. A better solution, assuming that the length of block of data in a DMA transfer is always a multiple of 8 bytes, is to use LBE#[7] (see the LBE# local bus signal) to qualify actually committing the data to the FIFO. When a word is written to the FPGA with LBE#[7] deasserted, the FPGA latches the data for those byte enables that are asserted, but does not yet commit the data to the FIFO. Eventually, the DMA transfer will pick up where it left off and assert LBE#[7] along with the data when it begins the next block in the linked-list DMA transfer. At this point, the FPGA commits the completed word to the FIFO. This does not result in any restrictions on the alignment of buffers in host memory.

 


 Top of page