Linux 2.6.x driver notes for Tahoma Technology's PCI DR11-W interface Model 10118. Tahoma Technology (formerly IKON Corporation) 107 2nd Avenue, Seattle, WA, USA 98109 206.728.6465 http://www.tahomatech.com tahoma@tahomatech.com 5 October, 2005 updates to reflect x86_64 compatibility and support for 10116 NOTE: The driver and install script now support both 10116 and 10118 boards. This document is written assuming a 10118, but it also applies to the 10116. Board types may be mixed in a system. This document contains a brief set of notes on the calling sequences used with the driver for the 10118 DR11 emulator from Tahoma Technology (formerly Ikon Corporation). It will describe the general purpose of each ioctl() call, and attempt to give suggestions as to how they might be used. It will also describe the compile-time configuration defaults and choices available. Since the DR11 is a very general-purpose board, it is impossible to give examples of all possible uses of the board and driver. The user is referred to the hardware manual provided with the 10118. It contains register descriptions, and some discussion of how DMA transfers and interrupts are used. NOTE: Many compile-time defaults can be overridden by the insmod loader at driver install time. This is most easily accomplished by editing the idr_install-2.6 script before installing the driver. The driver's header files: idr_io.h, contains the #define statements for the arguments to be used in ioctl() calls to the 10118 driver. It must be included in the code of the program calling the driver. (#include "idr_io.h") This will also cause the idr_reg.h file to be included. idr_reg contains the register bit definitions for the pci DR11 which may be used as arguments in some of the ioctls. idr_io.h also includes register bit definitions that match the Sbus DR11, in order to preserve compatibility with existing drivers. The idr_var.h and idr_util headers are used internally by the driver. The c programs, link.master.c, and link.slave.c, which are included on the driver diskette, are very simple examples of how the 10118 might be used. Also included is link.master.manual, which is an example of operation in manual mode. Manual mode is useful for continuous data collection, since it allows the DR11's fifos to continue to run between unix read or write calls, and allows multiple read/write calls to be used to satisfy the DR11 block range count. NOTE: The headers included with the Linux, Solaris 1.x (SunOS), and Solaris 2.x drivers may not be intermixed. They will produce the same ioctl command codes when used with the appropriate compiler and source. They will not compile correclty if used with the wrong source or compiler. (That is, you can't compile idrio.h with a Solaris 2.x driver or application, you must use idr_io.h supplied with the Solaris 2.x driver, and likewise for idr_io.h supplied with the Linux driver). A call to the 10118 driver looks like: ioctl(filedes,cmd,arg) ; Filedes is a filedescriptor returned by the open() call. Cmd is one of the pre-defined commands from idr_io.h. Arg (when used as an input to the driver) is a pointer to a variable containing one, or a combination, of the arguments defined in idr_io.h. Arg is typically defined as an array of unsigned integers (32 bit), or a pointer to an unsigned integer (32 bit). The variable pointed to by arg should be 32 bits wide for both 32 and 64 bit systems. #including idr_io.h will pick up linux/types.h wich defined __u32. An code fragment that opens the 10118 named idr0, and then sets the dma timeout might look like (without any error checking): #define DMATIMEOUT 120 /* 120 seconds */ ... int filedes ; __u32 arg; ... filedes = open("/dev/idr0",O_RDWR) ; ... arg = DMATIMEOUT ; ioctl(filedes, IDRIO_SET_DMA_TIME, &arg) ; All of the ioctl() calls conform to this general example. What follows is a description of each command, and the possible arguments used with that command. In the argument descriptions for each command, including a particular argument in arg is equivalent to setting it to one. Omitting it is equivalent to setting it to zero. IDRIO_SET_MODE Sets the operating mode of the board and driver. This is generally used once - if at all - at the beginning of a program to configure the board and driver's "static" parameters, which will not change during this use of the board. This ioctl corresponds to the mode register on the 10103 Sbus DR11 emulator. The pci board and driver will set bits appropriately to duplicate these functions. IDR_RDISX Disables the DR11 range counter. May be useful for continuous data collection applications. IDR_RDYT If 0, READY goes true during BUSY. If 1, READY goes true at BUSY trailing edge. READY true during BUSY is the normal mode for DR11-W applications. IDR_FMOD Not used on pci board. IDR_BDIS A one here disables the CYCLE RQ B input. Used if for some reason the device connected to the 10118 floats this input. IDR_SWAP A one causes DMA bytes to and from the attached device to be swapped within a 16 bit DR11 word. Does not affect the 10118 registers, only device data. THIS FUNCTION IS NOT IMPLEMENTED IN EARLY VERSIONS OF THE 10116 BOARD, WHICH ALWAYS SWAPS DMA DATA. (This is due to the way the sparc processor and memory are connected to the pci bus). Later versions of the '116 board - eeprom revision x.2 and later, and all 10118 boards implement byte swapping for dma data. Their default is no swap. IDR_CRQP 0 causes the CYCLE RQ A & B inputs to be rising edge active. 1 selects falling edge active. Rising edge active is normal for DR11-W applications. IDR_BSYP 0 selects the BUSY output to be asserted low, 1 selects asserted high. Asserted low is normal for interprocessor links to another DR11. An attached device may require either high or low assertions, depending on how it is designed. IDR_SPEED_0, IDR_SPEED_1, IDR_SPEED_2, IDR_SPEED_3 One of these arguments may be used to select the speed of the DR11 handshake with the attached device. This choice affects the input data de-skew, the output data settling time, and the length of BUSY. Speed 0 is the fastest selection. See the 10116/10118 manual. A call to set the mode to byte swapping, and the slowest speed would be: arg[0] = IDR_SWAP | IDR_SPEED_3 ; ioctl(filedes,IDRIO_SET_MODE,arg) ; IDRIO_IMM_FCN This call sets the outgoing function bits to the requested value immediately. The desired function bit combinations are very application dependent, although they usually have a fixed meaning in interprocessor links. In links, FCN1 on means that the local DR11 is to be the receiver of the transfer, FCN3 is always set to 1 during the DMA block transfer, and FCN2 can be used to pulse the other DR11's ATTENTION input (the ACF2 pulse can be used to do this automatically, which avoids having to use this comand once to set FCN2 and again to reset it). IDR_FCN3, IDR_FCN2, IDR_FCN1 These arguments may be or'd together. IDRIO_READ_FCN, IDRIO_WRITE_FCN These calls pre-define the function bit patterns to be set at the start of a read or write operation. They have no effect until the read or write is actually issued. It is only necessary to make one or both of these calls once if at all. After being initially set, they will apply to all later reads and/or writes. The arguments used are the same as for IDRIO_IMM_FCN. There are pre-defined defaults for the function bits that may be changed by editing the idr_io.h file. See a later section of these notes for further information. IDRIO_IMM_PULSE This call issues certain pulses to the attached device and to the 10118 logic. Some of the available pulses affect the internal operation of the 10118, and should be used with caution! The internal pulses are IDR_RATN, _TERM, and _MCLR. Most of these pulses will be set using the read and write deferred pulse calls. The IDR_ACF2 pulse is the only one likely to be used as an immediate pulse, unless operating in MANUAL mode. The arguments to this command may be or'd together. IDR_REOR Not supported by the pci driver IDR_RATN Clears the internal ATTENTION flag. IDR_TERM Sets READY true. Can be used to terminate a DR11 block transfer that was started in MANUAL mode. NOT RECOMMENDED!!! Not necessary when operating in AUTO mode. (See later for descriptions of these modes). IDR_MCLR Clears all 10118 idr (not PLX) logic. Use with extreme caution!!!! Better yet, don't use it at all!!!!!!! IDR_INIT Sends an INITAILIZE pulse to the attached device. Has no effect on the 10116/10118. IDR_ACF2 Pulses the ACLO_FNCT2 output. In interprocessor links, this will cause an ATTENTION pulse at the other end of the link. Typically used for preliminary handshaking and synchronization prior to transferring a DMA block. IDR_CYCL Forces an immediate cycle request. This simulates a cycle request from the attached device. It is normally used to start a link-mode output transfer. It is generally NOT used to start input in any application. There are some non-link applications that will use it to start output transfers. These are situations where the attached device does not issue the first cycle request at the beginnig of a transfer, but does issue all subsequent requests. IDR_GO Issues a GO pulse to the attached device. The GO indicates that DMA block transfers may begin. IDRIO_READ_PULSE, IDRIO_WRITE_PULSE These calls set the pulse commands to be issued at the start of a read or write. They need to be set only once. There are pre-defined defaults for the read and write pulses. Typically, (IDR_GO | IDR_CYCLE) would be used at the beginning of a link mode write. (IDR_GO | IDR_ACF2) would be used at the start of a link mode read. GO enables the block transfer, CYCLE forces the first word of the transfer for output transfers, and ACF2 indicates to the other end of the link that this board is ready to receive data for input transfers. IDRIO_SET_DMA_TIME, IDRIO_SET_ATTN_TIME, IDRIO_SET_RDY_TIME These calls set the timeout values (in seconds) used during a wait for the end of a DMA block transfer, wait for ATTENTION, and wait for READY to go true. Currently, the pre-defined defaults are 30 seconds for each of these parameters. In addition, the ioctl() code will not let the calling program set the timeout(s) to less than 10 seconds, or more than 10 minutes. This is left over from an earlier version of the driver that couldn't receive signals while waiting. The defaults, and/or the limits, can be changed by editing the idr_io.h file. IDRIO_ATTN_WAIT, IDRIO_RDY_WAIT These calls can be used to wait (by sleeping if necessary) for ATTENTION, or READY, respectively. The sleeps are subject to the above timeout values. ATTN_WAIT is typically used when waiting for an attention pulse from the attached device, or other DR11. In an interprocessor link, ATTENTION is often used to synchronize the start of a DMA block transfer. See interprocessor links in the 10116/10118 manual. RDY_WAIT is usually used in MANUAL mode. If multiple unix reads or writes are used to satisfy a larger DR11 range count, this wait can be used to test for DR11 end of range - or termination of a block transfer by ATTENTION (which sets READY). IDRIO_GET_STATUS This call can be used to determine board and driver status. It may be useful when polling for certain device or board conditions. Any or all of these conditions/flags may be true at the same time. These bits match those of the 10103 Sbus DR11 emulator, they are gathered from various registers in the 10118 and presented here for compatibility. IDR_EORF This flag sets when the DR11 range counter underflows. IDR_ATTF Sets at the rising edge of an incoming ATTENTION pulse. IDR_ATTN This flag follows the state of the ATTENTION input. Note that GO has no effect on the 10118 if ATTENTION is true when GO is issued. This bit can be used to make sure that ATTENTION is off prior to issuing GO in MANUAL mode. In AUTO mode, this is taken care of by the driver code. IDR_MCER This is an error flag that sets if the attached device requests a transfer while the 10118 is busy processing the previous transfer request. (CYCLE RQ while BUSY is true). In normal transfers that use a full handshake - such as interprocessor link transfers - this bit will never set. It is useful for applications that must use a continuous input clock to drive CYCLE RQ. In this case, the 10118 must always be fast enough to process each request in the time between clocks, and more important, must always have room in its fifo. The 10118 is extremely fast. However, in MANUAL mode, the fifo will continue to fill in the gap between reads. In this case, it is possible to overflow the fifo if the system takes too long to start the next read. The MCER flag will alert the user to this error if tested after each unix read call. This flag does not cause an interrupt or affect the transfer in any way. It is a status flag only. IDR_STTA, IDR_STTB, IDR_STTC These follow the status A, B, and C inputs from the attached device. They have no effect on the 10116/10118 logic. IDR_REDY This is the master READY bit in the 10118. It goes to 0 when GO is issued, and set to 1 when the DR11 range count is exhausted, ATTENTION is received, or TERM or MCLR is pulsed. IDRIO_GET_RANGE This call returns the DR11 range counter residual value. It is returned as the 24 least significant bits of an unsigned integer. It is equal to the remaining transfer count MINUS ONE. At the end of a complete block, the value returned will be 0x00FFFFFF. This call may be useful when doing an input of a word count less than the requested block size. Some applications may involve a variable block size. The range count may be set to the maximum allowed - or desired - and read after the transfer to determine the number of words actaully received. It may be necessary to use this count to determine how many unix read calls are necessary to input to memory the data actually received. IDRIO_GET_REGS (requires array as argument) This ioctl command returns 16 of the board's registers, and the device IDs and revision level in an arg array of 32 bit unsigned integers. The PLX registers are 32 bits wide. The DR11 registers are 8 bits wide except the data in register which is 16 bits wide. See the 10116/10118 manual for complete register descriptions. The organization of the returned array is: arg[0] device ID (high 16 bits) & vendor ID (low 16 bits) arg[1] revision ID arg[2] PLX interrupt control/status register arg[3] PLX EEPROM control and user bits register arg[4] PLX DMA mode register arg[5] PLX DMA PCI address arg[6] PLX DMA local address register arg[7] PLX DMA transfer count register arg[8] PLX DMA descriptor pointer register arg[9] PLX DMA command/status register arg[10] latched functions register arg[11] flags register arg[12] status register arg[13] mode register arg[14] DR11 range register - low byte arg[15] DR11 range register - mid byte arg[16] DR11 range register - high byte arg[17] fifo status register arg[18] data in register IDRIO_GET_FLAGS Returns various driver software flags and error bits. This is primarily diagnostic. IDR_DVMA_WAIT waiting for DVMA terminal count IDR_EOR_WAIT waiting for DR11 end of range IDR_ATTN_WAIT waiting for ATTENTION IDR_RDY_WAIT waiting for READY IDR_DVMA_TIMEOUT DVMA wait timeout IDR_EOR_TIMEOUT EOR wait timeout IDR_ATTN_TIMEOUT ATTENTION wait timeout IDR_RDY_TIMEOUT READY wait timeout IDR_MANUAL manual mode IDR_INPUT input mode IDR_MCYL_ERR multicycle error IDR_SIG_RECEIVED signal received IDR_PAR_ERR DR11 parity error - new for pci (only works with DR11 parity compatible boards) IDRIO_DATA_OUT Sends a 16 bit value to the attached device. This is a programmed i/o output, and does not use any of the DVMA handshake signals (GO, BUSY, etc). ACF2 and ATTENTION pulses are often used for software synchronization of p-i/o data transfer. The data sent can be anything useful to the user. For instance, it could be the requested transfer count in an interprocessor link, or command bits to a display or data collection device. IDRIO_DATA_IN Reads the 16 bit value from the DR11 input data lines. IDRIO_SET_RANGE Sets the DR11 range counter to an initial value. This value is must be the desired WORD count MINUS 1. This call is only used in MANUAL mode. The maximum value allowed is 0xFFFFFF, which gives a maximum DR11 block size of 32Mbytes. IDRIO_AUTO, IDRIO_MANUAL These calls determine the overall operating mode of the 10118 driver. In AUTO mode, each call to the idr_strategy routine (made via unix read or write calls), is treated as a complete transfer. The DR11 and DMA range couters are initialized to equivalent counts, and the appropriate read or write pulses and function bits are set. The driver sleeps until the counts are exhausted, or an ATTENTION is received. If read or write is called with a block size to big for the DMA copy buffer, multiple DMA and copy operations will be made invisibly to the user. AUTO mode operation pulses GO and sets the read/write function bits and sets the DR11 range counter only once per read or write call, not per each DMA block. AUTO mode is the default. NOTE: With Tahoma's Solaris driver, a user buffer too large to be handled by a single DMA transfer will also be broken into multiple transfers. However, GO, read/write function bits, and other pulses will be issued with each DMA transfer, not just once per read or write call. The DR11 range counter is also set with each DMA transfer in the Solaris driver. The difference is dictated by the maximum size of the DMA transfer allowed by Linux (limited by user DMA mapping/copy boffer capacity) and Solaris (limited by mapping resources - since DMA is done direcly to/from the users buffer). With current hardware (PC and SPARC), mapping resources are probably not an issue, and the allowed max DMA size can be quite large. MANUAL mode may be selected for cases when multiple read or write calls will be used explicitly to satisfy a larger DR11 block size - or for the case of continuous data transfers where data must be transferred to or from disk without terminating the overall DR11 block. In MANUAL mode, unix read or write calls only communicate with the 10118's DVMA logic. The user must set the DR11 range counter, issue all necessary function bit changes and pulses, and wait for DR11 block completion - if desired. The calling program must also use the START_READ and START_WRITE calls prior to issuing GO to start the overall block, and prior to issuing the first unix read or write call within the block. The BLOCK_END call must be used after all DR11 transfers are complete, typically after a wait for READY (for output) or after the last read (for input). IDRIO_START_READ, IDRIO_START_WRTITE, IDRIO_BLOCK_END See the description of MANUAL mode, above. IDRIO_DEV_AND_VEND_ID Returns the pci ids for the 10118 board(s). The vendor id is the low 16 bits, the device id is the high 16 bits. IDRIO_REVISION_ID Returns the revision of the board's eeprom - which indicates the board's features. This is not the same as the artwork revision of the board, which is indicated in the board's silk screen legend. IDRIO_SET_NEW_MODE This ioctl allows direct access to the board's mode register (similar to the Sbus board's mode register). See the IDRIO_SET_MODE ioctl above for descriptions of these functions. Use the NEW_MODE definitions with this ioctl, and the OLD_MODE definitions with IDRIO_SET_MODE. NEW_MODE OLD_MODE LONG_READY same as IDR_RDYT DISABLE_RANGE same as IDR_RDISX DISABLE_CRQ_B same as IDR_BDIS CYCLE_POLARITY same as IDR_CRQP BUSY_POLARITY same as IDR_BSYP SPEED_0,1,2,3 same as IDR_SPEED_0,1,2,3 IDRIO_GET_NEW_STATUS Returns the pci DR11 emulator's status register. This register is similar to the Sbus DR11 emulator's status register. See IDRIO_GET_STATUS ioctl above for descriptions of these functions. Use the NEW_STATUS definitions with this ioctl, and the OLD_STATUS definitions with IDRIO_GET_STATUS. NEW_STATUS OLD_STATUS READY same as IDR_READY ATTENTION same as IDR_ATTN STATUS_A,B,C same as IDR_STTA,B,C A00_INPUT reports the state of the A00 input (driven by the other end's READY output in DR11-DR11 interprocessor link) IDRIO_GET_NEW_FLAGS Returns the state of the hardware flags register (not the unit_flags driver state word). END_OF_RANGE_FLAG set when DR11 counter underflows ATTENTION_FLAG set by false to true transition on ATTENTION input PARITY_ERROR_FLAG set by DR11 input data parity error - only valid if connected to another device that supports DR11 cable parity MULTI_CYCLE_ERROR_FLAG set if attached device issues a cycle request while the board is still busy processing the previous cycle request - indicates data overrun/underrun in applications where the device does not use a full cycle request/busy handshake IDRIO_MASTER_CLEAR This ioctl does a full reset of the board, including a reset of the PLX pci interface chip and an eeprom reload. This should be used for extreme emergencies only (or perhaps when initially debugging an application)!! DRIVER DEFAULT PARAMETERS There are several default driver parameters that may be overridden by insmod when the driver is loaded. They may be set by manually loading the driver and specifying each parameter to be overridden, or by editing the idr_install-2.6 script and using that script to load the driver. They may also be modified at run time by using the above ioctl() calls. See idr_install-2.6 and README.idr-2.6 for additinal information.