IKON Corporation 2617 Western Ave Seattle, WA 98121 (206) 728-6465 7 June, 2000 ************************************************* * * * Solaris 7/8 compatible * * * * x86 version * * * ************************************************* Driver installation instructions and notes for IKON Corporation's Model 10118 PCI DR11-W emulator. THE x86 VERSION OF THIS DRIVER SUPPORTS ONLY THE 1/2 SIZE 10118, WHICH SUPPORTS DMA CHAINING FROM PCI MEMORY. THE MODEL 10116, WITH THE PLX 9060 CHIP INSTALLED, IS NOT SUPPORTED. AT SUCH TIME AS THE 10116 BEGINS SHIPPING WITH THE PLX 9080 INSTALLED, IT WILL ALSO FUNCTION CORRECTLY WITH THE x86 DRIVER. Files provided on the distribution diskette include: idr loadable PCI DR11-W driver module idr.debug loadable PCI DR11-W driver module with debug prints enabled idr.conf PCI DR11-W driver configuration file idr_driver.c code source module for the Solaris 2.x PCI DR11-W driver idr_var.h structure definition file for the PCI DR11-W driver idr_reg.h register definition file for the PCI DR11-W driver idr_io.h ioctl command definition file for the PCI DR11-W driver - to be #include'd' in the calling program idr_driver.make make file for Solaris 2.x PCI DR11-W driver idr_driver.make.debug makes idr.debug idr_driver.lint.ex example lint file from good compilation link.master.c very simple interprocessor link program (master end of link) that calls the DR11-W driver link.master link.master executable link.slave.c simple interprocessor link program (slave end of link) link.slave link.slave executable link.master.manual.c version of link master that demonstrates use of MANUAL mode link.master.manual link.master.manual executable idr.loop.test.c loop-back test source for PCI DR11-W board idr.loop.test loop-back test executable 118notes information on using the PCI DR11-W driver After a successful pkgadd, these files will be found in /opt/IKONidr (assuming that the default base directory - /opt - has been used). idr and idr.conf will also have been copied to /usr/kernel/drv, /etc/devlink.tab will be modified, and add_drv will have been run. It should not be necessary to perform any of the following manual installation procedures unless the driver source has been modified by the customer, or it is necessary to load the idr.debug version of the driver. To compile driver: make -f idr_driver.make The make available on IKON's machine is in /usr/ccs/bin, so: /usr/ccs/bin/make -f idr_driver.make The compiler called in the makefile is the SparcWorks compiler. After running make, check the idr_driver.lint file for warnings and errors. The file idr_driver.lint.ex contains a lint file from a successful compilation. The result of a successful make is the loadable module: idr. NOTE: It is not necessary to make the driver unless it has been modified. Install and "un-install" require superuser privileges. Either log in as root, or execute the su command. The following operations should be executed from the directory containing the files provided on the IKON diskette. To install loadable driver: Copy the driver module and conf file to the driver directory: cp idr /usr/kernel/drv cp idr.conf /usr/kernel/drv If it is desired to run with driver debugging enabled, copy the debug version of the driver to the driver directory instead of the normal version: cp idr.debug /usr/kernel/drv/idr Edit the /etc/devlink.tab file to add the idr device. Add the following line(s) to the file: type=ddi_pseudo;name=pci11d5,116 \M0 type=ddi_pseudo;name=pci11d5,118 \M0 (the white space before \M0 is a tab - the 0 is a zero) It is not necessary to add all of the above lines to devlink.tab, only the line(s) that refer the the board(s) actually being installed. It is harmless to add both lines, and allows for later installation of a different model. Then run add_drv to actually install the driver: add_drv -m '* 0666 bin bin' -i '"pci11d5,116" "pci11d5,118"' idr This installs the module idr with permissions 0666 owned by bin in group bin. The node(s) created in /dev/ will be idr0..idrx with one node created per board. The entries in /dev/ will be sybolic links to entries in /devices/pci@1f,2000 or its equivalent on the target machine. The above command will install the driver if either or both 10116 and 10118 boards are present. To "un-install" loadable driver: rem_drv idr Use the modinfo command to verify that the driver is installed and attached. The modinfo line for the idr driver will look something like: 105 ff3e9000 3a84 84 1 idr (IKON DR11-W - 1 Aug, 1996) The driver will show up in modinfo only when it is actually loaded into the kernel. add_drv forces the driver to load, and any call of the driver`s entry points will cause an autoload. The kernel may unload the driver if it has been idle for some (unknown!) length of time. When a card is to be removed or moved to another slot, rem_drv should be run before (re)moving the board. If a second card is added, it seems to be necessary to rem_drv & add_drv to establish the correct device mapping. If a 10116 is removed, and replaced with a 10118, or the reverse, a new idrx entry will be created. The kernel remembers what type of board was in a given slot, and does not re-use an instance number unless the board is replaced with another of the same type. If a two slot machine had two 10116 boards installed, the one in slot 0 would be named idr0 and the one in slot one would be idr1. If the slot 0 board is replaced with a 10118, the 10118 would become idr2, and the 10116 would remain idr1. NOTES: Residual Byte Counts: It is a unix convention that the read and write routines return a count of the number of bytes actually transferred. Unfortunately, the pci interface chip on the 10115, 6, 7, and 8 does not have a readable dma range counter. Therefore, the DR11 driver always indicates that all bytes requested in a read or write call were transferred. Unless an error was encountered, this will in fact always be the case. Errors will be indicated by the driver returning -1 after a read or write call. In general, bytes lost in an aborted read or write cannot be re-sent successfully anyway, so the "actual" byte count would serve little purpose. There are some applications that do not know in advance how many bytes will be received by a read call. In this case, a read is issued with a count larger than the anticipated maximum. The read is typically terminated by an ATTENTION signal from the attached device. AT this time, the application can read the DR11 range counter (not the dma counter) to determine how many bytes were received. "Rewinding" the Driver: A quirk of the unix file system calls treats the ihcp driver as if it were a tape or disk controller. The system stuff that handles the write calls adds up the total of all bytes transferred to a given device. When this 32 bit counter goes negative, the write will not be permitted, and the call will return an error value. The solution to this problem is to have the routine that calls the ihcp driver do an lseek(filedescriptor, (off_t)0, 0) call to "rewind" the ihcp driver. This call should be made often enough that the hidden counter never goes negative. IOCTL parameters: The PCI DR11-W driver's ioctl() commands are compatible with IKON's Sbus DR11-W driver, with a few commands new with this driver. See the xxxx_io.h files for each driver for ioctl command descriptions. Note that the xxxx_io.h files for Solaris 2.x are slightly different from the xxxxio.h files provided with the drivers for Solaris 1. The SVR4 (Solaris 2.x) ioctl macros require that the ASCII portion of the macro argument be quoted - the Solaris 1 macros required that the ASCII argument not be quoted. The old and new ....io.h files will generate the same ioctl commands, but are not interchangeable. Use the new xxxx_io.h files when compiling on a Solaris 2 machine. DR11 Driver IOCTL Calls: See the file called 118notes for info on driver calls and defaults. DR11 Byte Swapping: The way the sparc processor and memory are connected to the pci bus causes the byte lanes of a 32 bit word to be "swizzled" that is, byte 0 -> byte 3, byte 1 -> byte 2, byte 2 -> byte 1, and byte 3 -> byte 0. This causes no problem with the hardcopy board, which is a byte oriented device (in fact, the byte swizzling solves a byte ordering problem). Unfortunately, the DR11 uses 16 bit data. The pci to sparc byte swizzling causes the 16 bit words to come out in the correct order, but the bytes within each 16 bit word are swapped. The 10116 will begin using a new PLX chip (the 9080) with byte swapping capabilities in late 1997. This will take care of the byte swap issue with no software overhead. Until then - for development purposes, the applications using the 10116/Solaris driver will have to swap bytes in software -- if required. Fortunately, given the speed of the new sparc/pci machines, the overhead will not be great. All 10118 boards use a PLX chip (9060SD-1A or 9080) that provides byte swapping. The driver - sparc or x86 - will default to the swap mode appropriate for the host architecture. DR11 Sample Link Programs: The link.master and link.slave programs implement a simple example of a DR11 to DR11 interprocessor link using the 10116/10118 and driver. These programs are compatible with the equivalent program for the Sbus board and driver, and with IKON's in house test link software for IKON's other DR11 emulator products. They may be used for testing, or as examples of the link protocol, and how to communicate with the driver. Also included on this diskette is a version of the link master program - link.master.manual - that runs in manual mode. This is a simple demonstration of how to use multiple read and write calls per DR11 block. This technique is useful in continuous data collection applications. A link connection is accomplished by cross-connecting J1 and J2 between two DR11 emulators (or the real thing!). J1 connects to J2, and J2 to J1. This can be done using short transition cables from the DR11s to a pair of ground-plane ribbon cables, or in the case of IKON pci or Sbus boards, by using a single round, jacketed, shielded link cable. DR11 Loop-back Test Program The idr.loop.test program can be used to perform a repeating loop-back test of the 10116/10118. It requires that an appropriate loop-back cable be installed. This cable accomplishes a pin-for-pin connection between the two sides of the 80-pin high-density connector, or between J1 and J2 of the transition cable assembly. The test program first tests the three software timeout functions of the driver, and then loops indefinitely on several tests of the features of the board and driver. At the end of each pass, it outputs an asterisk "*". In the event of an error, it prints a brief message and exits. This test works correctly only on boards that can "un-swap" DMA bytes that are swapped by the pci-memory path. This includes all 10118s and later 10116 boards that use the PLX 9060SD or 9080 chips. Driver Default Proprties: Several driver defaults may be changed by editing the idr.conf file (in /usr/kernel/drv) and reloading the driver. This is far simpler than the old approach of editing and re-making the driver itsself. It is the only way to modify defaults in installations that do not have Sunsoft's unbundled C compiler available. All default may be overridden by the application, using idr_ioctl() calls. The timing of the handshake used between the DR11 emulator and the attached device is set by the speed_def property. Permissible values are 0, 1, 2, and 3, with 0 giving the fastest timing. In general, the fastest timing that gives reliable data transfers should be used. See the manual for details. There are three software timeout clocks used by the driver during waits: a dma timer, a wait-for-ready timer, and a wait for ATTENTION timer. The defualts for these timers are set by the dma_time_def, rdy_time_def, and attn_time_def properties, respectively. All default times are in seconds. The default byte ordering mode used during DMA transfers is set by the byte_swap_def property. When 0, it gives un-swapped transfers. When one, bytes within 16 bit words will be swapped. The default polarities of the BUSY and CYCLE REQUEST signals are determined by the cycle_pol_def and busy_pol_def properties. Cycle_pol_def = 0 causes the CYCLE REQUEST input to assert on a rising edge; 1 selects falling edge active. Busy_pol_def = 0 causes busy to be asserted low; 1 selects high assertion. These polarities should be selected to match the requirements of the attached device. At the beginning of each DR11 block transfer, some applications may require that the software cycle pulse be used to start the transfer (in link mode, it is required only for the sending end of the link). There are separate soft cycle properties for read and write operations. Write_cycle_def controls the write soft cycle, and read_cycle_def controls reads. Setting either to 1 will cause a soft cycle pulse to be issued at the start of EACH corresponding type of DR11 block. Link mode also requires that the driver issue an ACLO FCN2 pulse (which becomes ATTENTION at the other end of the link) at the start of each DR11 read block. This tells the other end of the link that reads are enabled, and a cycle request from the sender would be honored. This pulse is enabled by setting the read_acf2_def property to 1. The default will be 0 for now - to stay consistent with the Sbus version of the driver. Several properties set the default states of the DR11 function bit outputs, FCN3, FCN2, and FCN1. This is done separately for read and write blocks, and when the device is opened. Each function bit has a separate property for each situation. The names are (hopefully) self- explanatory. Permitted valued are 0 or 1. The x86 version of this driver uses host-based dma chaining, which forms scatter-gather lists in host memory. sg_list_length controls the size of the scatter-gather list, and therefore the amount of host memory (kernel memory) used for lists, and the maximum size of a dma transfer. Each list entry (one per 4K buffer page plus perhaps two more for alignment) requires 16 bytes of host memory. A 256 link chaining list consumes 4K bytes of host memory, and allows a maximum of 1M bytes per dma buffer. This property may be adjusted up or down to allow large buffers (if you get a "too many cookies" message from ihcp_strategy) or to reduce usage of kernel memory. Minphys (buffer sizes): The idr and ihcp drivers both use the system minphys routine when calling the kernel physio from their read or write routine. Minphys determines how large a buffer may be passed from physio to the driver's strategy routine. If the read or write call involves a user buffer larger than minphys, physio will break it into pieces, and make miltiple calls to strategy. This can be a problem for DR11 compatible devices, which MAY want to see a well-defined (via GO and/or READY) beginning and end to each block transferred. Multiple strategy calls per buffer may confuse the device. A way around this is to edit the driver source and add a local minphys routine. A sample of a typical minphys routine is shown below. If a MAXPHYS number larger than the system's mapping capability is chosed, ddi_dma_buf_setup will fail. It will be necessary to choose an appropriate MAXPHYS number for the particular target system. Another way to get around this problem is to use the "MANUAL" mode in the DR11 driver. This allows controlling the beginning and ending conditions of the externally visible block transfer directly with ioctl calls, while satisfying the block size requirement with multiple read/write calls or by using a single read/write call with a large buffer and letting physio break it into pieces. In this mode, the strategy routine does not issue a GO pulse with each strategy buffer transferred. See the file 10118notes and the driver source for further information. xxxx_minphys sample: #define IDR_MAXPHYS some appropriate buffer size static void idr_minphys(struct buf *bp) { bp->b_bcount = (min(bp->b_bcount,IDR_MAXPHYS)) ; }