dt - Man Page

Data Test Program

Synopsis

dt [options]

Extreme Warning!!!

Use of this program is almost guaranteed to find problems and cause your schedules to slip. If you are afraid to find bugs or otherwise break your system, then please do not use this program for testing. You can pay now or pay later, but you've been warned!

Description

dt is a generic data test program used to verify the proper operation of peripherals and I/O sub-systems, and for obtaining performance information. Since verification of data is performed, dt can be thought of as a generic diagnostic tool.

Although the original design goals of being a generic test tool were accomplished, it quickly become evident that device specific tests, such as terminals, and different programming interfaces such as memory mapped files and POSIX asynchronous I/O API's were necessary. Therefore, special options were added to enable these test modes and to specify necessary test parameters.

dt command lines are similar to the dd program, which is popular on most UNIX systems. dt contains numerous options to provide user control of most test parameters so customized tests can be written easily and quickly by specifying simple command line options. Since the exit status of the program always reflects the completion status of a test, scripts can easily detect failures to perform automatic regression tests.

dt has been used to successfully test disks, tapes, serial lines, parallel lines, pipes & FIFO's, memory mapped files, and POSIX Asynchronous I/O. In fact, dt can be used with any device that supports the standard open, read, write, and close system calls. Special support is necessary for some devices, such as serial lines, for setting up the speed, parity, data bits, etc, but dt's design provides easy addition of this setup.

Most tests can be initiated by a simple dt command line, and lots of I/O can be initiated quickly using multiple processes and/or POSIX AIO, for those operating systems supporing AIO. More complex tests are normally initiated by writing shell scripts and using dt in conjunction with other tools, such as scu (SCSI Command Utility). Several shell scripts for testing disks, tapes, and serial lines are also supplied with this kit which can used as templates for developing other specialized test scripts.

Specific system features are now being added to dt so more extensive testing can be accomplished. The program has been restructured to allow easy inclusion of new device specific tests by dispatching to test functions through a function lookup table. This table gets setup automatically, based on options enabled, or via the device type "dtype=" option.

WARNING: dt does not perform any sanity checking of the output device specified. This means if you are running as root on Unix and you specify a raw disk device, dt will overwrite existing file systems, so please be careful! I HATE TO ADMIT, I'VE DONE THIS MYSELF!

Operating Systems Supported

dt is conditionalized to run on AIX, HP-UX, SUN, ULTRIX, OSF, QNX, SCO Unixware, Windows, and Linux operating systems. Porting is simple for OSes with POSIX APIs.

Test Uses

Those people with an imagination will find many uses for dt, but I'll list a few I've used it for, just to whet your appetite:

Program Options

This section describes program options and and special notes related to each. The dt help file provides a summary of the options, and the default value of most options.

Input File if= Option

This option specifies the input file to open for reads. The device is opened read-only so devices which only permit or support read access, e.g., parallel input devices, can be opened successfully.

Special Notes:

  • Data read is automatically verified with the default data pattern, unless you disable this action via the "disable=compare" option.
  • Extra pad bytes of sizeof(int), are allocated at the end of data buffers, initialized with the inverted data pattern, and then verified after each read request to ensure the end of data buffers didn't get overwritten by file system and/or device drivers. This extra check has found problems with flushing DMA FIFO's on several machines.

Syntax:

if=filename

The input file to read.

Output File of= Option

This option specifies the output file to open for writes. After the write portion of the test, the device is closed (to reposition to start of file or to rewind the tape), re-opened, and then a read verification pass is performed. If you wish to prevent the read verify pass, you must specify the "disable=verify" option.

Special Notes:

  • Terminal devices are closed between passes so previously set terminal characteristics don't get reset. This also caused a race condition when doing loopback testing with two processes.
  • When testing terminal (serial) devices, modem control is disabled (via setting CLOCAL) to prevent tests from hanging. If the "enable=modem" option is specified, then CLOCAL is reset, hangup on close HUPCL is set, and testing will not preceed until carrier or DSR is detected. This code is not fully tested, but this description accurately describes the code.
  • At the present time, tapes are rewound by closing the device, so you must specify the rewind device during testing if the read verify pass is being performed. This restriction will probably change in the next release since magtape control commands will be supported (tape specific tests as well).
  • O_CREAT open flag is cleared to prevent accidently creating files in this directory when not specifying the correct device name (very easy to do when running tests as super-user 'root').
  • When writing to raw disks on Tru64 UNIX, if the disk was previously labeled, you must issue the "disklabel -z" command to destroy the label block or else you cannot write to this area of this disk (block 0). Failure to do this results in the error "Read-only file system" (errno=EROFS) being returned on write requests.

Syntax:

of=filename

The output file to write.

Pattern File pf= Option

This option specifies a pattern file to use for the data pattern during testing. This option overrides the "pattern=" option and allows you to specify specialized patterns. The only restriction to this option is that the entire file must fit in memory. A buffer is allocated to read the entire pattern file into memory before testing starts so performance is not affected by reading the pattern file.

Syntax:

pf=filename

The data pattern file to use.

Block Size bs= Option

This option specifies the block size, in bytes, to use during testing. At the present time, this option sets both the input and output block sizes. At the time I originally wrote this program, I didn't have the need for seperate block sizes, but this may change in a future release where I'll add back the "ibs=" and "obs=" options available with dd.

Special Notes:

  • When enabling variable length records via the "min=" option, this also sets the maximum record size to be written/read.
  • For memory mapped files, the block size be a multiple of the system dependent page size (normally 4k or 8k bytes).

Syntax:

bs=value

The block size to read/write.

Log File log[tu]= Options

This option specifies the log file to redirect all program output to. This is done by re-opening the standard error stream (stderr) to the specifed log file. Since all output from dt is directed to stderr, library functions such as perror() also write to this log file.

Special Notes

  • A seperate buffer is allocated for the stderr stream, and this stream is set buffered so timing isn't affected by program output.
  • When starting multiple processes via the "procs=" option, all output is directed to the same log file. The output from each process is identified by the process ID (PID) as part of the message (errors & statistics).
  • logt=filename will truncate the existing log file.
  • logu=filename will create unique log files with multiple processes (w/pid).

Syntax:

log[tu]=filename

The log file name to write.

Special format keywords are now expanded when part of the log file name, so unique names can be created for each test:

Log File Format Keywords:

%iodir

The I/O direction.

%iotype

The I/O type.

%host

The host name.

%pid

The process ID.

%user

The user name.

Example:

log=dt_%host_%user_%iodir_%iotype-%pid.log

Please see the DiskTests.ksh script for examples of using this.

POSIX Asynchronous I/O aios= Option

This option enables and controls the number of POSIX Asychronous I/O requests used by the program.

Special Notes

  • The default is to queue up to 8 requests.
  • The system limit for AIO on Tru64 UNIX is dynamic, and can be queried by using the "sysconfig -q rt" command.
  • You can use the "enable=aio" option to enable AIO and use the default request limit.
  • AIO is only supported for character devices and is disabled for terminals. On Tru64 UNIX, you can alter the Makefile and link against libaio.a, which allows AIO with any device/file by mimic'ing AIO using POSIX threads.
  • AIO requests can be cancelled on Tru64 UNIX, so queuing many requests to 1/2in tape devices will probably result in running off the end of the tape reel. This is not a problem for cartridge tapes.

Syntax:

aios=value

Set number of AIO's to queue.

Keepalive Alarm Time alarm= Option

Keepalive Message *keepalive= Options

These options control a user defined message that will be emitted during the test. The user defines how often to display the keepalive message, via the "alarm=time" option, and the format of the message(s), via the "*keepalive=string" options. The normal "keepalive=" option defines the script emitted during the test, while "pkeepalive=" is the per pass message string, and "tkeepalive=" is the totals message string (overriding what dt normally displays). For a full description of the message control strings see the UserGuide.

Syntax:

alarm=time

The keepalive alarm time.

keepalive=string

The keepalive message string.

pkeepalive=str

The pass keepalive msg string.

tkeepalive=str

The totals keepalive msg string.

Keepalive Message Format Control

The keepalive string is free format like a printf(), with the following format control strings:

Keepalive Format Control:

%b

The bytes read or written.

%B

Total bytes read and written.

%c

Record count for this pass.

%C

Total records for this test.

%d

The device name.

%D

The real device name.

%e

The number of errors.

%E

The error limit.

%f

The files read or written.

%F

Total files read and written.

%h

The host name.

%H

The full host name.

%k

The kilobytes this pass.

%K

Total kilobytes for this test.

%l

Blocks read or written.

%L

Total blocks read and written.

%m

The megabytes this pass.

%M

Total megabytes for this test.

%p

The pass count.

%P

The pass limit.

%r

Records read this pass.

%R

Total records read this test.

%s

The seconds this pass.

%S

The total seconds this test.

%t

The pass elapsed time.

%T

The total elapsed time.

%i

The I/O mode (read/write)

%u

The user (login) name.

%w

Records written this pass.

%W

Total records written this test.

Performance Keywords:

%bps

The bytes per second.

%lbps

Logical blocks per second.

%kbps

Kilobytes per second.

%mbps

The megabytes per second.

%iops

The I/O's per second.

%spio

The seconds per I/O.

Lowercase means per pass stats, while uppercase means total stats.

Default:

%d Stats: mode %i, blocks %l, %m Mbytes, pass %p/%P, elapsed %t

or if pass statistics summary is disabled:

%d Stats: mode %i, blocks %L, %M Mbytes, pass %p/%P, elapsed %T

Here's an example used by Hazards' diskdt process:

keepalive="count = %C; e = %e; t = %S; IOpS = %IOPS; SpIO = %SPIO"
tkeepalive="STAT +RawMbytes %MBPS +RawReads %R +RawWrites %W";

Buffer Alignment align= Option

This option controls the alignment of the normally page aligned data buffer allocated. This option is often useful for testing certain DMA boundary conditions not easily reproduced otherwise. The rotate option automatically adjust the data buffer pointer by (0, 1, 2, 3, ...) for each I/O request to ensure various boundaries are fully tested.

Syntax:

align=offset

Set offset within page aligned buffer.

align=rotate

Rotate data address through sizeof(ptr).

File Disposition dispose= Option

This option controls the disposition of test files created on file systems. By default, the test file created is deleted before exiting, but sometimes you may wish to keep this file for further examination, for use as a pattern file, or simply for the read verify pass of another test (e.g., reading the file via memory map API).

Syntax:

dispose=mode

Set file dispose to: {delete, keep, or keeponerror}.

Dump Data Limit dlimit= Option

This option allows you to specify the dump data limit used when data compare errors occur. The default dump data limit is 64 bytes.

Syntax:

dlimit=value

Sets the data dump limit to value.

Device Size dsize= Option

This option allows you to specify the device block size used. On Tru64 Unix, the device block size is obatined automatically by an OS specific IOCTL. For all other systems, random access devices default to 512 byte blocks. You'll likely use this option with C/DVD's, since their default block size to 2048 bytes per block.

Syntax:

dsize=value

Set the device block (sector) size.

Device Type dtype= Option

Input Device Type idtype= Option

Output Device Type odtype= Option

These options provide a method to inform dt of the type of device test to be performed. Without this knowledge, only generic testing is possible.

Special Notes

  • On Tru64 UNIX systems, these options are not necessary, since this information is obtained via the DECIOCGET or DEVGETINFO IOCTL's.
  • Although the program accepts a large number of device types, as shown below, specific tests only exists for "disk", "tape", "fifo", and "terminal" device types. Others may be added in the future.
  • In the case of "disk" device type, reports the relative block number when read, write, or data compare errors occur.
  • Also for "disk" devices, will automatically determine the disk capacity if a data or record limit is not specified. This is done via a series of seek/read requests.
  • On each operating system supported, string compares are done on well known device names to automatically select the device type. For example on QNX, "/dev/hd" for disk, "/dev/tp" for tapes, and "/dev/ser" for serial lines.
  • The device type gets displayed in the total statictics.

Syntax:

dtype=string

Sets the device type.

idtype=string

Sets the input device type.

odtype=string

Sets the output device type.

The Valid Device Types Are:

audio comm disk graphics

memory mouse network fifo

pipe printer processor socket

special streams tape terminal

unknown

Note: Although dt does not provide specific test support for each of the devices shown above, its' design makes it easy to add new device specific tests. Specific support exists for disk, fifo, pipe, tape, and terminals. Support for "ptys" may be added in the future as well.

Error Limit errors= Option

This option controls the maximum number of errors tolerated before the program exits.

Special Notes

  • The default error limit is 1.
  • All errors have a time stamp associated with them, which are useful for characterizing intermittent error conditions.
  • The error limit is adjusted for read, write, or data compare failures. This limit is not enforced when flushing data, or for certain AIO wait operations which are considered non-fatal (perhaps this will change).
  • A future release may support an "onerr=" option to control the action of errors (e.g., loop, ignore (continue), or exit).

Syntax:

errors=value

The number of errors to tolerate.

File Limit files= Option

This option controls the number of tape files to process with tape devices.

Special Notes

  • During the write pass, a tape file mark is written after each file. After all files are written, 1 or 2 file marks will be written automatically by the tape driver when the device is closed.
  • During reads, each file is expected to be terminated by a file mark and read() system calls are expected to return a value of  denoting the end of file. When reading past all tapes files, an errno of ENOSPC is expected to flag the end of media condition.
  • Writing tape file marks is currently not supported on the QNX Operating System. The release I currently have does not support the mtio commands, and unfortunately the POSIX standard does define this interface (the mtio interface appears to be a UNIX specific standard). Multiple tape files can still be read on QNX systems however.

Syntax:

files=value

Set number of tape files to process.

Terminal Flow Control flow= Option

This option specifies the terminal flow control to use during testing.

Special Notes

  • The default flow control is "xon_xoff".
  • When using XON/XOFF flow control, you must make sure these byte codes (Ctrl/Q = XON = ' 21', Ctrl/S = XOFF = ' 23), since the program does not filter these out automatically. Also be aware of terminal servers (e.g., LAT), or modems (e.g., DF296) which may eat these characters.
  • Some serial lines do support clear-to-send (CTS) or request-to-send (RTS) modem signals. For example on Alpha Flamingo machines, only one port (/dev/tty00) supports full modem control, while the alternate console port (/dev/tty01) does not. Therefore, if running loopback between both ports, you can not use flow control, the test will hang waiting for these signals to transition (at least, I think this is the case).

Syntax:

flow=type

Set flow to: none, cts_rts, or xon_xoff.

History history= Option

This option sets the number of I/O history entries to record. During failures, the history is dumped, which can be helpful when troubleshooting failures.

Syntax:

history=value

Set the number of history request entries.

History Data Size hdsize= Option

When I/O history is enabled, this option controls how many data bytes are saved for each I/O.

Syntax:

hdsize=value Set the history data size (bytes to save). Default hdsize=32 (set to 0 to disable copy)

Record Increment incr= Option

This option controls the bytes incremented when testing variable length records. After each record, this increment value (default 1), is added to the last record size (starting at "min=", up to the maximum record size "max=").

Special Notes

  • If variable length record testing is enabled on fixed block disks and this option is omitted, then "incr=" defaults to 512 bytes.

Syntax:

incr=value

Set number of record bytes to increment.

or
incr=variable

Enables variable I/O request sizes.

I/O Direction iodir= Option

This option allows you to control the I/O direction with random access devices. The default direction is forward.

Syntax:

iodir=direction

Set I/O direction to: {forward or reverse}.

I/O Mode iomode= Option

This option controls the I/O mode used, either copy, test, or verify modes. The copy option was added to do a byte for byte copy between devices, while skipping bad blocks and keeping file offsets on both disks in sync. I've used this option to (mostly) recover my system disk which developed bad blocks which could not be re-assigned. A verify operation automatically occurs after the copy, which is real handy for unreliable diskettes.

Syntax:

iomode=mode

Set I/O mode to: {copy, test, or verify}.

IOT Pass iotpass= Option

This option is used to specify the IOT pass number. When multiple passes occur, dt factors in the pass count to generate unique data during each pass. For example, the IOT seed is normally 0x01010101, and will be multiplied by the pass specified, useful for re-reading previously written IOT data patterns.

Syntax:

iotpass=value

Set the IOT pattern for specified pass.

IOT Seed iotseed= Option

This option is used to specify the last IOT pattern seed dt used. When multiple passes occur, dt now factors in the pass count to generate unique data during each pass. For example, the IOT seed is normally 0x01010101, but this is now multiplied by the pass count for uniqueness.

Syntax:

iotseed=value

Set the IOT pattern block seed value.

I/O Type iotype= Option

This option controls the type of I/O performed, either random or sequential. The default is to do sequential I/O.

Special Notes

  • The random number generator used is chosen by defines: RAND48 to select srand48()/lrand48(), RANDOM to select srandom()/random(), and if neither are defined, srand()/rand() gets used by default. Refer to your system literature or manual pages to determine which functions are supported.

Syntax:

iotype=type

Set I/O type to: {random or sequential}.

The seeks are limited to the data limited specified or calculated from other options on the dt command line. If data limits are not specified, seeks are limited to the size of existing files, or to the entire media for disk devices (calculated automatically by dt). If the data limits exceed the capacity of the media/partition/file under test, a premature end-of-file will be encountered on reads or writes, but this is treated as a warning (expected), and not as an error.

Minimum Record Size min= Option

This option controls the minimum record size to start at when testing variable length records.

Special Notes

  • By default, tests using fixed length records of block size "bs=" bytes.
  • This option, in conjuntion with the "max=" and "incr=" control variable length record sizes.
  • If variable length record testing is enabled on fixed block disks and this option is omitted, then "min=" defaults to 512 bytes.

Syntax:

min=value

Set the minumum record size to transfer.

Maxmimum Record Size max= Option

The option controls the maximum record size during variable length record testing.

Special Notes

  • If the "min=" option is specified, and this option is omitted, then the maximum record size is set to the block size "bs=".
  • This option, in conjuntion with the "min=" and "incr=" control variable length record sizes.

Syntax:

max=value

Set the maximum record size to transfer.

Logical Block Address lba= Option

This option sets the starting logical block address used with the "lbdata" option. When specified, the logical block data "enable=lbdata" option is automatically enabled.

Syntax:

lba=value Set starting block used w/lbdata option.

Special Notes

  • Please do not confuse this option with the disks' real logical block address. See 's "seek=" or "position=" options to set the starting file position.
  • Also note that doesn't know about disk partitions, so any position specified is relative to the start of the partition used.

Logical Block Size lbs= Option

This option sets the starting logical block size used with the lbdata option. When specified, the logical block data (enable=lbdata) option is automatically enabled.

Syntax:

lbs=value

Set logical block size for lbdata option.

Data Limit limit= Option

This option specifies the number of data bytes to transfer during each write and/or read pass for the test.

Special Notes

  • You must specify either a data limit, record limit, or files limit to initiate a test, unless the device type is "disk", in which case dt will automatically determine the disk capacity.  
  • When specifying a runtime via the "runtime=" option, the data limit controls how many bytes to process for each pass (write and/or read pass).
  • If you specify a infinite "limit=inf" value, each pass will continue until the end of media or file is reached.
  • When the "step=value" option is used, limit controls the maximum offset stepped to.

Syntax:

limit=value

The number of bytes to transfer.

Munsa (DLM) munsa= Option

This option is used on Tru64 Cluster systems to specify various distributed lock manager (DLM) options with devices or files.

Syntax:

munsa=string

Set munsa to: cr, cw, pr, pw, ex.

MUNSA Lock Options:

cr

Concurrent Read (permits read access, cr/pr/cw by others)

pr

Protected Read (permits cr/pr read access to all, no write)

cw

Concurrent Write (permits write and cr access to resource by all)

pw

Protected Write (permits write access, cr by others)

ex

Exclusive Mode (permits read/write access, no access to others)

For more details, please refer to the dlm(4) reference page.

Special Notes

  • MUNSA is an obsolete Tru64 Cluster term which meant MUltiple Node Simultaneous Access. The new term is DAIO for Direct Access I/O. Finally, the last term used is DRD for Distributed Request Dispatcher.

Common Open Flags flags= Option

Output Open Flags oflags= Option

These options are used to specify various POSIX compliant open flags, and system specific flags, to test the affect of these open modes.

Special Notes

  • Each operating system has different flags, which can be queried by reviewing the help text (dt help).

Syntax:

flags=flags

Set open flags: {excl,sync,...}.

oflags=flags

Set output flags: {append,trunc,...}.

On Child Error oncerr= Option

This option allows you to control the action taken by dt when a child process exits with an error. By default, the action is continue, which allows all child processes to run to completion. If the child error action is set to abort, then dt aborts all child processes if any child process exits with an error status.

Syntax:

oncerr={abort|continue}

Set child error action.

No Progress Time noprogt= Option

This option allows you to specify a time (in seconds) to report when I/O is not making progress. This option is used in conjunction with the "alarm=" option to periodically check for an report when I/O is taking too long. This is especially useful during controller failover type testing.

Syntax:

noprogt=value

Set the no progress time (in seconds).

No Progress Time Trigger noprogtt= Option

This option allows you to specify a time (in seconds) when to initiate the no-progress time trigger script. Note: This option has no effect, unless the noprogt= option is enabled.

Syntax:

noprogtt=value

Set the no progress time trigger (in seconds).

No Time notime= Option

This option allows you to disable timing of certain operations (system calls), when the no-progress options is enabled. Valid optype's are: open close read write ioctl fsync msync aiowait

Special Notes

  • This option has no effect, unless the option is enabled.

Syntax:

notime=optype

Disable timing of specified operation type.

Terminal Parity Setting parity= Option

This option specifies the terminal parity setting to use during testing.

Syntax:

parity=string

Set parity to: even, odd, or none.

parity=string

(QNX) Set parity to: even, odd, mark, space, or none.

Pass Limit passes= Option

This option controls the number of passes to perform for each test.

Special Notes

  • The default is to perform 1 pass.
  • When using the "of=" option, each write/read combination is considered a single pass.
  • When multiple passes are specified, a different data pattern is used for each pass, unless the user specified a data pattern or pattern file. [ Please keep this in mind when using the "dispose=keep" option, since using this same file for a subsequent read verify pass, will report comparison errors... I've burnt myself this way. :-( ]

Syntax:

passes=value

The number of passes to perform.

Data Pattern pattern= Option

This option specifies a 32 bit hexadecimal data pattern to be used for the data pattern. dt has 12 built-in patterns, which it alternates through when running multiple passes. The default data patterns are:

0x39c39c39, 0x00ff00ff, 0x0f0f0f0f, 0xc6dec6de, 0x6db6db6d, 0x00000000, 0xffffffff, 0xaaaaaaaa, 0x33333333, 0x26673333, 0x66673326, 0x71c7c71c

You can also specify the special keyword "incr=" to use an incrementing data pattern, or specify a character string (normally contained within single or double quotes).

Syntax:

pattern=value

The 32 bit hex data pattern to use.

or
pattern=iot

Use DJ's IOT test pattern.

or
pattern=incr

Use an incrementing data pattern.

or
pattern=string

The string to use for the data pattern.

File Position position= Option

This option specifies a byte offset to seek to prior to starting each pass of each test.

Syntax:

position=offset

Position to offset before testing.

Prefix prefix= Option

This option allows the user to define a free format prefix string which is written at the beginning of each block. It is used to generate uniqueness useful when data corruption occur. Certain format control strings are interpreted as shown below.

Syntax:

prefix=string

The data pattern prefix string.

The prefix format controls permitted are:

Prefix Format Control:
%d

The device name.

%D

The real device name.

%h

The host name.

%H

The full host name.

%p

The process ID.

%P

The parent PID.

%u = The user name.

Example:

prefix="%u@%h (pid %p)"

Multiple Processes procs= Option

This option specifies the number of processes to initiate performing the same test. This option allows an easy method for initiating multiple I/O requests to a single device or file system.

Special Notes

  • The per process limit on Tru64 UNIX is 64, and can be queried by using the "sysconfig -q proc" command.
  • Spawning many processes can render your system useless, well at least very slow, and consumes large amounts of swap space (make sure you have plenty!).
  • The parent process simply monitors (waits for) all child prcoesses.
  • When writing to a file system, the process ID (PID) is appended to the file name specified with the "of=" option to create unique file names. If no pattern is specified, each process is started with a unique data pattern. Subsequent passes cycle through the 12 internal data patterns. Use "disable=unique" to avoid this behaviour.
  • The spawn() facility, used to execute on a different node, is not implemented on the QNX Operating System at this time.

Syntax:

procs=value

The number of processes to create.

Set Queue Depth qdepth= Option

This option is currently only implemented on HP-UX. It allow you to set the queue depth of the device under test, overriding its' default. Note: The settings is sticky (retained).

Syntax:

qdepth=value

Set the queue depth to specified value.

Random I/O Offset Alignment wralign= Option

This option is used when performing random I/O, to align each random block offset to a particular alignment, for example 32K.

Syntax:

ralign=value

The random I/O offset alignment.

Random I/O Data Limit rlimit= Option

This option is used with random I/O to specify the number of bytes to limit random I/O between (starting from block  to this range). This option is independent of the data limit option.

Syntax:

rlimit=value

The random I/O data byte limit.

Random Seed Value rseed= Option

This options sets the seed to initialize the random number generator with, when doing random I/O. When selecting random I/O, the total statistics displays the random seed used during that test. This option can be used to repeat the random I/O sequence of a test.

Syntax:

rseed=value

The random seed to initialize with.

Record Limit records= Option

This option controls the number of records to process for each write and/or read pass of each test. The "count=" option is an alias for this option (supported for dd compatibility).

Special Notes

  • You must specify either a data limit, record limit, or files limit to initiate a test, unless the device type is "disk", in which case dt will automatically determine the disk capacity.
  • When specifying a runtime via the "runtime=" option, the record limit controls how many records process for each pass (write and/or read pass).
  • If you specify a infinite "records=Inf" value, each pass will continue until the end of media or file is reached.

Syntax:

records=value

The number of records to process.

Run Time runtime= Option

This option controls how long the total test should run. When used in conjunction with a data limit or record limit, multiple passes will be performed until the runtime limit expires. A later section entitled "Time Input Parameters", describes the shorthand notation for time values.

Syntax:

runtime=time

The number of seconds to execute.

Retry Delay retry_delay= Option

This option controls the number of seconds to delay between reads performed after a data corruption. (see enable=retryDC option)

Syntax:

retry_delay=value

Delay before retrying operation. (Def: 5)

Slice slice= Option

This option is used with random access devices. This option is used in conjunction with the "slices=value" option, which divides the media into slices (see below), then "slice=value" defines the slice to do testing to. Since dt does the calculations, this simplifies simultaneous testing from multiple hosts to shared storage (usually a multi-initiator test requrement).

Syntax:

slice=value

The specific disk slice to test.

Slices slices= Option

This option is used with random access devices. This option divides the media into slices. Each slice contains a different range of blocks to operate on in a separate process. If no pattern is specified, then each slice is started with a unique data pattern. Subsequent passes alternate through dt's 12 internal patterns.

Syntax:

slices=value

The number of disk slices to test.

Note: This option can be used in conjuntion with multiple processes and/or asynchronous I/O options to generate a heavy I/O load, great for stress testing!

Record Skip skip= Option

This option specifies the numer of records to skip prior to starting each write and/or read pass of each test. The skips are accomplished by reading records.

Syntax:

skip=value

The number of records to skip past.

Record Seek seek= Option

This option specifies the number of records to seek past prior to starting each write and/or read test. The seeks are accomplished by lseek()'ing past records, which is much faster than skipping when using random access devices.

Syntax:

seek=value

The number of records to seek past.

Data Step step= Option

This option is used to specify non-sequential I/O requests to random access devices. Normally, dt does sequential read & writes, but this option specifies that step bytes to be seeked past after each request.

Special Notes

  • The "limit=value" option can be used to set the maximum offset.

Syntax:

step=value

The number of bytes seeked after I/O.

Terminal Speed speed= Option

This option specifies the terminal speed (baud rate) to setup prior to initiating the test. Although dt supports all valid baud rates, some speeds may not be supported by all serial line drivers, and in some cases, specifying higher speeds may result in hardware errors (e.g., silo overflow, framing error, and/or hardware/software overrun errors). The valid speeds accepted by dt are:

0 50 75 110 134 150

200 300 600 1200 1800 2400

4800 9600 19200 38400 57600 115200

Although a baud rate of zero is accepted, this is done mainly for testing purposes (some systems use zero to hangup modems). The higher baud rates are only valid on systems which define the Bxxxxx speeds in termios.h.

Special Notes

  • The default speed is 9600 baud.

Syntax:

speed=value

The tty speed (baud rate) to use.

Terminal Read Timeout timeout= Option

This option specifies the timeout to use, in 10ths of a second, when testing terminal line interfaces. This is the timeout used between each character after the first character is received, which may prevent tests from hanging when a character is garbled and lost.

Special Notes

  • The default terminal timeout is 3 seconds.
  • The default timeout is automatically adjusted for slow baud rates.

Syntax:

timeout=value

The tty read timeout in .10 seconds.

Terminal Read Minimum ttymin= Option

This option specifies the minmum number of characers to read, sets the VMIN tty attribute.

Special Notes

  • The tty VMIN field normally gets sets to the value of the block size ().
  • Note that on some systems, the VMIN field is an unsigned char, so the maximum value is 255.
  • On QNX, this field is an unsigned short, so a maximum of 65535 is valid.

Syntax:

ttymin=value

The tty read minimum count (sets vmin).

Trigger Action trigger= Option

This option specifies a trigger action to take whenever an error occurs and/or when the no-progress time has been exceeded (see "enable=noprog"). It's main purpose is for triggering an anlyzer and/or stopping I/O by some means (panic, etc) when trouble-shooting.

Syntax:

trigger=type The trigger to execute during errors.

Trigger Types:

br

Execute a bus reset.

bdr

Execute a bus device reset.

seek

Issue a seek to the failing lba.

cmd:string

Execute command with these args: string dname op dsize offset position lba errno

The first three options require Scu in your PATH.

When specifying the "cmd:" type, which invokes a program/script, the following arguments are passed on the command line:

Format: cmd dname op dsize offset position lba errno noprogtime

Where:
dname = The device/file name.
op = open/close/read/write/miscompare/noprog
dsize = The device block size.
offset = The current file offset.
position = The failing offset within block.
lba = The logical block address (relative for FS).
errno = The error number on syscall errors.
noprogtime = The no-progress time (in seconds).

Multiple Volumes volumes= Option

Multi-Volume Records vrecords= Option

These options are used with removal media devices, to define how many volumes and records on the last volume to process (i.e., tapes, etc). By using these options, you do not have to guess at a data limit or record limit, to overflow onto subsequent volumes. These options automatically sets the "enable=multi" option.

Syntax:

volumes=value

The number of volumes to process.

vrecords=value

The record limit for the last volume.

Enable enable= and Disable disable= Options

These options are used to either enable or disable program flags which either alter default test modes, test actions, or provide additional debugging information. You can specify a single flag or multiple flags each seperated by a comma (e.g., "enable=aio,debug,dump").

Syntax:

enable=flag Enable one or more of the flags below.

disable=flag Disable one or more of the flags below.

The flags which can be enabled or disabled are described below.

POSIX Asynchronous I/O aio Flag

This flag is used to control use of POSIX Asynchronous I/O during testing, rather than the synchronous I/O read() and write() system calls.

Special Notes

  • Beware, you may need to rebuild on new versions of Tru64 Unix due to POSIX changes and/or AIO library changes between major releases.
  • Reference the "aios=" option, for more special notes.

Flag:

aio

POSIX Asynchronous I/O.(Default: disabled)

Reporting Close Errors cerror Flag

This flag controls where close errors are reported as an error or a failure. When disabled, close errors are reported as a warning. This flag is meant to be used as a workaround for device drivers which improperly return failures when closing the device. Many system utilities ignore close failures, but when testing terminals and tapes, the close status us very important. For example with tapes, the close reflects the status of writing filemarks (which also flush buffered data), and the rewind status.

Flag:

cerrors

Report close errors. (Default: enabled)

Data Comparison compare Flag

This flag disables data verification during the read pass of tests. This flag should be disabled to read to end of file/media to obtain maximum capacity statistics, or to obtain maximum performance statistics (less overhead).

Flag:

compare

Data comparison. (Default: enabled)

Core Dump on Errors coredump Flag

This flag controls whether a core file is generated, via abort(), when dt is exiting with a failure status code. This is mainly used for program debug, and is not of much interest to normal users. When testing multiple processes, via fork(), this is useful if your OS debugger does not support debugging child processes.

Flag:

coredump

Core dump on errors. (Default: disabled)

Diagnostic Logging diag Flag

This option is only valid on Tru64 Unix. When enabled, error messages get logged to the binary error logger. This is useful to correlate device error entries with test failures. Please note, the logging only occurs when running as superuser (API restriction, not mine!).

Flag:

diag

Log diagnostic msgs. (Default: disabled)

Debug Output debug Flag

Verbose Debug Output Debug Flag

Random I/O Debug Output rdebug Flag

These flags enable two different levels of debug, which are useful when trouble-shooting certain problems (i.e., what is dt doing to cause this failure?). Both flags can be specified for full debug output.

Flag:

debug

Debug output. (Default: disabled)

Debug

Verbose debug output. (Default: disabled)

edebug

End of file debug. (Default: disabled)

rdebug

Random debug output. (Default: disabled)

tdebug

Timer debug output. (Default: disabled)

Dump Data Buffer dump Flag

This flag controls dumping of the data buffer during data comparision failures. If a pattern file is being used, then the pattern buffer is also dumped for easy comparision purposes. To prevent too many bytes from being dumped, esp. when using large block sizes, dumping is limited to 512 bytes of data (was 64, recently increased).

Special Notes

  • When the failure occurs within the first 64 bytes of the buffer, dumping starts at the beginning of the buffer.
  • When the failure occurs at some offset within the data buffer, then dumping starts at (data limit/2) bytes prior to the failing byte to provide context.
  • The start of the failing data is marked by an asterisk '*'.
  • You can use the option to override the default dump limit.
  • Buffer addresses are displayed for detection of memory boundary problems.

Flag:

dump

Dump data buffer. (Default: enabled)

Tape EEI Reporting eei Flag

This option controls the reporting of Extended Error Information (EEI) on Tru64 UNIX systems, for tape devices when errors occur. The standard tape information available from mt is reported, along with the EEI status, CAM status, and SCSI request sense data. This is excellent information to help diagnose tape failures. (thank-you John Meneghini!)

Flag:

eei

Tape EEI reporting. (Default: enabled)

Flush Terminal I/O Queues flush Flag

This flag controls whether the terminal I/O queues get flushed before each test begins. This must be done to ensure no residual characters are left in the queues from a prior test, or else data verification errors will be reported. Residual characters may also be left from a previous XOFF'ed terminal state (output was suspended).

Flag:

flush

Flush tty I/O queues. (Default: enabled)

History Dumping hdump Flag

This flag controls dumping the history entries at the end of a test. Normally dt only dumps the history during errors, but this option when enabled, dumps the history when exiting. This is useful if you are timing I/O's, or wish to see the LBA's I/O went to, etc.

Flag:

hdump

History dump. (Default: disabled)

History Timing htiming Flag

This flag controls the timing of history entries. Please be aware, that enabling timing of each I/O will impact your overall test performance, as an extra system call is used to obtain system time.

Flag:

htiming

History timing. (Default: disabled)

Log File Header header Flag

When a log file is specified, dt automatically writes the command line and dt version information at the beginning of the log file. This option allows you to control whether this header should be written.

Flag:

header

Log file header. (Default: enabled)

Loop On Error looponerror Flag

This flag controls lopping on data corruption rereads. This can be helpful in capturing the failing read request on an analyzer.

Special Notes

  • Also see "retry_delay=value" and retryDC flag control.

Flag:

looponerror

Loop on error. (Default: disabled)

Logical Block Data Mode lbdata Flag

This option enables a feature called logical block data mode. This feature allows reading/writing of a 4-byte (32-bit) logical block address at the beginning of each data block tested. The block number is stored using SCSI byte ordering (big-endian), which matches what the SCSI Write Same w/lbdata option uses, so dt can verify this pattern, generated by scu's "write same" command.

Special Notes

  • The starting logical block address defaults to 0, unless overridden with the "lba=" option.
  • The logical block size defaults to 512 bytes, unless overridden with the "lbs=" option.
  • The logical block address is always inserted started at the beginning of each data block.
  • Enabling this feature will degrade performance statistics (slightly).

Enable Loopback Mode loopback Flag

This flag specifies that either the input or output file should be used in a loopback mode. In loopback mode, dt forks(), and makes the child process the reader, while the parent process becomes the writer. In previous versions of dt, you had to specify both the same input and output file to enable loopback mode. When specifying this flag, dt automatically duplicates the input or output device, which is a little cleaner than the old method (which still works).

Some people may argue that dt should automatically enable loopback mode when a single terminal or FIFO device is detected. The rationale behind not doing this is described below:

  • You may wish to have another process as reader and/or writer (which also includes another program, not necessarily ).
  • You may wish to perform device loopback between two systems (e.g., to verify the terminal drivers of two operating systems are compatible).
  • A goal of is to force (hardcode) actions or options to make the program more flexible. A minimum of validity checking is done to avoid being too restrictive, although hooks exists to do this.

Special Notes

  • The read verify flag is automatically disabled.
  • This mode is most useful with terminal devices and/or FIFO's (named pipes).

Microsecond Delays microdelay Flag

This flag tells dt that delay values, i.e. "sdelay=" and others, should be executed using microsecond intervals, rather the second intervals. (thank-you George Bittner for implementing this support!)

Flag:

microdelay

Microsecond delays. (Default: disabled)

Memory Mapped I/O mmap Flag

This flag controls whether the memory mapped API is used for testing. This test mode is currently supported on SUN/OS, Tru64 UNIX, and Linux operating systems.

Special Notes

  • The block size specified "bs=" be a multiple of the system dependent page size (normally 4k or 8k).
  • An msync() is done after writing and prior to closing to force modified pages to permanent storage. It may be useful to add an option to inhibit this action at some point, but my testing was specifically to time mmap performance. Obviously, invalidating the memory mapped pages, kind of defeats the purpose of using memory mapped files in the first place.
  • Specifying multiple passes when doing a read verify test, gives you a good indication of the system paging utilization on successive passes.
  • Memory mapping large data files (many megabytes) may exhaust certain system resources. On an early version of SUN/OS V4.0?, I could hang my system by gobbling up all of physical memory and forcing paging (this was certainly a bug which has probably been corrected since then).

Flag:

mmap

Memory mapped I/O. (Default: disabled)

Test Modem Lines modem Flag

This flag controls the testing of terminal modem lines. Normally, dt disables modem control, via setting CLOCAL, to prevent tests from hanging. When this flag is enabled, dt enables modem control, via clearing CLOCAL, and then monitoring the modem signals looking for either carrier detect (CD) or dataset ready (DSR) before allowing the test to start.

Special Notes

  • The program does not contain modem signal monitoring functions for the all operating systems. The functions in are specific to Tru64 UNIX and ULTRIX systems, but these can be used as templates for other operating systems.

Flag:

modem

Test modem tty lines. (Default: disabled)

Multiple Volumes multi Flag

This flag controls whether multiple volumes are used during testing. When this flag is enabled, if the data limit or record count specified does not fit on the current loaded media, the user is prompted to insert the next media to continue testing. Although this is used mostly with tape devices, it can be used with any removeable media.

Flag:

multi

Multiple volumes. (Default: disabled)

No I/O Progress noprog Flag

This flag controls whether dt will check for slow or no I/O progress during testing.

Special Notes Enabling this flag will do nothing by itself. The "alarm=" option specifies the frequency of how often dt checks for no progress.

  • The "noprogt=secs" option specified the no I/O progress time.
  • If "noprogt=" is omitted, it defaults to the "alarm" time value.
  • The noprog flag is implicitly enabled by the "noprogt=value" option.

Flag:

noprog

No progress check. (Default: disabled)

Prefill prefill Flag

This flag controls the buffer prefill normally performed prior to reads. Normally, dt prefills the buffer with the inverted data pattern (1st four bytes). This, of course, ensures the data is overwritten with data read, but also imposes overhead not always desirable.

Special Notes

  • When IOT pattern is used, this flag is automatically enabled, since IOT blocks are unique.

Flag:

prefill

Prefill read buffer. (Default: enabled)

Control Per Pass Statistics pstats Flag

This flag controls whether the per pass statistics are displayed. If this flag is disabled, a single summary line is still displayed per pass and the total statistics are still displayed in the full format.

Flag:

pstats

Per pass statistics. (Default: enabled)

Read After Write raw Flag

This flag controls whether a read-after-write will be performed. Sorry, raw does not mean character device interface. Normally dt performs a write pass, followed by a read pass. When this flag is enabled the read/verify is done immediately after the write.

Flag:

raw

Read after write. (Default: disabled)

Tape Reset Handling resets Flag

This option is used during SCSI bus and device reset testing, to reposition the tape position (tapes rewind on resets), and to continue testing. This option is only enabled for Tru64 UNIX systems (currently), since this option requires reset detection from EEI status, and tape position information from the CAM tape driver (although dt also maintains the tape position as a sanity check against the drivers' data)

Flag:

resets

Tape reset handling. (Default: disabled)

Retry Data Corruptions retryDC Flag

This flag controls whether a data corruption retry is performed. A second read is done to re-read the data, with direct I/O for file systems, and the data is compared against the previous read data, and the expected data. If the reread data matches the expected data, then dt assumes a "read failure" occurred, otherwise if the reread data matches the previous read, dt assumes a "write failure" (the data was written incorrectly).

Flag:

retryDC

Retry data corruptions.(Default: enabled)

Control Program Statistics stats Flag

This flag controls whether any statistics get displayed (both pass and total statistics). Disabling this flag also disabled the pass statistics described above.

Flag:

stats

Display statistics. (Default: enabled)

Table(sysinfo) timing table Flag

On Tru64 UNIX systems, this option enables additional timing information which gets reported as part of the statistics display. (thanks to Jeff Detjen for adding this support!)

Flag:

table

Table(sysinfo) timing. (Default: disabled)

System Log syslog Flag

This flag controls logging startup/finish and errors being logged to the system logger. This can be helpful when correlating dt's errors with system (driver/file system) error messages.

Flag:

syslog

Log errors to syslog. (Default: disabled)

Timestamp Blocks timestamp Flag

This flag controls whether blocks are timestamped when written. The timestamp is skipped during data comparisions, but is displayed if any remaining data is incorrect.

Special Notes

  • When IOT or lbdata patterns are used, the block number is overwritten by the timestamp.
  • This flag is a stop-gap, until block tagging (w/more information) is implemented.

Flag:

timestamp

Timestamp each block. (Default: disabled)

Unique Pattern unqiue Flag

This flag controls whether multiple process, get a unqiue data pattern. This affects processes started with the "slices=" or the "procs=" options. This only affects the procs= option when writing to a regular file.

Flag:

unique

Unique pattern. (Default: enabled)

Verbose Output verbose Flag

This flag controls certain informational program messages such as reading and writing partial records. If you find these messages undesirable, then they can be turned off by disabling this flag.

But beware, partial reads or writes of disk records if not at EOF is usually a problem!

Flag:

verbose

Verbose output. (Default: enabled)

Verify Data verify Flag

This flag controls whether the read verify pass is performed automatically after the write pass. Ordinarily, when specifying an output device via the "of=" option, a read verify pass is done to read and perform a data comparision. If you only wish to write the data, and omit the data verification read pass, then di able this flag.

Flag:

verify

Verify data written. (Default: enabled)

Special Notes

  • If you don't plan to ever read the data being written, perhaps for performance reasons, specifying "disable=compare" prevents the data buffer from being initialized with a data pattern.
  • This verify option has no affect when reading a device. You must disable data comparsions via "disable=compare".

Program Delays

dt allows you to specify various delays to use at certain points of the test. These delays are useful to slow down I/O requests or to prevent race conditions when testing terminals devices with multiple processes, or are useful for low level driver debugging. All delay values are in seconds, unless you specify "enable=microdelay", to enable micro-second delays.

Close File cdelay= Delay

This delay, when enabled, is performed prior to closing a file descriptor.

Delay

cdelay=value

Delay before closing the file. (Def: 0)

End of Test edelay= Delay

This delay, when enabled, is used to delay after closing a device, but prior to re-opening the device between multiple passes.

Delay:

edelay=value

Delay between multiple passes. (Def: 0)

Read Record rdelay= Delay

This delay, when enabled, is used prior to issuing each read request (both synchronous read()'s and asynchronous aio_read()'s).

Delay:

rdelay=value

Delay before reading each record. (Def: 0)

Start Test sdelay= Delay

This delay, when enabled, is used prior to starting the test. When testing terminal devices, when not in self loopback mode, the writing process (the parent) automatically delays 1 second, to allow the reading process (the child) to startup and setup its' terminal characteristics. If this delay did not occur prior to the first write, the reader may not have its' terminal characteristics (flow, parity, & speed) setup yet, and may inadvertantly flush the writers data or receive garbled data.

Delay:

sdelay=value

Delay before starting the test. (Def: 0)

Child Terminate tdelay= Delay

This delay is used by child processes before exiting, to give the parent process sufficient time to cleanup and wait for the child. This is necessary since if the child exits first, a SIGCHLD signal may force the parent to it's termination signal handler before it's ready to. This is a very simplistic approach to prevent this parent/child race condition and is only currently used by the child for terminal loopback testing.

Delay:

tdelay=value

Delay before child terminates. (Def: 1)

Write Record wdelay= Delay

This delay, when enabled, is used prior to issuing each write request (both synchronous write()'s and asynchronous aio_write()'s).

Delay:

wdelay=value

Delay before writing each record. (Def: 0)

Numeric Input Parameters

For any options accepting numeric input, the string entered may contain any combination of the following characters:

Special Characters:

w

words (4 bytes)

q

quadwords (8 bytes)

b

blocks (512 bytes)

k

kilobytes (1024 bytes)

m

megabytes (1048576 bytes)

p

page size (8192 bytes)

g

gigabytes (1073741824 bytes)

t

terabytes (1099511627776 bytes)

inf or INF

infinity (18446744073709551615 bytes)

Arithmetic Characters:

+

addition

-

subtraction

* or x

multiplication

/

division

%

remainder

Bitwise Characters:

~

complement of value

>>

shift bits right

<<

shift bits left

&

bitwise 'and' operation

|

bitwise 'or' operation

^

bitwise exclusive 'or'

The default base for numeric input is decimal, but you can override this default by specifying 0x or 0X for hexadecimal coversions, or a leading zero '0' for octal conversions.

NOTE: Certain values will vary depending on the operating system and/or machine you are running on. For example, the page size is system dependent, and the value for Infinity is the largest value that will fit into an unsigned long long (value shown above is for 64-bit systems), or double for systems which don't support "long long".)

Time Input Parameters

When specifying the run time "runtime=" option, the time string entered may contain any combination of the following characters:

Time Input:

d

days (86400 seconds)

h

hours (3600 seconds)

m

minutes (60 seconds)

s

seconds (the default)

Arithmetic characters are permitted, and implicit addition is performed on strings of the form '1d5h10m30s'.

Future Enhancements

Initially dt was written to be a generic test tool, designed to test any device, and although that was (mostly) accomplished, device specific tests needed to be and were developed, based on the device type detected or specified by the "dtype=" option if not determined automatically.

Some of the features requested include:

Final Comments

I'm happy to report that is getting wide spread use all over the world! Storage groups, terminal/lat groups, Q/A, developers, and other peripheral qualification groups are using as part of their testing. I guess maybe this will be my (computer) legacy?

Anyways, I hope you find as useful as I have. This is usually one of the first tools I port to a new operating system, since it's an excellent diagnostic and performance tool (it gives me a warm and fuzzy feeling ).

Author

Written by Robin Miller <Robin.T.Miller@gmail.com>

Info

27 Jul 2009 dt version 16.20