nfstest.file_io - Man Page

File I/O module

Description

Provides an interface to create and manipulate files of different types. The arguments allow running for a specified period of time as well as running multiple processes. Each process modifies a single file at a time and the file name space is different for each process so there are no collisions between two different processes modifying the same file.

File types:
 - Regular file
 - Hard link
 - Symbolic link

File operations:
 - Open (create or re-open)
 - Open downgrade
   This is done by opening the file for read and write, then the file is
   opened again as read only and finally closing the read and write file
   descriptor
 - Read (sequential or random access)
 - Write (sequential or random access)
 - Remove
 - Rename
 - Truncate (path or file descriptor)
 - Readdir
 - Lock
 - Unlock
 - Tlock

Classes

class FileIO(baseobj.BaseObj)

FileIO object

Usage:
    from nfstest.file_io import FileIO

    # Instantiate FileIO object given top level directory
    x = FileIO(datadir="/tmp/data")

    # Run workload creating the top level directory if necessary
    x.run()


Methods defined here:
---------------------

__del__(self)
Destructor

__init__(self, **kwargs)
Constructor

Initialize object's private data

        datadir:
    Top level directory where files will be created,
    it will be created if it does not exist
        seed:
    Seed to initialized the random number generator
    [default: automatically generated]
        nprocs:
    Number of processes to use [default: 1]
        runtime:
    Run time [default: 0 (indefinitely)]
        verbose:
    Verbose level: none|info|debug|dbg1-7|all [default: 'none']
        exiterr:
    Exit on first error [default: False]
        read:
    Read file percentage [default: 40.0]
        write:
    Write file percentage [default: 40.0]
        rdwr:
    Read/write file percentage [default: 20.0]
        randio:
    Random file access percentage [default: 50.0]
        iodelay:
    Seconds to delay I/O operations [default: 0.0]
        direct:
    Use direct I/O [default: False]
        rdwronly:
    Use read and write only, no rename, remove, etc. [default: False]
        create:
    Create file percentage [default: 5.0]
        odgrade:
    Open downgrade percentage [default: 5.0]
        osync:
    Open file with O_SYNC [default: 10.0]
        fsync:
    Percentage of fsync after write [default: 2.0]
        rename:
    Rename file percentage [default: 5.0]
        remove:
    Remove file percentage [default: 5.0]
        trunc:
    Truncate file percentage [default: 2.0]
        ftrunc:
    Truncate opened file percentage [default: 2.0]
        link:
    Create hard link percentage [default: 1.0]
        slink:
    Create symbolic link percentage [default: 0.2]
        readdir:
    List contents of directory percentage [default: 0.5]
        lock:
    Lock file percentage [default: 20.0]
        unlock:
    Unlock file percentage [default: 80.0]
        tlock:
    Lock test percentage [default: 20.0]
        lockfull:
    Lock full file percentage [default: 50.0]
        minfiles:
    Minimum number of files to create before any file operation
    is executed [default: 10]
        fsizeavg:
    File size average [default: 1m]
        fsizedev:
    File size standard deviation [default: 256k]
        rsize:
    Read block size [default: 64k]
        rsizedev:
    Read block size standard deviation [default: 8k]
        wsize:
    Write block size [default: 64k]
        wsizedev:
    Write block size standard deviation [default: 8k]
        sizemult:
    Size multiplier [default: 1.0]
        createlog:
    Create log file [default: False]
        createlogs:
    Create a log file for each process [default: False]
        logdir:
    Log directory [default: '/tmp']

get_mountpoint(self)
Get mount point from data directory

run(self)
Main function where all processes are started

run_process(self, tid=0)
Main loop for each process

class FileObj(baseobj.BaseObj)

Base class so objects will inherit the methods providing the string
representation of the object and a simple debug printing and logging
mechanism.

Usage:
    from baseobj import BaseObj

    # Named arguments
    x = BaseObj(a=1, b=2)

    # Dictionary argument
    x = BaseObj({'a':1, 'b':2})

    # Tuple arguments: first for keys and second for the values
    x = BaseObj(['a', 'b'], [1, 2])

    # All of the above will create an object having two attributes:
    x.a = 1 and x.b = 2

    # Add attribute name, this will be the only attribute to be displayed
    x.set_attrlist("a")

    # Add list of attribute names to be displayed in that order
    x.set_attrlist(["a", "b"])

    # Set attribute with ordered display rights
    x.set_attr("a", 1)
    # This is the same as
    setattr(x, "a", 1) or x.a = 1
    x.set_attrlist("a")

    # Set attribute with switch duplicate
    # The following creates an extra attribute "switch" with
    # the same value as attribute "a":
    #   x.a == x.switch
    #   x.a is x.switch
    x.set_attr("a", 1, switch=True)

    # Make the current object flat by allowing all the attributes
    # for the new attribute to be accessed directly by the current
    # object so the following is True:
    #   x.d == x.c.d
    x.set_attr("c", BaseObj(d=11, e=22), switch=True)

    # Set the comparison attribute so x == x.a is True
    x.set_eqattr("a")

    # Set verbose level of object's string representation
    x.debug_repr(level)

    # Set string format for verbose level 1
    x.set_strfmt(1, "arg1:{0}")
    # In the above example the first positional argument is "a"
    # so the str(x) gives "arg1:1"

    # Set attribute shared by all instances
    # If a global or shared attribute is set on one instance,
    # all other instances will have access to it:
    #   y = BaseObj(d=2, e=3)
    # then the following is true
    #   x.g == y.g
    #   x.g is y.g
    x.set_global("g", 5)

    # Set level mask to display all debug messages matching mask
    x.debug_level(0xFF)

    # Add a debug mapping for mask 0x100
    x.debug_map(0x100, 'opts', "OPTS: ")

    # Set global indentation to 4 spaces for dprint
    x.dindent(4)

    # Set global indentation to 4 spaces for displaying objects
    x.sindent(4)

    # Set global truncation to 64 for displaying string objects
    x.strsize(64)

    # Do not display timestamp for dprint messages
    x.tstamp(enable=False)

    # Change timestamp format to include the date
    x.tstamp(fmt="{0:date:%Y-%m-%d %H:%M:%S.%q} ")

    # Get timestamp if enabled, else return an empty string
    out = x.timestamp()

    # Open log file
    x.open_log(logfile)

    # Close log file
    x.close_log()

    # Write data to log file
    x.write_log(data)

    # Format the given arguments
    out = x.format("{0:x} - {1}", 1, "hello")

    # Format the object attributes set by set_attrlist()
    out = x.format("{0:x} - {1}")

    # Print debug message only if OPTS bitmap matches the current
    # debug level mask
    x.dprint("OPTS", "This is an OPTS debug message")

class TermSignal(builtins.Exception)

Exception to be raised on SIGTERM signal

Functions

stop_handler(signum, frame)
Signal handler to catch SIGTERM and allow for graceful termination
of subprocesses

See Also

baseobj(3), formatstr(3)

Bugs

No known bugs.

Author

Jorge Mora (mora@netapp.com)

Referenced By

nfstest_io(1).

21 March 2023 NFStest 3.2 file_io 1.3