bcc-biolatpcts - Man Page

biolatpcts [-h] [-i INTERVAL] [-w which] [-p PCT,...] [-j] [-v] DEV

biolatpcts traces block device I/O (disk I/O) of the specified device, and calculates and prints the completion latency distribution percentiles per IO type periodically. Example:

# biolatpcts /dev/nvme0n1
nvme0n1    p1    p5   p10   p16   p25   p50   p75   p84   p90   p95   p99  p100
read     95us 175us 305us 515us 895us 985us 995us 1.5ms 2.5ms 3.5ms 4.5ms  10ms
write     5us   5us   5us  15us  25us 135us 765us 855us 885us 895us 965us 1.5ms
discard   5us   5us   5us   5us 135us 145us 165us 205us 385us 875us 1.5ms 2.5ms
flush     5us   5us   5us   5us   5us   5us   5us   5us   5us 1.5ms 4.5ms 5.5ms
[...]

biolatpcts prints a number of pre-set latency percentiles in tabular form every three seconds. The interval can be changed with the -i option.

The latency is measured between issue to the device and completion. The starting point can be changed with the -w option.

Any number of percentiles can be specified using the -p option. The input percentile string is used verbatim in the output to ease machine consumption.

-j option enables json output. The result for each interval is printed on a single line.

This tool works by tracing blk_account_io_done() with kprobe and bucketing the completion latencies into percpu arrays. It may need updating to match the changes to the function.

Since this uses BPF, only the root user can use this tool.

CONFIG_BPF, CONFIG_KPROBES and bcc.

-h Print usage message.

-i INTERVAL, --interval INTERVAL

Report interval. (default: 3)

-w {from-rq-alloc,after-rq-alloc,on-device}, --which {from-rq-alloc,after-rq-alloc,on-device}

Which latency to measure. (default: on-device)

-p PCT,..., --pcts PCT,...

Percentiles to calculate. (default: 1,5,10,16,25,50,75,84,90,95,99,100)

-j, --json

Output in json.

-v, --verbose

Enable debug output.

DEV

Target block device. /dev/DEVNAME, DEVNAME or MAJ:MIN.

Print sda's I/O latency percentiles every second

# biolatpcts -i 1 sda

Print nvme0n1's all-9 I/O latency percentiles in json every second

# biolatpcts -p 99,99.9,99.99,99.999 -j -i 1 /dev/nvme0n1

This traces kernel functions and maintains in-kernel per-cpu latency buckets, which are asynchronously copied to user-space. This method is very efficient, and the overhead for most storage I/O rates should be negligible. If you have an extremely high IOPS storage device, test and quantify the overhead before use.

This is from bcc.

https://github.com/iovisor/bcc

Also look in the bcc distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

Linux

Unstable - in development.

Tejun Heo

biolatency(8), biosnoop(8)