nbdkit-python-plugin - Man Page

Assuming you have a Python script which is an nbdkit plugin, you run it like this:

 nbdkit python /path/to/plugin.py

You may have to add further key=value arguments to the command line.  Read the Python script to see if it requires any.

Since nbdkit ≥ 1.16 only Python 3 is supported.  If you wish to continue using nbdkit plugins written in Python 2 then you must use nbdkit ≤ 1.14, but we advise you to update your plugins.

The version of Python 3 is chosen when nbdkit is built.  This is compiled in and can't be changed at runtime.  ./configure looks for (in order):

• the PYTHON variable (eg ./configure PYTHON=/usr/bin/python3.9)
• python3 on $PATH
• python on $PATH

./configure will fail if the first interpreter found is a Python 2 interpreter.

To find out which version of Python nbdkit-python-plugin was compiled for, use the --dump-plugin option:

 $ nbdkit python --dump-plugin
 ...
 python_version=3.7.0
 python_pep_384_abi_version=3

The nbdkit API has evolved and new versions are released periodically. To ensure backwards compatibility plugins have to opt in to the new version.  From Python you do this by declaring a constant in your module:

 API_VERSION = 2

(where 2 is the latest version at the time this documentation was written).  All newly written Python modules must have this constant.

If you want you can make the script executable and include a "shebang" at the top:

 #!/usr/sbin/nbdkit python

See also "Shebang scripts" in nbdkit(1).

These scripts can also be installed in the $plugindir.  See "WRITING PLUGINS IN OTHER PROGRAMMING LANGUAGES" in nbdkit-plugin(3).

Your script may use import nbdkit to have access to the following methods in the nbdkit module:

nbdkit.debug(msg)

Send a debug message to stderr or syslog if verbose messages are enabled.

nbdkit.disconnect(force)

Disconnect from the client.  If force is True then nbdkit will disconnect the client immediately.

nbdkit.export_name()

Return the export name negotiated with the client as a Unicode string. Note this should not be trusted because the client can send whatever it wants.

nbdkit.is_tls()

Returns True if the client completed TLS authentication, or False if the connection is plaintext.

nbdkit.nanosleep(secs, nsecs)

Sleep for seconds and nanoseconds.

nbdkit.parse_delay(what, str)

Parse a delay or sleep (such as "10ms") into a pair (sec, nsec). Wraps the nbdkit_parse_delay(3) function.

nbdkit.parse_size(str)

Parse a string (such as "100M") into a size in bytes.  Wraps the nbdkit_parse_size(3) C function.

nbdkit.parse_probability(what, str)

Parse a string (such as "100%") into a probability, returning a floating point number.  Wraps the nbdkit_parse_probability(3) function.

nbdkit.peer_pid(), nbdkit.peer_uid(), nbdkit.peer_gid(), nbdkit.peer_security_context()

Return the client process ID, user ID, group ID or security context. The PID, UID and GID are only available when the client connects by Unix domain socket, and then only on some operating systems.  The security context is usually the SELinux label, IPSEC label or NetLabel.

nbdkit.peer_tls_dn()

Return the client TLS Distinguished Name. See nbdkit_peer_tls_dn(3).

nbdkit.peer_tls_issuer_dn()

Return the client certificate issuer's TLS Distinguished Name. See nbdkit_peer_tls_issuer_dn(3).

nbdkit.read_password(value)

Read a password from a config parameter.  This returns the password as a Python bytes object.  See nbdkit_read_password(3) for more information on the different ways that the value parameter can be parsed.

nbdkit.set_error(err)

Throwing a Python exception from a callback causes an error message to be sent back to the NBD client.  The NBD protocol allows an error code (ie. errno) to be sent to the client, but by default the Python plugin always sends EIO.  To control what error code is sent call nbdkit.set_error:

 def pread(h, buf, offset):
   if access_denied:
     nbdkit.set_error(errno.EPERM)
     raise RuntimeError()

nbdkit.shutdown()

Request asynchronous server shutdown.

nbdkit.stdio_safe()

Returns True if it is safe to interact with stdin and stdout during the configuration phase.

After import nbdkit the following constants are available.  These are used in the callbacks below.

nbdkit.THREAD_MODEL_SERIALIZE_CONNECTIONS

nbdkit.THREAD_MODEL_SERIALIZE_ALL_REQUESTS

nbdkit.THREAD_MODEL_SERIALIZE_REQUESTS

nbdkit.THREAD_MODEL_PARALLEL

Possible return values from thread_model().

nbdkit.FLAG_MAY_TRIM

nbdkit.FLAG_FUA

nbdkit.FLAG_REQ_ONE

nbdkit.FLAG_FAST_ZERO

Flags bitmap passed to certain plugin callbacks.  Not all callbacks with a flags parameter use all of these flags, consult the documentation below and nbdkit-plugin(3).

nbdkit.FUA_NONE

nbdkit.FUA_EMULATE

nbdkit.FUA_NATIVE

Possible return values from can_fua().

nbdkit.CACHE_NONE

nbdkit.CACHE_EMULATE

nbdkit.CACHE_NATIVE

Possible return values from can_cache().

nbdkit.EXTENT_HOLE

nbdkit.EXTENT_ZERO

Used in the type field returned by extents().

The thread model for Python callbacks defaults to nbdkit.THREAD_MODEL_SERIALIZE_ALL_REQUESTS.

Since nbdkit 1.22 it has been possible to set this by implementing a thread_model() function which returns one of the constants nbdkit.THREAD_MODEL_*.

The Python Global Interpreter Lock (GIL) is still used, so Python code does not run in parallel.  However if a plugin callback calls a library which blocks (eg. to make an HTTP request), then another callback might be executed in parallel.  Plugins which use nbdkit.THREAD_MODEL_SERIALIZE_REQUESTS or nbdkit.THREAD_MODEL_PARALLEL may need to use locks on shared data.

Python callbacks should throw exceptions to indicate errors.  Remember to use nbdkit.set_error if you need to control which error is sent back to the client; if omitted, the client will see an error of EIO.

This just documents the arguments to the callbacks in Python, and any way that they differ from the C callbacks.  In all other respects they work the same way as the C callbacks, so you should go and read nbdkit-plugin(3).

dump_plugin

(Optional)

There are no arguments or return value.

config

(Optional)

 def config(key, value):
   # no return value

config_complete

(Optional)

There are no arguments or return value.

thread_model

(Optional, nbdkit ≥ 1.22)

 def thread_model():
   return nbdkit.THEAD_MODEL_SERIALIZE_ALL_REQUESTS

See "Threads" above.

get_ready

(Optional)

There are no arguments or return value.

after_fork

(Optional, nbdkit ≥ 1.26)

There are no arguments or return value.

cleanup

(Optional, nbdkit ≥ 1.28)

There are no arguments or return value.

list_exports

(Optional)

 def list_exports(readonly, is_tls):
   # return an iterable object (eg. list) of
   # (name, description) tuples or bare names:
   return [ (name1, desc1), name2, (name3, desc3), ... ]

default_export

(Optional)

 def default_export(readonly, is_tls):
   # return a string
   return "name"

preconnect

(Optional, nbdkit ≥ 1.26)

 def preconnect(readonly):
   # no return value

open

(Required)

 def open(readonly):
   # return handle

You can return any Python value (even None) as the handle.  It is passed back as the first arg 'h' in subsequent calls.  To return an error from this method you must throw an exception.

close

(Optional)

 def close(h):
   # no return value

After close returns, the reference count of the handle is decremented in the C part, which usually means that the handle and its contents will be garbage collected.

export_description

(Optional)

 def export_description(h):
   # return a string
   return "description"

get_size

(Required)

 def get_size(h):
   # return the size of the disk

block_size

(Option)

 def block_size(h):
   # return triple (minimum, preferred, maximum) block size

is_rotational

(Optional)

 def is_rotational(h):
   # return a boolean

can_multi_conn

(Optional)

 def can_multi_conn(h):
   # return a boolean

can_write

(Optional)

 def can_write(h):
   # return a boolean

can_flush

(Optional)

 def can_flush(h):
   # return a boolean

can_trim

(Optional)

 def can_trim(h):
   # return a boolean

can_zero

(Optional)

 def can_zero(h):
   # return a boolean

can_fast_zero

(Optional)

 def can_fast_zero(h):
   # return a boolean

can_fua

(Optional)

 def can_fua(h):
   # return nbdkit.FUA_NONE or nbdkit.FUA_EMULATE
   # or nbdkit.FUA_NATIVE

can_cache

(Optional)

 def can_cache(h):
   # return nbdkit.CACHE_NONE or nbdkit.CACHE_EMULATE
   # or nbdkit.CACHE_NATIVE

can_extents

(Optional)

 def can_extents(h):
   # return a boolean

pread

(Required)

 def pread(h, buf, offset, flags):
   # read into the buffer

The body of your pread function should read exactly len(buf) bytes of data starting at disk offset and write it into the buffer buf.  flags is always 0.

NBD only supports whole reads, so your function should try to read the whole region (perhaps requiring a loop).  If the read fails or is partial, your function should throw an exception, optionally using nbdkit.set_error first.

pwrite

(Optional)

 def pwrite(h, buf, offset, flags):
   length = len(buf)
   # no return value

The body of your pwrite function should write the buffer buf to the disk.  You should write count bytes to the disk starting at offset.  flags may contain nbdkit.FLAG_FUA.

NBD only supports whole writes, so your function should try to write the whole region (perhaps requiring a loop).  If the write fails or is partial, your function should throw an exception,
optionally using nbdkit.set_error first.

flush

(Optional)

 def flush(h, flags):
   # no return value

The body of your flush function should do a sync(2) or fdatasync(2) or equivalent on the backing store. flags is always 0.

If the flush fails, your function should throw an exception, optionally using nbdkit.set_error first.

trim

(Optional)

 def trim(h, count, offset, flags):
   # no return value

The body of your trim function should "punch a hole" in the backing store.  flags may contain nbdkit.FLAG_FUA.  If the trim fails, your function should throw an exception, optionally using nbdkit.set_error first.

zero

(Optional)

 def zero(h, count, offset, flags):
   # no return value

The body of your zero function should ensure that count bytes of the disk, starting at offset, will read back as zero.  flags is a bitmask which may include nbdkit.FLAG_MAY_TRIM, nbdkit.FLAG_FUA, nbdkit.FLAG_FAST_ZERO.

NBD only supports whole writes, so your function should try to write the whole region (perhaps requiring a loop).

If the write fails or is partial, your function should throw an exception, optionally using nbdkit.set_error first.  In particular, if you would like to automatically fall back to pwrite (perhaps because there is nothing to optimize if flags & nbdkit.FLAG_MAY_TRIM is false), use nbdkit.set_error(errno.EOPNOTSUPP).

cache

(Optional)

 def cache(h, count, offset, flags):
   # no return value

The body of your cache function should prefetch data in the indicated range.

If the cache operation fails, your function should throw an exception, optionally using nbdkit.set_error first.

extents

(Optional)

 def extents(h, count, offset, flags):
   # return an iterable object (eg. list) of
   # (offset, length, type) tuples:
   return [ (off1, len1, type1), (off2, len2, type2), ... ]

Missing: load

This is not needed since you can use regular Python mechanisms like top level statements to run code when the module is loaded.

Missing: unload

This is missing, but in nbdkit ≥ 1.28 you can put code in the cleanup() function to have it run when nbdkit exits.  In earlier versions of nbdkit, using a Python atexit handler is recommended.

Missing: name, version, longname, description, config_help, magic_config_key.

These are not yet supported.