collectd-python - Man Page

  LoadPlugin python
  # ...
  <Plugin python>
    ModulePath "/path/to/your/python/modules"
    LogTraces true
    Interactive false
    Import "spam"

    <Module spam>
      spam "wonderful" "lovely"
    </Module>
  </Plugin>

The python plugin embeds a Python-interpreter into collectd and provides an interface to collectd's plugin system. This makes it possible to write plugins for collectd in Python. This is a lot more efficient than executing a Python-script every time you want to read a value with the exec plugin (see collectd-exec(5)) and provides a lot more functionality, too.

The minimum required Python version is 2.6.

LoadPlugin Plugin

Loads the Python plugin Plugin.

Encoding Name

The default encoding for Unicode objects you pass to collectd. If you omit this option it will default to ascii on Python 2. On Python 3 it will always be utf-8, as this function was removed, so this will be silently ignored. These defaults are hardcoded in Python and will ignore everything else, including your locale.

ModulePath Name

Prepends Name to sys.path. You won't be able to import any scripts you wrote unless they are located in one of the directories in this list. Please note that it only has effect on plugins loaded after this option. You can use multiple ModulePath lines to add more than one directory.

LogTraces bool

If a Python script throws an exception it will be logged by collectd with the name of the exception and the message. If you set this option to true it will also log the full stacktrace just like the default output of an interactive Python interpreter. This does not apply to the CollectError exception, which will never log a stacktrace. This should probably be set to false most of the time but is very useful for development and debugging of new modules.

Interactive bool

This option will cause the module to launch an interactive Python interpreter that reads from and writes to the terminal. Note that collectd will terminate right after starting up if you try to run it as a daemon while this option is enabled so make sure to start collectd with the -f option.

The collectd module is not imported into the interpreter's globals. You have to do it manually. Be sure to read the help text of the module, it can be used as a reference guide during coding.

This interactive session will behave slightly differently from a daemonized collectd script as well as from a normal Python interpreter:

• 1. collectd will try to import the readline module to give you a decent way of entering your commands. The daemonized collectd won't do that.

• 2. Python will be handling SIGINT. Pressing Ctrl+C will usually cause collectd to shut down. This would be problematic in an interactive session, therefore Python will be handling it in interactive sessions. This allows you to use Ctrl+C to interrupt Python code without killing collectd. This also means you can catch KeyboardInterrupt exceptions which does not work during normal operation.

  To quit collectd send EOF (press Ctrl+D at the beginning of a new line).

• 3. collectd handles SIGCHLD. This means that Python won't be able to determine the return code of spawned processes with system(), popen() and subprocess. This will result in Python not using external programs like less to display help texts. You can override this behavior with the PAGER environment variable, e.g. export PAGER=less before starting collectd. Depending on your version of Python this might or might not result in an OSError exception which can be ignored.

  If you really need to spawn new processes from Python you can register an init callback and reset the action for SIGCHLD to the default behavior. Please note that this will break the exec plugin. Do not even load the exec plugin if you intend to do this!

  There is an example script located in contrib/python/getsigchld.py  to do this. If you import this from collectd.conf SIGCHLD will be handled normally and spawning processes from Python will work as intended.

Import Name

Imports the python script Name and loads it into the collectd python process. If your python script is not found, be sure its directory exists in python's sys.path. You can prepend to the sys.path using the ModulePath configuration option.

<Module Name> block

This block may be used to pass on configuration settings to a Python module. The configuration is converted into an instance of the Config class which is passed to the registered configuration callback. See below for details about the Config class and how to register callbacks.

The name identifies the callback.

There are a lot of places where strings are sent from collectd to Python and from Python to collectd. How exactly this works depends on whether byte or unicode strings or Python2 or Python3 are used.

Python2 has str, which is just bytes, and unicode. Python3 has str, which is a unicode object, and bytes.

When passing strings from Python to collectd all of these object are supported in all places, however str should be used if possible. These strings must not contain a NUL byte. Ignoring this will result in a TypeError exception. If a byte string was used it will be used as is by collectd. If a unicode object was used it will be encoded using the default encoding (see above). If this is not possible Python will raise a UnicodeEncodeError exception.

When passing strings from collectd to Python the behavior depends on the Python version used. Python2 will always receive a str object. Python3 will usually receive a str object as well, however the original string will be decoded to unicode using the default encoding. If this fails because the string is not a valid sequence for this encoding a bytes object will be returned instead.

Writing your own plugins is quite simple. collectd manages plugins by means of dispatch functions which call the appropriate callback functions registered by the plugins. Any plugin basically consists of the implementation of these callback functions and initializing code which registers the functions with collectd. See the section “Examples” below for a really basic example. The following types of callback functions are known to collectd (all of them are optional):

configuration functions

These are called during configuration if an appropriate Module block has been encountered. It is called once for each Module block which matches the name of the callback as provided with the register_config method - see below.

Python thread support has not been initialized at this point so do not use any threading functions here!

init functions

These are called once after loading the module and before any calls to the read and write functions. It should be used to initialize the internal state of the plugin (e. g. open sockets, ...). This is the earliest point where you may use threads.

read functions

These are used to collect the actual data. It is called once per interval (see the Interval configuration option of collectd). Usually it will call plugin_dispatch_values to dispatch the values to collectd which will pass them on to all registered write functions. If this function throws any kind of exception the plugin will be skipped for an increasing amount of time until it returns normally again.

write functions

These are used to write the dispatched values. It is called once for every value that was dispatched by any plugin.

flush functions

These are used to flush internal caches of plugins. It is usually triggered by the user only. Any plugin which caches data before writing it to disk should provide this kind of callback function.

log functions

These are used to pass messages of plugins or the daemon itself to the user.

notification function

These are used to act upon notifications. In general, a notification is a status message that may be associated with a data instance. Usually, a notification is generated by the daemon if a configured threshold has been exceeded (see the section “THRESHOLD Configuration” in collectd.conf(5) for more details), but any plugin may dispatch notifications as well.

shutdown functions

These are called once before the daemon shuts down. It should be used to clean up the plugin (e.g. close sockets, ...).

Any function (except log functions) may throw an exception in case of errors. The exception will be passed on to the user using collectd's logging mechanism. If a log callback throws an exception it will be printed to standard error instead.

See the documentation of the various register_ methods in the section “Functions” below for the number and types of arguments passed to each callback function. This section also explains how to register callback functions with collectd.

To enable a module, copy it to a place where Python can find it (i. e. a directory listed in sys.path) just as any other Python plugin and add an appropriate Import option to the configuration file. After restarting collectd you're done.

The following complex types are used to pass values between the Python plugin and collectd:

 class CollectdError(Exception)

 class Signed(long)

 class Unsigned(long)

 class Config(object)

 class PluginData(object)

 class Values(PluginData)

The following functions provide the C-interface to Python-modules.

register_*(callback[, data][, name]) -> identifier

There are eight different register functions to get callback for eight different events. With one exception all of them are called as shown above.

• callback is a callable object that will be called every time the event is triggered.
• data is an optional object that will be passed back to the callback function every time it is called. If you omit this parameter no object is passed back to your callback, not even None.
• name is an optional identifier for this callback. The default name is python.module. module is taken from the __module__ attribute of your callback function. Every callback needs a unique identifier, so if you want to register the same callback multiple times in the same module you need to specify a name here. Otherwise it's safe to ignore this parameter.
• identifier is the full identifier assigned to this callback.

These functions are called in the various stages of the daemon (see the section “Writing Your Own Plugins” above) and are passed the following arguments:

register_config

The only argument passed is a Config object. See above for the layout of this data type. Note that you cannot receive the whole config files this way, only Module blocks inside the Python configuration block. Additionally you will only receive blocks where your callback identifier matches python.blockname.

register_init

The callback will be called without arguments.

register_read(callback[, interval][, data][, name]) -> identifier

This function takes an additional parameter: interval. It specifies the time between calls to the callback function.

The callback will be called without arguments.

register_shutdown

The callback will be called without arguments.

register_write

The callback function will be called with one argument passed, which will be a Values object. For the layout of Values see above. If this callback function throws an exception the next call will be delayed by an increasing interval.

register_flush

Like register_config is important for this callback because it determines what flush requests the plugin will receive.

The arguments passed are timeout and identifier. timeout indicates that only data older than timeout seconds is to be flushed. identifier specifies which values are to be flushed.

register_log

The arguments are severity and message. The severity is an integer and small for important messages and high for less important messages. The least important level is LOG_DEBUG, the most important level is LOG_ERR. In between there are (from least to most important): LOG_INFO, LOG_NOTICE, and LOG_WARNING. message is simply a string without a newline at the end.

If this callback throws an exception it will not be logged. It will just be printed to sys.stderr which usually means silently ignored.

register_notification

The only argument passed is a Notification object. See above for the layout of this data type.

unregister_*(identifier) -> None

Removes a callback or data-set from collectd's internal list of callback functions. Every register_* function has an unregister_* function. identifier is either the string that was returned by the register function or a callback function. The identifier will be constructed in the same way as for the register functions.

get_dataset(name) -> definition

Returns the definition of a dataset specified by name. definition is a list of tuples, each representing one data source. Each tuple has 4 values:

name

A string, the name of the data source.

type

A string that is equal to either of the variables DS_TYPE_COUNTER, DS_TYPE_GAUGE, DS_TYPE_DERIVE or DS_TYPE_ABSOLUTE.

min

A float or None, the minimum value.

max

A float or None, the maximum value.

flush(plugin[, timeout][, identifier]) - None

Flush one or all plugins. timeout and the specified identifiers are passed on to the registered flush-callbacks. If omitted, the timeout defaults to -1. The identifier defaults to None. If the plugin argument has been specified, only named plugin will be flushed.

error,  warning,  notice,  info,  debug(message)

Log a message with the specified severity.

Any Python module will start similar to:

  import collectd

A very simple read function might look like:

  import random

  def read(data=None):
    vl = collectd.Values(type='gauge')
    vl.plugin='python.spam'
    vl.dispatch(values=[random.random() * 100])

A very simple write function might look like:

  def write(vl, data=None):
    for i in vl.values:
      print "%s (%s): %f" % (vl.plugin, vl.type, i)

To register those functions with collectd:

  collectd.register_read(read)
  collectd.register_write(write)

See the section “Classes” above for a complete documentation of the data types used by the read, write and match functions.

• collectd is heavily multi-threaded. Each collectd thread accessing the Python plugin will be mapped to a Python interpreter thread. Any such thread will be created and destroyed transparently and on-the-fly.

  Hence, any plugin has to be thread-safe if it provides several entry points from collectd (i. e. if it registers more than one callback or if a registered callback may be called more than once in parallel).

• The Python thread module is initialized just before calling the init callbacks. This means you must not use Python's threading module prior to this point. This includes all config and possibly other callback as well.
• The python plugin exports the internal API of collectd which is considered unstable and subject to change at any time. We try hard to not break backwards compatibility in the Python API during the life cycle of one major release. However, this cannot be guaranteed at all times. Watch out for warnings dispatched by the python plugin after upgrades.

• Not all aspects of the collectd API are accessible from Python. This includes but is not limited to filters.

collectd(1), collectd.conf(5), collectd-perl(5), collectd-exec(5), types.db(5), python(1),

The python plugin has been written by Sven Trenkel <collectd at semidefinite.de>.

This manpage has been written by Sven Trenkel <collectd at semidefinite.de>. It is based on the collectd-perl(5) manual page by Florian Forster <octo at collectd.org> and Sebastian Harl <sh at tokkee.org>.

collectd.conf(5).