smtpd-filters - Man Page

The smtpd(8) daemon provides a Simple Mail Transfer Protocol (SMTP) implementation, which allows ordinary machines to become Mail eXchangers (MX). Some features that are commonly used by MX, such as delivery reporting or spam filtering, are outside the scope of SMTP and too complex to fit in smtpd(8).

Because an MX may need to provide these features, smtpd(8) provides an API to extend its behavior through smtpd-filters.

At runtime, smtpd(8) can report events to smtpd-filters, querying what it should answer to these events. This allows the decision logic to rely on third-party programs.

smtpd-filters are programs that run as unique standalone processes, they do not share smtpd(8) memory space. They are executed by smtpd(8) at startup and expected to run in an infinite loop, reading events and filtering requests from stdin(4), writing responses to stdout(4) and logging to stderr(4). They are not allowed to terminate.

Because smtpd-filters are standalone programs that communicate with smtpd(8) through fd(4), they may run as different users than smtpd(8) and may be written in any language. smtpd-filters must not use blocking I/O, they must support answering asynchronously to smtpd(8).

The API relies on two streams, report and filter.

The report stream is a one-way stream which allows smtpd(8) to inform smtpd-filters in real-time about events. Report events do not expect an answer from smtpd-filters; they are just meant to provide information. A filter should be able to replicate the smtpd(8) state for a session by gathering information coming from report events. No decision is ever taken by the report stream.

The filter stream is a two-way stream which allows smtpd(8) to query smtpd-filters about what it should do with a session at a given phase. Filter requests expect an answer from smtpd-filters; smtpd(8) will not let the session move forward until then. A decision must always be taken by the filter stream.

It is sometimes possible to rely on filter requests to gather information, but because a response is expected by smtpd(8), this is more costly than using report events. The correct pattern for writing filters is to use report events to create a local state for a session, then use filter requests to take decisions based on this state. The only case when using filter requests instead of report events is correct is when a decision is required for the filter request and there is no need for more information than that of the event itself.

The protocol consists of human-readable lines exchanged between smtpd-filters and smtpd(8), through fd(4).

The protocol begins with a handshake. First, smtpd(8) provides smtpd-filters with general configuration information in the form of key-value lines:

config|smtpd-version|6.6.1
config|smtp-session-timeout|300
config|subsystem|smtp-in
config|ready

Then, smtpd-filters register the stream, subsystem and event they want to handle:

register|report|smtp-in|link-connect
register|ready

Finally, smtpd(8) emits report events and filter requests, expecting smtpd-filters to respond or not depending on the stream:

report|0.5|1576146008.006099|smtp-in|link-connect|7641df9771b4ed00|mail.openbsd.org|pass|199.185.178.25:33174|45.77.67.80:25
report|0.5|1576147242.200225|smtp-in|link-connect|7641dfb3798eb5bf|mail.openbsd.org|pass|199.185.178.25:31205|45.77.67.80:25
report|0.5|1576148447.982572|smtp-in|link-connect|7641dfc063102cbd|mail.openbsd.org|pass|199.185.178.25:24786|45.77.67.80:25

The character “|” may only appear in the last field of a payload, in which case it should be considered a regular character and not a separator. No other field may contain a “|”.

The list of subsystems and events, as well as the format of requests and responses, are documented in the sections below.

During the initial handshake, smtpd(8) emits a series of configuration keys and values. The list is meant to be ignored by smtpd-filters that do not require it and consumed gracefully by filters that do.

There are currently three keys:

config|smtpd-version|6.6.1
config|smtp-session-timeout|300
config|subsystem|smtp-in

When smtpd(8) has sent all configuration keys, it emits the following line:

config|ready

There is currently only one subsystem supported in the API: smtp-in.

Each report event is generated by smtpd(8) as a single line similar to the one below:

report|0.5|1576146008.006099|smtp-in|link-connect|7641df9771b4ed00|mail.openbsd.org|pass|199.185.178.25:33174|45.77.67.80:25

The format consists of a protocol prefix containing the stream, the protocol version, the timestamp, the subsystem, the event and the unique session identifier, separated by “| :”

report|0.5|1576146008.006099|smtp-in|link-connect|7641df9771b4ed00

It is followed by a suffix containing the event-specific parameters, also separated by “| :”

mail.openbsd.org|pass|199.185.178.25:33174|45.77.67.80:25

The list of events and event-specific parameters for smtp-in are as follows:

There is currently only one subsystem supported in the API: smtp-in.

Filter requests allow smtpd(8) to query smtpd-filters about what to do with a session at a particular phase. In addition, they allow smtpd-filters to alter the content of a message by adding, modifying, or suppressing lines of input in a way that is similar to what program like sed(1) or grep(1) would do.

Each filter request is generated by smtpd(8) as a single line similar to the one below. Fields are separated by the “|” character.

filter|0.5|1576146008.006099|smtp-in|connect|7641df9771b4ed00|1ef1c203cc576e5d|mail.openbsd.org|pass|199.185.178.25:33174|45.77.67.80:25

The format consists of a protocol prefix containing the stream, the protocol version, the timestamp, the subsystem, the filtering phase, the unique session identifier and an opaque token that the filter should provide in its response:

filter|0.5|1576146008.006099|smtp-in|connect|7641df9771b4ed00|1ef1c203cc576e5d

It is followed by a suffix containing the phase-specific parameters of the filter request, also separated by “| :”

mail.openbsd.org|pass|199.185.178.25:33174|45.77.67.80:25

Unlike with report events, smtpd(8) expects answers from filter requests and will not allow a session to move forward until the filter has instructed smtpd(8) how to treat it.

For all phases except “data-line”, responses must follow the same construct: a message of type “filter-result”, followed by the unique session id, the opaque token, a decision and optional decision-specific parameters:

filter-result|7641df9771b4ed00|1ef1c203cc576e5d|proceed
filter-result|7641df9771b4ed00|1ef1c203cc576e5d|reject|550 nope

The possible decisions for a “filter-result” message are documented below.

For the “data-line” phase, smtpd-filters are fed a stream of lines corresponding to the message to filter, terminated by a single dot:

filter|0.5|1576146008.006099|smtp-in|data-line|7641df9771b4ed00|1ef1c203cc576e5d|line 1
filter|0.5|1576146008.006103|smtp-in|data-line|7641df9771b4ed00|1ef1c203cc576e5d|line 2
filter|0.5|1576146008.006105|smtp-in|data-line|7641df9771b4ed00|1ef1c203cc576e5d|.

They are expected to return an output stream similarly terminated by a single dot. A filter may add to, suppress, modify or echo back the lines it receives. Ultimately, smtpd(8) assumes that the message consists of the output from smtpd-filters.

Note that filters may be chained, and the lines that are input into a subsequent filter are the lines that are output from a previous filter.

The response to “data-line” requests use their own construct. A “filter-dataline” prefix, followed by the unique session identifier, the opaque token and the output line as follows:

filter-dataline|7641df9771b4ed00|1ef1c203cc576e5d|line 1
filter-dataline|7641df9771b4ed00|1ef1c203cc576e5d|line 2
filter-dataline|7641df9771b4ed00|1ef1c203cc576e5d|.

The list of events and event-specific parameters for smtp-in are as follows:

connect: rdns fcrdns src dest

This request is emitted after connection, before the banner is displayed.

helo: identity

This request is emitted after the client has emitted “HELO”.

ehlo: identity

This request is emitted after the client has emitted “EHLO”.

starttls: tls-string

This request is emitted after the client has requested “STARTTLS”.

auth: auth

This request is emitted after the client has requested “AUTH”.

mail-from: address

This request is emitted after the client has requested “MAIL FROM”.

rcpt-to: address

This request is emitted after the client has requested “RCPT TO”.

data

This request is emitted after the client has requested “DATA”.

data-line: line

This request is emitted for each line of input in the “DATA” phase. The lines are raw dot-escaped SMTP DATA input, terminated with a single dot.

commit

This request is emitted after the final single dot is received.

For every filtering phase, excepted “data-line”, the following decisions may be taken by a filter:

smtpd(8)

smtpd-filters first appeared in OpenBSD 6.6.

smtpd.conf(5).