fennel-api - Man Page

The fennel module provides the following functions for use when embedding Fennel in a Lua program. If you're writing a pure Fennel program or working on a system that already has Fennel support, you probably don't need this.

Only the fennel module is part of the public API. The other modules are implementation details subject to change. Most functions will error upon failure.

Any time a function takes an options table argument, that table will usually accept these fields:

• allowedGlobals: a sequential table of strings of the names of globals which the compiler will allow references to. Set to false to disable checks. Defaults to the contents of the env table, if provided, or the current environment.
• correlate: when this is set, Fennel attempts to emit Lua where the line numbers match up with the Fennel input code; useful for situation where code that isn't under your control will print the stack traces. This is meant as a debugging aid and cannot give exact numbers in all cases.
• useMetadata: enables or disables metadata, allowing use of the ,doc repl command. Intended for development purposes (see performance note); defaults to true for REPL only.
• requireAsInclude: Alias any static require calls to the include special, embedding the module code inline in the compiled output. If the module name isn't a string literal that is resolvable at compile time it falls back to require at runtime. Can be used to embed both Fennel and Lua modules.
• assertAsRepl: Replace calls to assert with assert-repl to allow for interactive debugging.
• env: an environment table in which to run the code; see the Lua manual.
• compilerEnv: an environment table in which to run compiler-scoped code for macro definitions and eval-compiler calls. Internal Fennel functions such as list, sym, etc. will be exposed in addition to this table. Defaults to a table containing limited known-safe globals. Pass _G to disable sandboxing.
• unfriendly: disable friendly compiler/parser error messages.
• plugins: list of compiler plugins.
• error-pinpoint: a list of two strings indicating what to wrap compile errors in
• keywords: a table of the form {:keyword1 true :keyword2 true} containing symbols that should be treated as reserved Lua keywords.

You can pass the string "_COMPILER" as the value for env; it will cause the code to be run/compiled in a context which has all compiler-scoped values available. This can be useful for macro modules or compiler plugins.

Note that only the fennel module is part of the public API. The other modules (fennel.utils, fennel.compiler, etc) should be considered compiler internals subject to change.

If you are embedding Fennel in a context where ANSI escape codes are not interpreted, you can set error-pinpoint to false to disable the highlighting of compiler and parse errors.

fennel.repl([options])

Takes these additional options:

• readChunk(): a function that when called, returns a string of source code. Should return nil when there is no more source, which will exit the repl.
• pp: a pretty-printer function to apply on values (default: fennel.view).
• view-opts: an options table passed to pp (default: {:depth 4}).
• onValues(values): a function that will be called on all returned top level values. Takes a table of values.
• onError(errType, err, luaSource): a function that will be called on each error. errType is a string with the type of error, can be either, 'parse', 'compile', 'runtime', or 'lua'. err is the error message, and luaSource is the source of the generated lua code.

By default, metadata will be enabled and you can view function signatures and docstrings with the ,doc command in the REPL.

In Fennel 1.4.1 fennel.repl was changed from a normal function to a callable table. This mostly behaves the same, but it can cause problems with certain functions that are very picky about functions. Unfortunately this includes coroutine.create. You can pass fennel.repl.repl instead.

fennel.repl.onError = custom_error_handler
-- In rare cases this needs to be temporary, overrides
-- can be cleared by simply clearing the entire table
for k in pairs(fennel.repl) do
  fennel.repl[k] = nil
end

local result = fennel.eval(str[, options[, ...]])

The options table may also contain:

• filename: override the filename that Lua thinks the code came from.

Additional arguments beyond options are passed to the code and available as ....

local result = fennel.dofile(filename[, options[, ...]])

Additional arguments beyond options are passed to the code and available as ....

require("fennel").install().dofile("main.fnl")

This is the equivalent of this code:

local fennel = require("fennel")
table.insert(package.loaders or package.searchers, fennel.searcher)
fennel.dofile("main.fnl") -- require calls in main.fnl can load fennel modules

Normally Lua's require function only loads modules written in Lua, but you can install fennel.searcher into package.searchers (or in Lua 5.1 package.loaders) to teach it how to load Fennel code.

If you would rather change some of the options you can use fennel.makeSearcher(options) to get a searcher function that's equivalent to fennel.searcher but overrides the default options table.

The require function is different from fennel.dofile in that it searches the directories in fennel.path for .fnl files matching the module name, and also in that it caches the loaded value to return on subsequent calls, while fennel.dofile will reload each time. The behavior of fennel.path mirrors that of Lua's package.path. There is also a fennel.macro-path which is used to look up macro modules.

If you install Fennel into package.searchers then you can use the repl's ,reload mod command to reload modules that have been loaded with require.

The compiler sandbox makes it so that the module system is also isolated from the rest of the system, so the above require calls will not work from inside macros. However, there is a separate fennel.macro-searchers table which can be used to allow different modules to be loaded inside macros. By default it includes a searcher to load sandboxed Fennel modules and a searcher to load sandboxed Lua modules, but if you disable the compiler sandbox you may want to replace these with searchers which can load arbitrary modules.

The default fennel.macro-searchers functions also cannot load C modules. Here's an example of some code which would allow that to work:

table.insert(fennel["macro-searchers"], function(module_name)
  local filename = fennel["search-module"](module_name, package.cpath)
  if filename then
    local func = "luaopen_" .. module_name
    return function() return package.loadlib(filename, func) end, filename
  end
end)

Macro searchers store loaded macro modules in the fennel.macro-loaded table which works the same as package.loaded but for macro modules.

The fennel.traceback function works like Lua's debug.traceback function, except it tracks line numbers from Fennel code correctly.

If you are working on an application written in Fennel, you can override the default traceback function to replace it with Fennel's:

debug.traceback = fennel.traceback

Note that some systems print stack traces from C, which will not be affected.

The fennel.getinfo function works like Lua's debug.getinfo function, except it tracks line numbers from Fennel code correctly. Functions defined from Fennel will have the what field set to "Fennel" instead of "Lua".

local mymodule = require("module")
print(fennel.getinfo(mymodule.func1).linedefined)

The fennel.parser function returns a stateful iterator function. If a form was successfully read, it returns true followed by the AST node. Returns nil when it reaches the end. Raises an error if it can't parse the input.

local parse = fennel.parser(text)
local ok, ast = assert(parse()) -- just get the first form

-- Or use in a for loop
for ok, ast in parse do
  if ok then
    print(fennel.view(ast))
  end
end

The first argument can either be a string or a function that returns one byte at a time. It takes two optional arguments; a filename and a table of options. Supported options are both booleans that default to false:

• unfriendly: disable enhanced parse error reporting
• comments: include comment nodes in AST
• plugins: (since 1.2.0) An optional list of compiler plugins.

The list of common options at the top of this document do not apply here.

The AST returned by the parser consists of data structures representing the code. Passing AST nodes to the fennel.view function will give you a string which should round-trip thru the parser to give you the same data back. The same is true with tostring, except it does not work with non-sequence tables.

The fennel.ast-source function takes an AST node and returns a table with source data around filename, line number, et in it, if possible. Some AST nodes cannot provide this data, for instance numbers, strings, and booleans, or symbols constructed within macros using the sym function instead of backtick.

AST nodes can be any of these types:

print(fennel.searchModule("my.mod", package.path))

If you just want to find the file path that a module would resolve to without actually loading it, you can use fennel.searchModule. The first argument is the module name, and the second argument is the path string to search. If none is provided, it defaults to Fennel's own path.

Returns nil if the module is not found on the path.

The fennel.view function takes any Fennel data and turns it into a representation suitable for feeding back to Fennel's parser. In addition to tables, strings, numbers, and booleans, it can produce reasonable output from ASTs that come from the parser. It will emit an unreadable placeholder for coroutines, compiled functions, and userdata, which cannot be understood by the parser.

print(fennel.view({abc=123}[, options])
{:abc 123}

The list of common options at the top of this document do not apply here; instead these options are accepted:

• one-line? (default: false) keep the output string as a one-liner
• depth (number, default: 128) limit how many levels to go (default: 128)
• detect-cycles? (default: true) don't try to traverse a looping table
• metamethod? (default: true) use the __fennelview metamethod if found
• empty-as-sequence? (default: false) render empty tables as []
• line-length (number, default: 80) length of the line at which multi-line output for tables is forced
• byte-escape (function) If present, overrides default behavior of escaping special characters in decimal format (e.g. <ESC> -> \027). Called with the signature (byte-escape byte view-opts), where byte is the char code for a special character
• escape-newlines? (default: false) emit strings with \n instead of newline
• prefer-colon? (default: false) emit strings in colon notation when possible
• utf8? (default: true) whether to use utf8 module to compute string lengths
• max-sparse-gap (number, default: 10) maximum gap to fill in with nils in sparse sequential tables before switching to curly brackets.
• preprocess (function) if present, called on x (and recursively on each value in x), and the result is used for pretty printing; takes the same arguments as fennel.view

All options can be set to {:once some-value} to force their value to be some-value but only for the current level. After that, such option is reset to its default value. Alternatively, {:once value :after other-value} can be used, with the difference that after first use, the options will be set to other-value instead of the default value.

You can set a __fennelview metamethod on a table to override its serialization behavior. It should take the table being serialized as its first argument, a function as its second argument, options table as third argument, and current amount of indentation as its last argument:

(fn [t view options indent] ...)

view function contains a pretty printer that can be used to serialize elements stored within the table being serialized. If your metamethod produces indented representation, you should pass indent parameter to view increased by the amount of additional indentation you've introduced. This function has the same interface as __fennelview metamethod, but in addition accepts colon-string? as last argument. If colon? is true, strings will be printed as colon-strings when possible, and if its value is false, strings will be always printed in double quotes. If omitted or nil will default to value of :prefer-colon? option.

options table contains options described above, and also visible-cycle? function, that takes a table being serialized, detects and saves information about possible reachable cycle. Should be used in __fennelview to implement cycle detection.

__fennelview metamethod should always return a table of correctly indented lines when producing multi-line output, or a string when always returning single-line item. fennel.view will transform your data structure to correct multi-line representation when needed. There's no need to concatenate table manually ever - fennel.view will apply general rules for your data structure, depending on current options. By default multiline output is produced only when inner data structures contains newlines, or when returning table of lines as single line results in width greater than line-size option.

Multi-line representation can be forced by returning two values from __fennelview - a table of indented lines as first value, and true as second value, indicating that multi-line representation should be forced.

There's no need to incorporate indentation beyond needed to correctly align elements within the printed representation of your data structure. For example, if you want to print a multi-line table, like this:

@my-table[1
          2
          3]

__fennelview should return a sequence of lines:

["@my-table[1"
 "          2"
 "          3]"]

Note, since we've introduced inner indent string of length 10, when calling view function from within __fennelview metamethod, in order to keep inner tables indented correctly, indent must be increased by this amount of extra indentation.

Here's an implementation of such pretty-printer for an arbitrary sequential table:

(fn pp-doc-example [t view options indent]
  (let [lines (icollect [i v (ipairs t)]
                (let [v (view v options (+ 10 indent))]
                  (if (= i 1) v
                      (.. "          " v))))]
    (doto lines
      (tset 1 (.. "@my-table[" (or (. lines 1) "")))
      (tset (length lines) (.. (. lines (length lines)) "]")))))

Setting table's __fennelview metamethod to this function will provide correct results regardless of nesting:

>> {:my-table (setmetatable [[1 2 3 4 5]
                             {:smalls [6 7 8 9 10 11 12]
                              :bigs [500 1000 2000 3000 4000]}]
                            {:__fennelview pp-doc-example})
    :normal-table [{:c [1 2 3] :d :some-data} 4]}
{:my-table @my-table[[1 2 3 4 5]
                     {:bigs [500 1000 2000 3000 4000]
                      :smalls [6 7 8 9 10 11 12]}]
 :normal-table [{:c [1 2 3] :d "some-data"} 4]}

Note that even though we've only indented inner elements of our table with 10 spaces, the result is correctly indented in terms of outer table, and inner tables also remain indented correctly.

When using the :preprocess option or __fennelview method, avoid modifying any tables in-place in the passed function. Since Lua tables are mutable and passed in without copying, any modification done in these functions will be visible outside of fennel.view.

Using :byte-escape to override the special character escape format is intended for use-cases where it's known that the output will be consumed by something other than Lua/Fennel, and may result in output that Fennel can no longer parse. For example, to force the use of hex escapes:

(print (fennel.view {:clear-screen "\027[H\027[2J"}
                    {:byte-escape #(: "\\x%2x" :format $)}))
;; > {:clear-screen "\x1b[H\x1b[2J"}

While Lua 5.2+ supports hex escapes, PUC Lua 5.1 does not, so compiling this with Fennel later would result in an incorrect escape code in Lua 5.1.

When running a REPL or using compile/eval with metadata enabled, each function declared with fn or λ/lambda will use the created function as a key on fennel.metadata to store the function's arglist and (if provided) docstring. The metadata table is weakly-referenced by key, so each function's metadata will be garbage collected along with the function itself.

You can work with the API to view or modify this metadata yourself, or use the ,doc repl command to view function documentation.

In addition to direct access to the metadata tables, you can use the following methods:

• fennel.metadata:get(func, key): get a value from a function's metadata
• fennel.metadata:set(func, key, val): set a metadata value
• fennel.metadata:setall(func, key1, val1, key2, val2, ...): set pairs

• fennel.doc(func, fnName): print formatted documentation for function using name. Utilized by the ,doc command, name is whatever symbol you operate on that's bound to the function.

  local greet = fennel.eval('(λ greet [name] "Say hello" (print "Hello," name))',
                            {useMetadata = true})
  
  fennel.metadata[greet]
  -- > {"fnl/docstring" = "Say hello", "fnl/arglist" = ["name"]}
  
  fennel.doc(greet, "greet")
  -- > (greet name)
  -- >   Say hello
  
  fennel.metadata:set(greet, "fnl/docstring", "Say hello!!!")
  fennel.doc(greet, "greet!")
  --> (greet! name)
  -->   Say hello!!!

If you're writing a tool which performs syntax highlighting or some other operations on Fennel code, the fennel.syntax function can provide you with data about what forms and keywords to treat specially.

local syntax = fennel.syntax()
print(fennel.view(syntax["icollect"]))
--> {:binding-form? true :body-form? true :macro? true}

The table has string keys and table values. Each entry will have one of "macro?", "global?", or "special?" set to true indicating what type it is. Globals can also have "function?" set to true. Macros and specials can have "binding-form?" set to true indicating it accepts a [] argument which introduces new locals, and/or a "body-form?" indicating whether it should be indented with two spaces instead of being indented like a function call. They can also have a "define?" key indicating whether it introduces a new top-level identifier like local or fn.

This isn't Fennel-specific, but the loadCode function takes a string of Lua code along with an optional environment table and filename string, and returns a function for the loaded code which will run inside that environment, in a way that's portable across any Lua 5.1+ version.

local f = fennel.loadCode(luaCode, { x = y }, "myfile.lua")

This function does a best effort detection of the Lua VM environment hosting Fennel. Useful for displaying an "About" dialog in your Fennel app that matches the REPL and --version CLI flag.

(fennel.runtime-version)

print(fennel.runtimeVersion())
-- > Fennel 1.0.0 on PUC Lua 5.4

The fennel.version field will give you the version of just Fennel itself.

(since 1.3.1)

If an optional argument is given, returns version information as a table:

(fennel.runtime-version :as-table)
;; > {:fennel "1.3.1" :lua "PUC Lua 5.4"}

Fennel's plugin system is extremely experimental and exposes internals of the compiler in ways that no other part of the compiler does. It should be considered unstable; changes to the compiler in future versions are likely to break plugins, and each plugin should only be assumed to work with specific versions of the compiler that they're tested against. The backwards-compatibility guarantees of the rest of Fennel do not apply to plugins.

Compiler plugins allow the functionality of the compiler to be extended in various ways. A plugin is a module containing various functions in fields named after different compiler extension points. When the compiler hits an extension point, it will call each plugin's function for that extension point, if provided, with various arguments; usually the AST in question and the scope table. Each plugin function should normally do side effects and return nil or error out. If a function returns non-nil, it will cause the rest of the plugins for a given event to be skipped.

• symbol-to-expression
• call
• do
• fn
• destructure
• parse-error
• assert-compile

The destructure extension point is different because instead of just taking ast and scope it takes a from which is the AST for the value being destructured and a to AST which is the AST for the form being destructured to. This is most commonly a symbol but can be a list or a table.

The parse-error and assert-compile hooks can be used to override how fennel behaves down to the parser and compiler levels. Possible use-cases include building atop fennel.view to serialize data with EDN (https://clojure.github.io/clojure/clojure.edn-api.html)-style tagging, or manipulating external s-expression-based syntax, such as tree-sitter queries (https://tree-sitter.github.io/tree-sitter/using-parsers#query-syntax).

The scope argument is a table containing all the compiler's information about the current scope. Most of the tables here look up values in their parent scopes if they do not contain a key.

Plugins can also contain repl commands. If your plugin module has a field with a name beginning with "repl-command-" then that function will be available as a comma command from within a repl session. It will be called with a table for the repl session's environment, a function which will read the next form from stdin (ignoring newlines and other whitespace), a function which is used to print normal values, and one which is used to print errors.

(local fennel (require :fennel)
(fn locals [env read on-values on-error scope chars opts]
  "Print all locals in repl session scope."
  (on-values [(fennel.view env.___replLocals___)]))

{:repl-command-locals locals}

$ fennel --plugin locals-plugin.fnl
Welcome to Fennel 0.8.0 on Lua 5.4!
Use ,help to see available commands.
>> (local x 4)
nil
>> (local abc :xyz)
nil
>> ,locals
{
  :abc "xyz"
  :x 4
}

The docstring of the function will be used as its summary in the ",help" command listing. Unlike other plugin hook fields, only the first plugin to provide a repl command will be used.

Fennel Maintainers.