jello - Man Page

Filter JSON and JSON Lines data with Python syntax

Examples (TL;DR)

Synopsis

Jello is similar to jq in that it processes JSON and JSON Lines data except jello uses standard python dict and list syntax.

JSON or JSON Lines can be piped into jello (JSON Lines are automatically slurped into a list of dictionaries) and are available as the variable `_`. Processed data can be output as JSON, JSON Lines, bash array lines, or a grep-able schema.

Usage

cat data.json | jello [Options] [QUERY | -q <query_file>]

jello [Options] [QUERY | -q <query_file>] [-f <input_files>]

QUERY is optional and can be most any valid python code. Alternatively, a query file can be specified with `-q` to load the query from a file. Within the query, `_` is the sanitized JSON from STDIN or the specified input file(s) (via the `-f` option) presented as a python dict or list of dicts.

If QUERY or a query file is omitted then the original JSON input will simply be pretty printed.

You can use dot notation or traditional python bracket notation to access key names.

Note: Reserved key names that cannot be accessed using dot notation can be accessed via standard python dictionary notation. (e.g. _.foo["get"] instead of _.foo.get)

A simple query:

$ cat data.json | jello _.foo

or

$ cat data.json | jello '_["foo"]'

or

$ jello _.foo -f data.json

Options

-c compact print JSON output instead of pretty printing

-C force color output even when using pipes (overrides -m and the NO_COLOR env variable)

-e empty data (don't process data from STDIN or file)

-f load input data from JSON file or JSON Lines files (must be the final option, if used)

-i initialize environment with a custom config file

-l lines output (suitable for bash array assignment)

-m monochrome output

-n print selected null values

-q load query from a file

-r raw output of selected strings (no quotes)

-s print the JSON schema in grep-able format

-t print type annotations in schema view

-h help

-v version info

Simple Examples

Jello simply pretty prints the JSON if there are no options passed:

$ echo '{"foo":"bar","baz":[1,2,3]}' | jello
{
  "foo": "bar",
  "baz": [
    1,
    2,
    3
  ]
}

If you prefer compact output, use the -c option:

$ echo '{"foo":"bar","baz":[1,2,3]}' | jello -c
{"foo":"bar","baz":[1,2,3]}

Use the -l option to convert lists/arrays into lines:

$ echo '{"foo":"bar","baz":[1,2,3]}' | jello -l _.baz
1
2
3

The -l option also allows you to create JSON Lines:

$ echo '[{"foo":"bar","baz":[1,2,3]},{"fiz":"boo","buz":[4,5,6]}]' | jello -l
{"foo":"bar","baz":[1,2,3]}
{"fiz":"boo","buz":[4,5,6]}

You can print a grep-able schema by using the -s option:

$ echo '{"foo":"bar","baz":[1,2,3]}' | jello -s
_ = {};
_.foo = "bar";
_.baz = [];
_.baz[0] = 1;
_.baz[1] = 2;
_.baz[2] = 3;

Assigning Results to a Bash Array

Use the -l option to print JSON array output in a manner suitable to be assigned to a bash array. The -r option can be used to remove quotation marks around strings. If you want null values to be printed as null, use the -n option, otherwise they are printed as blank lines.

Bash variable:

variable=($(cat data.json | jello -rl _.foo))

Bash array variable:

variable=()
while read -r value; do
    variable+=("$value")
done < <(cat data.json | jello -rl _.foo)

Examples

Printing the Grep-able Schema

$ jc -a | jello -s
_ = {};
_.name = "jc";
_.version = "1.17.2";
_.description = "JSON CLI output utility";
_.author = "Kelly Brazil";
_.author_email = "kellyjonbrazil@gmail.com";
_.website = "https://github.com/kellyjonbrazil/jc";
_.copyright = "(C) 2019-2021 Kelly Brazil";
_.license = "MIT License";
_.parser_count = 80;
_.parsers = [];
...

Printing the Grep-able Schema with Type Annotations

$ jc -a | jello -st
_ = {};                                               //  (object)
_.name = "jc";                                        //  (string)
_.version = "1.17.2";                                 //  (string)
_.description = "JSON CLI output utility";            //  (string)
_.author = "Kelly Brazil";                            //  (string)
_.author_email = "kellyjonbrazil@gmail.com";          //  (string)
_.website = "https://github.com/kellyjonbrazil/jc";   //  (string)
_.copyright = "(C) 2019-2021 Kelly Brazil";           //  (string)
_.license = "MIT License";                            //  (string)
_.parser_count = 80;                                  //  (number)
_.parsers = [];                                       //   (array)
...

Printing the JSON Structure

$ jc dig example.com | jello -st | grep '(object)\|(array)'
_ = [];                                               //   (array)
_[0] = {};                                            //  (object)
_[0].flags = [];                                      //   (array)
_[0].opt_pseudosection = {};                          //  (object)
_[0].opt_pseudosection.edns = {};                     //  (object)
_[0].opt_pseudosection.edns.flags = [];               //   (array)
_[0].question = {};                                   //  (object)
_[0].answer = [];                                     //   (array)
_[0].answer[0] = {};                                  //  (object)
...

Lambda Functions and Math

$ echo '{"t1":-30, "t2":-20, "t3":-10, "t4":0}' | jello '\
keys = _.keys()
vals = _.values()
cel = list(map(lambda x: (float(5)/9)*(x-32), vals))
dict(zip(keys, cel))'
{
  "t1": -34.44444444444444,
  "t2": -28.88888888888889,
  "t3": -23.333333333333336,
  "t4": -17.77777777777778
}
$ jc -a | jello 'len([entry for entry in _.parsers if "darwin" in entry.compatible])'
45

For Loops

Output as JSON array

$ jc -a | jello '\
result = []
for entry in _.parsers:
  if "darwin" in entry.compatible:
    result.append(entry.name)
result'
[
  "airport",
  "airport_s",
  "arp",
  "crontab",
  "crontab_u",
  ...
]

Output as bash array

$ jc -a | jello -rl '\
result = []
for entry in _.parsers:
  if "darwin" in entry.compatible:
    result.append(entry.name)
result'
airport
airport_s
arp
crontab
crontab_u
...

List and Dictionary Comprehension

Output as JSON array

$ jc -a | jello '[entry.name for entry in _.parsers if "darwin" in entry.compatible]'
[
  "airport",
  "airport_s",
  "arp",
  "crontab",
  "crontab_u",
  ...
]

Output as bash array

$ jc -a | jello -rl '[entry.name for entry in _.parsers if "darwin" in entry.compatible]'
airport
airport_s
arp
crontab
crontab_u
...

Environment Variables

$ echo '{"login_name": "joeuser"}' | jello '\
True if os.getenv("LOGNAME") == _.login_name else False'
true

Using 3rd Party Modules

You can import and use your favorite modules to manipulate the data. For example, using glom:

$ jc -a | jello '\
from glom import *
glom(_, ("parsers", ["name"]))'
[
  "airport",
  "airport_s",
  "arp",
  "blkid",
  "crontab",
  "crontab_u",
  "csv",
  ...
]

Advanced Usage

Custom Configuration File

You can use the -i option to initialize the jello environment with your own configuration file. The configuration file accepts valid python code where you can enable/disable jello options, customize your colors, add import statements for your favorite modules, and define your own functions.

The file must be named .jelloconf.py and must be located in the proper directory based on the OS platform:

Linux, unix, macOS: ~/

Windows: %appdata%/

Setting Options

To set jello options in the .jelloconf.py file, import the jello.lib.opts class, add any of the following and set to True or False:

from jello.lib import opts
opts.mono = True            # -m option
opts.compact = True         # -c option
opts.lines = True           # -l option
opts.raw = True             # -r option
opts.force_color = True     # -C option
opts.nulls = True           # -n option
opts.schema = True          # -s option
opts.types = True           # -t option

Setting Colors

You can customize the colors by importing the jello.lib.opts class and setting the following variables to one of the following string values: black, red, green, yellow, blue, magenta, cyan, gray, brightblack, brightred, brightgreen, brightyellow, brightblue, brightmagenta, brightcyan, or white.

from jello.lib import opts
opts.keyname_color = 'blue'            # Key names
opts.keyword_color = 'brightblack'     # true, false, null
opts.number_color = 'magenta'          # integers, floats
opts.string_color = 'green'            # strings

Note: Any colors set via the JELLO_COLORS environment variable will take precedence over any color values set in the .jelloconf.py configuration file

Importing Modules

To import a module (e.g. glom) during initialization, just add the import statement to your .jelloconf.py file:

from glom import *

Then you can use glom in your jello filters without importing:

$ jc -a | jello -i 'glom(_, "parsers.25.name")'
"lsblk"

Adding Functions

You can also add functions to your initialization file. For example, you could simplify glom use by adding the following function to .jelloconf.py:

def g(query):
    import glom
    return glom.glom(_, query)

Then you can use the following syntax to filter the JSON data:

$ jc -a | jello -i 'g("parsers.6.compatible")'
[
  "linux",
  "darwin",
  "cygwin",
  "win32",
  "aix",
  "freebsd"
]

Or create names for commonly used queries:

def darwin_compatible():
    result = []
    for entry in _.parsers:
      if "darwin" in entry.compatible:
        result.append(entry.name)
    return result

Then use the predefined query like so:

$ jc -a | jello -i 'darwin_compatible()'
[
  "airport",
  "airport-s",
  "arp"
]

Setting Custom Colors via Environment Variable

In addition to setting custom colors in the .jelloconf.py initialization file, you can also set them via the JELLO_COLORS environment variable. Any colors set in the environment variable will take precedence over any colors set in the initialization file.

The JELLO_COLORS environment variable takes four comma separated string values in the following format:

JELLO_COLORS=<keyname_color>,<keyword_color>,<number_color>,<string_color>

Where colors are: black, red, green, yellow, blue, magenta, cyan, gray, brightblack, brightred, brightgreen, brightyellow, brightblue, brightmagenta, brightcyan, white, or default

For example, to set to the default colors:

JELLO_COLORS=blue,brightblack,magenta,green

or

JELLO_COLORS=default,default,default,default

Disable Colors via Environment Variable

You can set the NO_COLOR environment variable to any value to disable color output in jello. Note that using the -C option to force color output will override both the NO_COLOR environment variable and the -m option.

Author

Kelly Brazil (kellyjonbrazil@gmail.com)

https://github.com/kellyjonbrazil/jello

Info

2022-06-26 1.6.0 Jello JSON Filter