perlimports - Man Page

A command line utility for cleaning up imports in your Perl code

Synopsis

Create a config file called perlimports.toml at the top level of your application or repository:

    perlimports --create-config-file perlimports.toml

You can also have a config file with a leading ..

    perlimports --create-config-file .perlimports.toml

For system-wide defaults, you can create a file in $XDG_HOME. Something like:

    perlimports --create-config-file ~/.config/perlimports/perlimports.toml

After you have set up the config file to your liking, you can do away with most command line switches, other than -i, --lint or --read-stdin.

Now, let's update a file in place. (Make sure you can revert the file if you need to, as perlimports will not make a backup.)

    perlimports -i foo.pl

Lint a file:

    perlimports --lint foo.pl

In-place edits on directories:

    perlimports -i lib t xt

Lint directories:

    perlimports --lint lib t xt

In-place edits on files and directories

    perlimports -i foo.pl lib t xt

Run perlimports on a file and only print the results to STDOUT.

    perlimports foo.pl

If some of your imported modules are in local directories, you can give some hints as to where to find them:

    perlimports -i --libs t/lib,/some/dir/lib foo.pl

Redirect output to a new file:

    perlimports foo.pl > foo.new.pl

This distribution provides the perlimports command line interface (CLI), which automates the cleanup and maintenance of Perl use and require statements. Loosely inspired by goimports <https://pkg.go.dev/golang.org/x/tools/cmd/goimports>, this tool aims to be part of your linting and tidying workflow, in much the same way you might use perltidy or perlcritic.

For a detailed discussion of the problems this tool attempts to solve, see this "Conference in the Cloud" talk from June 2021: Where did that Symbol Come From? <https://www.youtube.com/watch?v=fKqxdTbGxYY>.

Slides for the above talk are also available:

     curl -O https://raw.githubusercontent.com/oalders/presentations/main/slides/6-perlimports/remark.html && open remark.html

Motivation

Many Perl modules helpfully export functions and variables by default. These provide handy shortcuts when you're writing a quick or small script, but they can quickly become a maintenance burden as code grows organically. When code increases in complexity, it leads to greater costs in terms of development time. Conversely, reducing code complexity can speed up development. This tool aims to reduce complexity to further this goal.

While importing symbols by default or using export tags provides a convenient shorthand for getting work done, this shorthand requires the developer to retain knowledge of these defaults and tags in order to understand the code. perlimports aims to allow you to develop your code as you see fit, while still giving you a viable option of tidying your imports automatically. In much the same way as you might use perltidy to format your code, you can now automate the process of making your imports easier to understand. Let's look at some examples.

Where is this function defined?

You may come across some code like this:

    use strict;
    use warnings;

    use HTTP::Request::Common;
    use LWP::UserAgent;

    my $ua = LWP::UserAgent->new;
    my $req = $ua->request( GET 'https://metacpan.org/' );
    print $req->content;

Where does GET come from? If you're not familiar with HTTP::Request::Common, you may not realize that the statement use HTTP::Request::Common has implicitly imported the functions GET, HEAD, PUT, PATCH, POST and OPTIONS into to this block of code.

What would happen if we used perlimports to import all needed functions explicitly? It might look something like this:

    use strict;
    use warnings;

    use HTTP::Request::Common qw( GET );
    use LWP::UserAgent ();

    my $ua = LWP::UserAgent->new;
    my $req = $ua->request( GET 'https://metacpan.org/' );
    print $req->content;

The code above makes it immediately obvious where GET originates, which in turn makes it easier for us to look up its documentation. It has the added bonus of also not importing HEAD, PUT or any of the other functions which HTTP::Request::Common exports by default. So, those functions cannot unwittingly be used later in the code. This makes for more understandable code for present day you, future you and any others tasked with reading your code at some future point.

Keep in mind that this simple act can save much time for developers who are not intimately familiar with Perl and the default exports of many CPAN modules.

Are we even using all of these imports?

Imagine the following import statement

    use HTTP::Status qw(
        is_cacheable_by_default
        is_client_error
        is_error
        is_info
        is_redirect
        is_server_error
        is_success
        status_message
    );

followed by 3,000 lines of code. How do you know if all of these functions are actually being used? Were they ever used? You can grep all of these function names manually or you can remove them by trial and error to see what breaks. This is a doable solution, but it does not scale well to scripts and modules with many imports or to large code bases with many imports. Having an unmaintained list of imports is preferable to implicit imports, but it would be helpful to automate maintaining this list.

perlimports can, in many situations, clean up your import statements and automate this maintenance burden away. This makes it easier for you to write clean code, which is easier to understand.

Are we even using all of these modules?

In cases where code is implicitly importing from modules or where explicit imports are not being curated, it can be hard to discover which modules are no longer being used in a script, module or even a code base. Removing unused modules from code can lead to gains in performance and decrease in consumption of resources. Removing entire modules from your code base can decrease the number of dependencies which you need to manage and decrease friction in your your deployment process.

perlimports can remove unused modules for you, making dependency management much easier.

Enforcing a consistent style

Having a messy list of module imports makes your code harder to read. Imagine this:

    use Cpanel::JSON::XS;
    use Database::Migrator::Types qw( HashRef ArrayRef Object Str Bool Maybe CodeRef FileHandle RegexpRef );
    use List::AllUtils qw( uniq any );
    use LWP::UserAgent    q{};
    use Try::Tiny qw/ catch     try /;
    use WWW::Mechanize  q<>;

perlimports turns the above list into:

    use Cpanel::JSON::XS ();
    use Database::Migrator::Types qw(
        ArrayRef
        Bool
        CodeRef
        FileHandle
        HashRef
        Maybe
        Object
        RegexpRef
        Str
    );
    use List::AllUtils qw( any uniq );
    use LWP::UserAgent ();
    use Try::Tiny qw( catch try);
    use WWW::Mechanize ();

Where possible, perlimports will enforce a consistent style of parentheses and will also sort your imports and break up long lines. As mentioned above, if some imports are no longer in use, perlimports will helpfully remove these for you.

Import tags

Import tags may obscure where symbols are coming from. While import tags provide a useful shorthand, they can contribute to code complexity by obscuring the origin of imported symbols. Consider:

    use HTTP::Status qw(:constants :is status_message);

The above line imports the status_message() function as well *some other things* via :constants and :is. What exactly are these things? We'll need to read the documentation to know for sure.

perlimports can audit your code and expand the line above to list the symbols which you are actually importing. So, the line above might now look something like:

    use HTTP::Status qw(
        HTTP_ACCEPTED
        HTTP_BAD_REQUEST
        HTTP_CONTINUE
        HTTP_I_AM_A_TEAPOT
        HTTP_MOVED_PERMANENTLY
        HTTP_NO_CODE
        HTTP_NOT_FOUND
        HTTP_OK
        HTTP_PAYLOAD_TOO_LARGE
        HTTP_PERMANENT_REDIRECT
        HTTP_RANGE_NOT_SATISFIABLE
        HTTP_REQUEST_ENTITY_TOO_LARGE
        HTTP_REQUEST_RANGE_NOT_SATISFIABLE
        HTTP_REQUEST_URI_TOO_LARGE
        HTTP_TOO_EARLY
        HTTP_UNORDERED_COLLECTION
        HTTP_URI_TOO_LONG
        is_cacheable_by_default
        is_client_error
        is_error
        is_info
        is_redirect
        is_server_error
        is_success
        status_message
    );

This is more verbose, but grepping your code will now reveal to you where something like is_cacheable_by_default gets defined. You have increased the lines of code, but you have also reduced complexity.

Command Line Parameters

--create-config-file

Expects a path to a .toml file which does not yet exist.

    --create-config-file perlimports.toml
    --create-config-file .perlimports.toml

For system-wide defaults, you can create a file in $XDG_HOME. Something like:

    perlimports --create-config-file ~/.config/perlimports/perlimports.toml

All of the above are default file locations which perlimports will search, unless --config-file has been provided as an argument.

--config-file

Path to a perlimports config file. If this parameter is not supplied, we will look for a file called perlimports.toml or .perlimports.toml in the current directory and then look for a perlimports.toml in XDG_CONFIG_HOME (usually something like $HOME/perlimports/perlimports.toml). This behaviour can be disabled via --no-config-file as described below.

--no-config-file

Prevents perlimports from automatically using a config file which is in one of the standard locations. Use this option if you want to define all of your behaviours on the command line.

--filename|-f

The absolute or relative path to a file (or directory) to process.

    --filename path/to/file

    -f path/to/file

    -f path/to/dir

Note that if you do not provide a --filename we will fall back to checking @ARGV for any remaining args. So,

    perlimports --filename path/to/file

is equivalent to

    perlimports path/to/file

You may also pass multiple file and dir names.

    perlimports path/to/file path/to/other/file lib t xt

--ignore-modules

A comma-separated list of module names which should be ignored by this script. Any modules in this list should remain unchanged after processing.

    --ignore-modules Foo,Foo::Bar

--ignore-modules-filename

The absolute or relative path to a file which contains a lost of module names to ignore. (See above for behaviour). The pattern is one module name per line.

    Foo
    Foo::Bar

--ignore-modules-pattern

A regular expression to match module names which should be ignored by this script. Any modules matched by this regular expression remain unchanged after processing.

    --ignore-modules-pattern '^(Foo|Foo::Bar)'

--ignore-modules-pattern-filename

The absolute or relative path to a file which contains a list of regular expression that matches modules that should be ignored. (See above for behaviour). The pattern is one regular expression per line.

    ^Foo
    ^Foo::Bar

--never-export-modules

A comma-separated list of module names which should never export symbols. If these modules are found, we will ensure that they have an empty import list. So, use Foo; becomes use Foo ();.

    --never-export-modules Foo,Foo::Bar

--never-export-modules-filename

The absolute or relative path to a file which contains a lost of module names which should never export symbols. (See above for behaviour). The pattern is one module name per line.

    Foo
    Foo::Bar

--inplace-edit|-i

Edit the file in place rather than printing the result to STDOUT. Make sure you have a backup copy first.

    --inplace--edit
    -i

Edit the file in place rather than printing the result to STDOUT. Make sure you have a backup copy first.

--json

(Experimental). If enabled, linting errors will be emitted as JSON objects with one object per error. This is intended to make it easier for editors to parse line numbers and column numbers as well as the error message. This flag can only be used in tandem with --lint.

--lint

If this argument is passed, perlimports will act as a linter, rather than a tidier. Failure (and success) messages will be reported on STDERR. Failures will also return a non-zero exit code.

This is still a bit experimental, so the output format and exit codes could change in a subsequent release.

This cannot be combined with tidying. So, passing --lint --inplace-edit will generate an error.

--[no-]padding

--padding is enabled by default, so you only need to pass this arg if you want to be explicit. This setting adds whitespace inside the parentheses.

    # --padding
    use Foo qw( bar baz );

The --no-padding arg allows you to disable the additional padding inside parentheses.

    # --no-padding
    use Foo qw(bar baz);

--[no-]tidy-whitespace

--tidy-whitespace is enabled by default. This means that use statements will be updated even when the only change is in whitespace. Disabling this can help reduce the churn involved when running perlimports, especially if the codebase does not have automated tidying.

If you have changed from --padding to --no-padding or vice versa, you'll probably want to ensure that --tidy-whitespace has also been enabled so that you can see the whitespace changes.

--libs

A comma separated list of module directories which are not in your @INC

    --libs lib,t/lib

--[no-]preserve-duplicates

When enabled, only one use statement per module will be preserved. Defaults to preserving duplicate statements.

For example, when enabled the following text

    use Foo qw( bar );
    use Foo qw (baz );

will be converted to:

    use Foo qw( bar baz );

If left disabled, the above will probably be converted to:

    use Foo qw( bar baz );
    use Foo qw( bar baz );

This allows you to determine manually how you'd like to handle the imports in question. Use this setting with care.

--[no-]preserve-unused

When enabled, unused modules will be removed. Defaults to preserving unused modules.

Enabling this may remove modules which are only present for the purposes of preloading or which aren't being detected for other reasons, so use this setting with care.

--range-begin

Experimental. First line of a range to tidy or lint. The line ranges begin at 1 (not 0). This will select the appropriate range of lines from STDIN. Requires --read-stdin. Mostly useful for editors. Unless you're writing an editor plugin or extension, you can probably ignore this.

--range-end

Experimental. Last line of a range to tidy or lint. The line ranges begin at 1 (not 0). This will select the appropriate range of lines from STDIN. Requires --read-stdin. Mostly useful for editors. Unless you're writing an editor plugin or extension, you can probably ignore this.

--read-stdin

Read statements to process from STDIN rather than processing the entire file. This is intended for use by editors, like vim. See the vim heading below for more information on how to set up an integration with your editor.

If this option is enabled, then --inplace-edit|-i is not available.

    --read-stdin

--log-level|-l

Generally only useful for debugging. notice notifies about progress, like which file or snippet is currently being processed. info will generally log the errors which were swallowed as text was being processed. All levels are subject to change.

    --log-level notice
    --log-level info
    -l notice
    -l info

See <https://metacpan.org/pod/Log::Dispatch#LOG-LEVELS> for a list of available log levels. Log output defaults to STDERR. See --log-filename if you'd rather log to a file.

--log-filename

Name of a file to redirect logs to, rather than STDERR.

--help

Output a concise help menu, with a summary of available parameters.

    --help

--verbose-help

Include the Synopsis section from this page after printing the --help menu listed above.

Annotations/Ignoring Modules

Aside from the documented command line switches for ignoring modules, you can add annotations in your code.

    use Encode; ## no perlimports

The above will tell perlimports not to attempt a tidy of this line.

    ## no perlimports
    use Encode;
    use Cpanel::JSON::XS;
    ## use perlimports

    use POSIX ();

The above will tell perlimports not to tidy the two modules contained inside of the annotations.

Please note that since perlimports needs to know as much as possible about what's going on in a file, the annotations don't prevent modules from being loaded. It's only a directive to leave the lines in the file unchanged after processing.

Integrations

You are encouraged to make this tool part of your automated tidying workflow. Some guidance on how to configure this follows.

Vim

If you're a vim user, you can pipe your import statements to perlimports directly.

    :vnoremap <silent> im :!perlimports --read-stdin --filename '%:p'<CR>

The above statement will allow you to visually select one or more lines of code and have them updated in place by perlimports. Once you have selected the code enter im to have your imports (re)formatted.

VIM and ALE

If you use ALE with vim, you can add something like this to your vim configuration. Note that this function will save your buffer before running perlimports.

    function! Perlimports(buffer) abort
      write
      return {
      \   'command': 'perlimports --read-stdin -f %s'
      \}
    endfunction

    let ale_fixers.perl = ['perlimports', 'perltidy']
    execute ale#fix#registry#Add('perlimports', 'Perlimports', ['perl'], 'Tidy Perl imports')

precious

If you're a <https://github.com/houseabsolute/precious> user, your configuration might look something like this:

    exclude = [
        # Used by Dist::Zilla
        ".build",
        "App-perlimports-*",
        "blib",
        "inc",
        "test-data",
        # All of these are generated by Dist::Zilla
        "t/00-*",
        "t/author-*",
        "t/release-*",
        "xt/author",
        "xt/release",
    ]

    [commands.perlimports]
    type = "both"
    include = [ "**/*.{pl,pm,t,psgi}" ]
    cmd = [ "perlimports" ]
    lint_flags = [ "--lint"]
    tidy_flags = [ "-i"]
    ok_exit_codes = 0
    expect_stderr = true

    [commands.perltidy]
    type = "both"
    include = [ "**/*.{pl,pm,t,psgi}" ]
    cmd = [ "perltidy", "--profile=$PRECIOUS_ROOT/perltidyrc" ]
    lint_flags = [ "--assert-tidy", "--no-standard-output", "--outfile=/dev/null" ]
    tidy_flags = [ "--backup-and-modify-in-place", "--backup-file-extension=/" ]
    ok_exit_codes = 0
    lint_failure_exit_codes = 2
    expect_stderr = true

Note that <https://github.com/houseabsolute/precious> runs plugins in order, so we've placed a perltidy config after perlimports. This is handy because perlimports could introduce changes which will later be reverted by perltidy. By running them sequentially we can avoid false positives which might be generated by perlimports changing an include which perltidy might revert.

For an up to date example, see the config file which this repository uses: <https://github.com/oalders/App-perlimports/blob/main/precious.toml>

Code::TidyAll

If you're a Code::TidyAll user, you can configure perlimports as a GenericTransformer. Your configuration might look something like this:

    [GenericTransformer perlimports]
    select = **/*.{pl,pm,t,psgi}
    ignore = .build/**/*
    ignore = App-perlimports-*/**/*
    ignore = blib/**/*
    ignore = fatlib/**/*
    ignore = inc/**/*
    ignore = t/00-*
    ignore = t/author-*
    ignore = t/release-*
    ignore = t/zzz-*
    ignore = test-data/**/*
    ignore = xt/**/*
    ignore = xt/author/{pod-coverage,pod-spell,tidyall}.t
    argv = --libs lib,t/lib --no-preserve-duplicates --no-preserve-unused --log-filename /tmp/perlimports.txt --log-level debug
    cmd = perlimports
    file_flag = -f
    ok_exit_codes = 0
    weight = 1

Note that in this case we've set the lowest possible weight. This is because we want perlimports to run before any other plugin which may transform the file. For example, you'll want perltidy to run after perlimports to avoid having to re-tidy files after your use statements have been rewritten.

If you want to use tidyall to run just perlimports you'll need to do something like:

    tidyall --plugin "GenericTransformer perlimports" -a

For an up to date example, see the config file which this repository uses: <https://github.com/oalders/App-perlimports/blob/main/tidyall.ini>

Recipes

Included are some examples to use if you want to employ your own file finding logic or prefer not to use config files. If this does not apply to you, you can safely skip this section.

Running perlimports on test files

    find t -type f |              \
      grep .t$ |                  \
      xargs perlimports           \
      --libs lib,t/lib            \
      --ignore-modules Test::More \
      --no-preserve-unused        \
      --no-preserve-duplicates    \
      --log-level debug           \
      -i

The above command:

finds all test files in ./t
pipes them to perlimports
adds lib and t/lib to @INC
ignores the Test::More module
removes unused modules
removes duplicated use statements
displays debugging info
edits files in place (-i)

Users of ack can make this a touch simpler:

      ack -f --perltest           \
      xargs perlimports           \
      --libs lib,t/lib            \
      --ignore-modules Test::More \
      --no-preserve-unused        \
      --no-preserve-duplicates    \
      --log-level debug           \
      -i

Running perlimports on modules:

    find lib -type f |         \
      grep .pm$ |              \
      xargs perlimports        \
      --libs lib               \
      --no-preserve-unused     \
      --no-preserve-duplicates \
      -i