Using the Python 'click' library's parser as a simple function
Using 'click' as more than just a command line entry point
The click library is a fantastic utility for command line parsing in Python. While I think it suffers from scope creep, it's a much better and more modern design than Python's own argparse. I should note that I feel some remorse about argparse being in the Python standard library, since I was probably the main core developer pushing for argparse's inclusion. But that's a story for another day.
I have a need to write a function that parses partial command lines, but I just want a normal function that takes an arg-like parameter list and returns the parsed values. The problem is that normally when click calls your program's entry point, it catches all click-related exceptions (which I don't care about) and automatically exits the script (which I do very much care about!).
The background for this is that True Blade has a comprehensive managed file transfer service which we use extensively for our FinTech clients. The service is written in Python, and has a plug-in system that allows for easy and powerful configuration. Over time I've been modifying the system to use click instead of argparse. I need to have the main program pass unused command line parameters to the plugins, and have the plugins process them. The general functionality I'm using is click's ignore_unknown_options feature.
This solves half of my problem: I now have the options that I wish to pass through to my plugin. But how do I write the plugin's option parser? You'll note that the click documentation for ignore_unknown_options doesn't address this: it just passes the parameters off to an external command, so click itself doesn't need to parse them.
Notice that in my case, while both the main program and the plugins are written in Python, they're completely decoupled from each other. In fact, in production I have a hybrid system: the main program is using click, and the plugins are using argparse. While I could run this way forever, I really want to remove argparse from all of our code, for a number of reasons. Not the least of which is that argparse has a problem with options with arguments that begin with dashes, as this decade+ old bug report shows. This behavior is deeply baked in to argparse, and isn't going to change.
So, how do I write the plugin code? Say that for a given plugin, I just want it to take an integer option: --max-age. I could write:
@click.command() @click.option('--max-age', type=int) def get_args(max_age): return max_age
But the problem is when I call get_args(extra_args), click will exit my program! This is not what I want, needless to say.
In solving this problem I went off on an elaborate tangent of subclassing click.Command, writing wrapper decorators, and a variety of other machinations involving meta programming. And in the end I was very pleased with myself for getting it all to work. I was so pleased that I started writing a library that I was going to post to PyPI and I wrote an earlier version of this blog post that described my solution as filling an obvious void in click.
And then, when I was putting the finishing touches on that blog post, I was looking through click's source code one last time. And I realized that click already has a solution to this problem: it's just a matter of specifying standalone_mode when invoking the plugin's option parser. So now I call get_args(extra_args, standalone_mode=False) and I've achieved my goal: A normal function that parses my extra arguments. No meta-prgramming involved!
In retrospect it should have been obvious to me that click has some way of providing this functionality. Otherwise it would be all but impossible to test click itself. I think standalone_mode is under-documented and under-appreciated. I'm hoping that this blog post exposes it to a larger audience that can leverage click to a greater extent.