# -*- coding: utf-8 -*-
from distutils.core import setup

modules = \
['pyfil']
entry_points = \
{'console_scripts': ['pyfil = pyfil:main', 'rep = pyfil:main']}

setup_kwargs = {
    'name': 'pyfil',
    'version': '1.9.2',
    'description': 'Python one-liners in the shell in the spirit of Perl and AWK',
    'long_description': 'pyfil\n=====\nPython one-liners in the spirit of Perl and AWK.\n\n``pyfil`` stands for PYthon FILter. One of the tenants of the `Unix\ndesign`_ is that every program is a filter. It\'s especially obvious of\nprograms, like ``grep``, ``sed``, ``sort``, ``tr``, etc.\n\nOne notable example is ``awk`` -- a Turing-complete, interpreted\nlanguage for parsing text. While AWK scripts are still in use and it\'s a\nfine language, it has been superseded for parsing scripts by more\ngeneral languages like Perl and later Python and Ruby. However, AWK was\ndesigned to be especially useful in the shell as a filter, and it is\nstill in very commonly used for that today (in part because it is on\nevery \\*nix system, but also because it\'s great at what it does). AWK is\nable to be any arbitrary text filter that doesn\'t come as a coreutil.\n``perl -e`` is also quite good as a filter, and Ruby has made a valiant\nattempt to do so as well.\n\nWhile python does have a few good one-line uses (``python -m\nhttp.server``), some elements of its design make it less suited than the\nafore-mentioned languages. ``pyfil`` is one of several attempts to\naddress this issue. In particular, it takes a lot of queues in the\ndesign of its CLI from AWK and Perl, and aims fundamentally to be a\ncapable text filter, though it will evaluate any arbitrary Python\nexpression and print its value (with modules being imported implicitly\nas required).\n\nAs a more modern touch, it also has a special emphasis on\ninteroperability with JSON. If the return value of the evaluated\nexpression is a container type, python will attempt to serialize it as\nJSON before printing, so you can pipe output into other tools that deal\nwith JSON, store it to a file for later use, or send it over http. This,\ncombined with the ability to read JSON from stdin (with --json) make\n``pyfil`` a good translator between the web, which tends to speak JSON\nthese days, and the POSIX environment, which tends to think about data\nin terms of lines in a file (frequently with multiple fields per line).\n\npyfil is in pypi (i.e. you can get it easily with pip, if you want)\n\nnote:\n  pyfil has only been tested with python3, and only has wheels available\n  for python3\n\n.. _unix design: https://en.wikipedia.org/wiki/Unix_philosophy\n\n.. contents::\n\nsimilar projects\n----------------\npyfil ain\'t the first project to try something like this. Here are some\nother cracks at this problem:\n\n- oneliner_\n- pyp_\n- pyle_\n- funcpy_\n- red_\n- pyeval_\n- quickpy_\n\nDon\'t worry. I\'ve stolen some of their best ideas already, and I will go\non stealing as long as it takes!\n\n.. _oneliner: http://python-oneliner.readthedocs.io/en/latest/\n.. _pyp: http://code.google.com/p/pyp\n.. _pyle: https://github.com/aljungberg/pyle\n.. _funcpy: http://www.pixelbeat.org/scripts/funcpy\n.. _red: https://bitbucket.org/johannestaas/red\n.. _pyeval: https://bitbucket.org/nejucomo/pyeval/wiki/Home\n.. _quickpy: https://github.com/slezica/quick-py\n\nusage\n-----\n\n.. code::\n\n pyfil [-h] [-l] [-x] [-q] [-j] [-o] [-b PRE] [-e POST] [-s] [-F PATTERN]\n       [-n STRING] [-R] [-S] [-H EXCEPTION_HANDLER]\n       expression [expression ...]\n\npositional arguments:\n  expression            expression(s) to be executed. If multiple expression\n                        arguments are given, and --exec is not used, the value\n                        of the previous expression is available as \'x\' in the\n                        following expression. if --exec is used, all\n                        assignment must be explicit.\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -l, --loop            for n, i in enumerate(stdin): expressions\n  -x, --exec            use exec instead of eval. statements are allowed, but\n                        automatic printing is lost. doesn\'t affect --post\n  -q, --quiet           suppress automatic printing. doesn\'t affect --post\n  -j, --json            load stdin as json into object \'j\'; If used with\n                        --loop, treat each line of stdin as a new object\n  -J, --real-dict-json  like -j, but creates real dictionaries instead of the\n                        wrapper that allows dot syntax.\n  -o, --force-oneline-json\n                        outside of loops and iterators, objects serialzed to\n                        json print with two-space indent. this forces this\n                        forces all json objects to print on a single line.\n  -b PRE, --pre PRE     statement to evaluate before expression args. multiple\n                        statements may be combined with \';\'. no automatic\n                        printing\n  -e POST, --post POST  expression to evaluate after the loop. always handeled\n                        by eval, even if --exec, and always prints return\n                        value, even if --quiet. implies --loop\n  -s, --split           split lines from stdin on whitespace into list \'f\'.\n                        implies --loop\n  -F PATTERN, --field-sep PATTERN\n                        regex used to split lines from stdin into list \'f\'.\n                        implies --loop\n  -n STRING, --join STRING\n                        join items in iterables with STRING\n  -R, --raise-errors    raise errors in evaluation and stop execution\n                        (default: print message to stderr and continue)\n  -S, --silence-errors  suppress error messages\n  -H EXCEPTION_HANDLER, --exception-handler EXCEPTION_HANDLER\n                        specify exception handler with the format \'Exception:\n                        alternative expression to eval\'\n\navailable objects\n~~~~~~~~~~~~~~~~~\n``pyfil`` automatically imports any modules used in expressions.\n\nIf you\'d like to create any other objects to use in the execution\nenvironment ~/.config/pyfil-env.py and put things in it.\n\ndefault objects:\n\n- l = []\n- d = {}\n\nThese are empty containers you might wish to add items to during\niteration, for example.\n\n- x is the value of the previous expression unless --exec was used.\n\nThe execution environment also has a special object for stdin,\ncreatively named ``stdin``. This differs from sys.stdin in that it\nstrips trailing newlines when you iterate over it, and it has\na property, ``stdin.l``, which returns a list of the lines (without\nnewlines). If you do want the newlines, access sys.stdin directly.\n\nstdin inherits the rest of its methods from sys.stdin, so you can use\nstdin.read() to get a string of all lines, if that\'s what you need.\n\nCertain other flags may create additional objects in the evaluation\ncontext.\n\n- --loop (or anything that implies --loop) create ``n`` and ``i``.\n- --json creates ``j``.\n- --split or --field_sep create ``f``\n  \nCheck the flag descriptions for further details.\n\noutput\n~~~~~~\nautomatic printing\n..................\nBy default, pyfil prints the return value of expressions. Different\ntypes of objects use different printing conventions.\n\n- ``None`` does not print (as in the REPL)\n- strings are sent directly to to ``print()``\n- iterators (not other iterables) print each item on a new line.\n- other objects are serialized as json. If an object cannot be\n  serialized as json, it is sent directly to print().\n- all of these are overridden by --join\n\nIterators will also try to serialize each returned object as json if\nthey are not strings. json objects will be indented if only one object\nis being printed. If --loop is set or several of objects are being\nserialzed from an iterator, it will be one object per-line.\n--force-oneline-json extends this policy to printing single json objects\nas well.\n\nexamples:\n\n.. code:: bash\n\n  $ # None gets skipped\n  $ pyfil None\n  $ # strings and numbers just print\n  $ pyfil sys.platfrom\n  linux\n  $ pyfil math.pi\n  3.141592653589793\n  $ # objects try to print as json\n  $ pyfil sys.path\n  [\n    "/home/ninjaaron/.local/bin",\n    "/usr/lib/python35.zip",\n    "/usr/lib/python3.5",\n    "/usr/lib/python3.5/plat-linux",\n    "/usr/lib/python3.5/lib-dynload",\n    "/home/ninjaaron/.local/lib/python3.5/site-packages",\n    "/usr/lib/python3.5/site-packages"\n  ]\n  $ pyfil \'{i: n for n, i in enumerate(sys.path)}\'\n  {\n    "/usr/lib/python3.5/plat-linux": 3,\n    "/usr/lib/python35.zip": 1,\n    "/usr/lib/python3.5": 2,\n    "/usr/lib/python3.5/lib-dynload": 4,\n    "/usr/lib/python3.5/site-packages": 6,\n    "/home/ninjaaron/.local/lib/python3.5/site-packages": 5,\n    "/home/ninjaaron/.local/bin": 0\n  }\n  $ # unless they can\'t\n  $ pyfil \'[list, print, re]\'\n  [<class \'list\'>, <built-in function print>, <module \'re\' from \'/usr/lib/python3.5/re.py\'>]\n  $ # iterators print each item on a new line, applying the same conventions\n  $ pyfil \'iter(sys.path)\'\n  /home/ninjaaron/src/py/pyfil/venv/bin\n  /home/ninjaaron/src/py/pyfil\n  /usr/lib/python35.zip\n  /usr/lib/python3.5\n  /usr/lib/python3.5/plat-linux\n  /usr/lib/python3.5/lib-dynload\n  /home/ninjaaron/src/py/pyfil/venv/lib/python3.5/site-package\n  $ pyfil \'(i.split(\'/\')[1:] for i in sys.path)\'\n  ["home", "ninjaaron", "src", "py", "pyfil", "venv", "bin"]\n  ["home", "ninjaaron", "src", "py", "pyfil"]\n  ["usr", "lib", "python35.zip"]\n  ["usr", "lib", "python3.5"]\n  ["usr", "lib", "python3.5", "plat-linux"]\n  ["usr", "lib", "python3.5", "lib-dynload"]\n  ["home", "ninjaaron", "src", "py", "pyfil", "venv", "lib", "python3.5", "site-packages"]\n\nMost JSON is also valid Python, but be aware that you may occasionally\nsee ``null`` instead of ``None`` along with ``true`` and ``false``\ninstead of ``True`` and ``False``, and your tuples will look like list.\nI guess that\'s a risk I\'m willing to take. (The rational for this is\nthat pyfil is more about composability in the shell than printing valid\nPython literals. JSON is becoming the defacto standard for\nserialization.)\n\nsuppressing output and using statements\n.......................................\nBecause these defaults use eval() internally to get value of\nexpressions, statements may not be used. exec() supports statements, but\nit does not return the value of expressions when they are evaluated.\nWhen the -x/--exec flag is used, automatic printing is suppressed, and\nexpressions are evaluated with exec, so statements, such as assignments,\nmay be used. Values may still be printed explicitly.\n\n--quite suppresses automatic printing, but eval is still used.\n\nThe --post option is immune from --quiet and --exec. It will always be\nevaluated with ``eval()``, and it will always try to print. The only\ndifference is that if --quiet or --exec was used, json will be printed\nwith indentation unless --force-oneline-json is used.\n\nusing files for input and output\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n``pyfil`` doesn\'t have any parameters for input and output files. Instead,\nuse redirection.\n\n.. code:: bash\n\n  pyfil -s \'i.upper()\' > output.txt < input.txt\n\nusing multiple expression arguments\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n``pyfil`` can take as many expressions as desired as arguments. When used\nwith --exec, this works pretty much as expected, and assignment must be\ndone manually.\n\nWithout --exec, the return value of each expression is assigned to the\nvariable ``x``, which can be used in the next expression. The final\nvalue of ``x`` is what is ultimately printed, not any intermediate\nvalues.\n\n.. code:: bash\n\n  $ pyfil \'reversed("abcd")\' \'i.upper() for i in x\'\n  D\n  C\n  B\n  A\n\nlooping over stdin\n~~~~~~~~~~~~~~~~~~\none can do simple loops with a generator expression. (note that any\nexpression that evaluates to an iterator will print each item on a new\nline unless the ``--join`` option is specified.)\n\n.. code:: bash\n\n    $ ls / | pyfil \'i.upper() for i in stdin\'\n    BIN@\n    BOOT/\n    DEV/\n    ETC/\n    HOME/\n    ...\n\nHowever, with the ``-l``/``--loop`` flag, pyfil loops over stdin in a\ncontext like this:\n\n.. code:: python\n\n    for n, i in enumerate(stdin):\n        expressions\n\nTherefore, the above loop can also be written thusly:\n\n.. code:: bash\n\n    $ ls / | pyfil -l \'i.upper()\'\n\n``--pre`` and ``--post`` (-b and -e) options can be used to specify\nactions to run before or after the loop. Note that the --pre option is\nrun with exec instead of eval, and therefore output is never printed,\nand statements may be used. This is for things like initializing\ncontainer types. --post is automatically printed and statements are not\nallowed (even if --exec is used). --loop is implied if ``--post`` is\nused. ``--pre`` can be used without a --loop to do assignments (or\nwhatever else you may want to do with a statement).\n\nUsing ``-s``/``--split`` or ``-F``/``--field-sep`` for doing awk things\nalso implies --loop. The resulting list is named ``f`` in the execution\nenvironment, in quazi-Perl fashion. (oh, and that list is actually a\nsubclass of collections.UserList that returns an empty string if the\nindex doesn\'t exist, so it acts more like awk with empty fields, rather\nthan throwing and error).\n\njson input\n~~~~~~~~~~\n``pyfil`` can parse json objects from stdin with the ``-j``/``--json``\nflag. They are passed into the environment as the ``j`` object.\ncombining with the --loop flag will treat stdin as one json object per\nline. json objects support dot syntax for attribute access, e.g.\n``j.someattr.attr_of_someattr``\n\nThere are occasionally functions that require "real" dictionaries and\nwon\'t work with this special subclass that supports dot access. In\nsuch cases, use ``-J``/``--real-dict-json`` to get unadultered Python\ndictionaries.\n\nformatting output (and \'awk stuff\')\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\nIt\'s probably obvious that the most powerful way to format strings is\nwith Python\'s str.format method and the ``-F`` or ``-s`` options.\n\n.. code:: bash\n\n  $ ls -l /|pyfil -s \'"{0}\\t{2}\\t{8}".format(*f)\'\n  IndexError: tuple index out of range\n  lrwxrwxrwx\troot\tbin\n  drwxr-xr-x\troot\tboot/\n  drwxr-xr-x\troot\tdev/\n  drwxr-xr-x\troot\tetc/\n  drwxr-xr-x\troot\thome/\n  lrwxrwxrwx\troot\tlib\n  ...\n\nHowever, you will note that using ``string.format(*f)`` produces an\nerror and does not print anything to stdout (error message is sent to\nstderr; see error handling for more options) for lines without enough\nfields, which may not be the desired behavior when dealing with lines\ncontaining arbitrary numbers of fields.\n\nFor simpler cases, you may wish to use the ``-n``/``--join`` option,\nwhich will join any iterables with the specified string before printing,\nand, in the case of the ``f`` list, will replace any none-existent\nfields with an empty string.\n\n.. code:: bash\n\n  $ ls -l /|pyfil -sn \'\\t\' \'f[0], f[2], f[8]\'\n  total\t\t\n  lrwxrwxrwx\troot\tbin\n  drwxr-xr-x\troot\tboot/\n  drwxr-xr-x\troot\tdev/\n  drwxr-xr-x\troot\tetc/\n  drwxr-xr-x\troot\thome/\n  lrwxrwxrwx\troot\tlib\n\nIn this case, the first line of ``ls -l /`` provides values for all\navailable fields.\n\nTechnical note:\n    The separator specified with the ``--join`` option is implemented\n    internally as ``ast.literal_eval("\'\'\'"+STRING.replace("\'",\n    r"\\\'")+"\'\'\'")``. If one works hard at it, it is possible to pass\n    values which will cause pyfil to crash; i.e. patterns ending with a\n    backslash. Keep in mind rules about escape sequences in the shell and\n    in python if you absolutely must have a pattern that terminates with\n    a backslash. (The reason it is implemented this way is to allow the\n    use of escape sequences that are meaningful to the python, but not\n    the shell, such as \\\\n, \\\\t, \\\\x, \\\\u, etc.)\n\nexamples\n~~~~~~~~\n\n*I realize that it\'s much better to do most of these things with the\noriginal utility. This is just to give some ideas of how to use `pyfil`*\n\nreplace ``wc -l``:\n\n.. code:: bash\n\n  $ ls / | pyfil \'len(stdin.l)\'\n  20\n\nreplace ``fgrep``:\n\n.. code:: bash\n\n  $ ls / | pyfil \'(i for i in stdin if "v" in i)\'\n  $ ls / | pyfil -l \'i if "v" in i else None\'\n\n\nreplace ``grep``:\n\n.. code:: bash\n\n  $ ls / | pyfil \'filter(lambda x: re.search("^m", x), stdin)\'\n  $ ls / | pyfil -lS \'re.search("^m", i).string)\'\n  $ # using the -S option to suppress a ton of error messages\n\nreplace ``sed \'s/...``:\n\n.. code:: bash\n\n  $ ls / | pyfil -l \'re.sub("^([^aeiou][aeiou][^aeiou]\\W)", lambda m: m.group(0).upper(), i)\'\n  BIN@\n  boot/\n  data/\n  DEV/\n  etc/\n  ...\n\nThis example illustrates that, while you might normally prefer ``sed``\nfor replacement tasks, the ability to define a replacement function with\n``re.sub`` does offer some interesting possibilities. Indeed, someone\nfamiliar with coreutils should never prefer to do something they already\ncomfortable doing the traditional way with ``pyfil`` (coreutils are\nheavily optimized). Python is interesting for this use-case because it\noffers great logic, anonymous functions and all kinds of other goodies\nthat only full-fledged, modern programming language can offer. Use\ncoreutiles for the jobs they were designed to excel in. Use ``pyfil`` to\ndo whatever they can\'t... and seriously, how will coreutils do this?:\n\n.. code:: bash\n\n  $ wget -qO- http://pypi.python.org/pypi/pyfil/json/ | pyfil -j \'j.urls[0].filename\'\n  pyfil-0.5-py3-none-any.whl\n  $ ls -l | pyfil -qSs \\\n  "d.update({f[8]: {\'permissions\': f[0], \'user\': f[2], \'group\': f[3],\n                    \'size\': int(f[4]), \'timestamp\': \' \'.join(f[5:8])}})" \\\n  --post \'d\'\n.. code:: json\n\n  {\n    "README.rst": {\n      "group": "users",\n      "user": "ninjaaron",\n      "permissions": "-rw-r--r--",\n      "timestamp": "Sep 6 20:55",\n      "size": 18498\n    },\n    "pyfil/": {\n      "group": "users",\n      "user": "ninjaaron",\n      "permissions": "drwxr-xr-x",\n      "timestamp": "Sep 6 20:20",\n      "size": 16\n    },\n    "setup.py": {\n      "group": "users",\n      "user": "ninjaaron",\n      "permissions": "-rw-r--r--",\n      "timestamp": "Sep 6 20:30",\n      "size": 705\n    },\n    "LICENSE": {\n      "group": "users",\n      "user": "ninjaaron",\n      "permissions": "-rw-r--r--",\n      "timestamp": "Sep 3 13:32",\n      "size": 1306\n    }\n  }\n\nOther things which might be difficult with coreutils:\n\n.. code:: bash\n\n  $ ls / | pyfil -n \'  \' \'reversed(stdin.l)\'\n  var/  usr/  tmp/  sys/  srv/  sbin@  run/  root/  proc/  opt/  ...\n  $ # ^^ also, `ls /|pyfil -n \'  \' \'stdin.l[::-1]\'\n\nerror handling\n~~~~~~~~~~~~~~\nIf pyfil encounters an exception while evaluating user input the default\nis to print the error message to stderr and continue (if looping over\nstdin), as we saw in the section on formatting output. However, errors\ncan also be silenced entirely with the ``-S``/``--silence-errors``\noption. In the below example, the first line produces an error, but we\ndon\'t hear about it.\n\n.. code:: bash\n\n  $ ls -l /|pyfil -sS \'"{0}\\t{2}\\t{8}".format(*f)\' \n  lrwxrwxrwx\troot\tbin\n  drwxr-xr-x\troot\tboot/\n  drwxr-xr-x\troot\tdev/\n  drwxr-xr-x\troot\tetc/\n  drwxr-xr-x\troot\thome/\n  lrwxrwxrwx\troot\tlib\n  ...\n\nAlternatively, errors may be raised when encountered, which will stop\nexecution and give a (fairly useless, in this case) traceback. This is\ndone with the ``-R``/``--raise-errors`` flag.\n\n.. code:: bash\n\n  $ ls -l /|pyfil -sR \'"{0}\\t{2}\\t{8}".format(*f)\'\n  Traceback (most recent call last):\n    File "/home/ninjaaron/src/py/pyfil/venv/bin/pyfil", line 9, in <module>\n      load_entry_point(\'pyfil\', \'console_scripts\', \'pyfil\')()\n    File "/home/ninjaaron/src/py/pyfil/pyfil/pyfil.py", line 242, in main\n      run(expressions, a, namespace)\n    File "/home/ninjaaron/src/py/pyfil/pyfil/pyfil.py", line 164, in run\n      handle_errors(e, args)\n    File "/home/ninjaaron/src/py/pyfil/pyfil/pyfil.py", line 134, in handle_errors\n      raise exception\n    File "/home/ninjaaron/src/py/pyfil/pyfil/pyfil.py", line 162, in run\n      value = func(expr, namespace)\n    File "<string>", line 1, in <module>\n  IndexError: tuple index out of range\n\nIn addition to these two handlers, it is possible to specify a\nrudimentary custom handler with the ``-H``/``--exception-handler``\nflags. The syntax is ``-H \'Exception: expression\'``, where ``Exception``\ncan be any builtin exception class (including Exception, to catch all\nerrors), and ``expression`` is the alternative expression to evaluate\n(and print, if not --quiet).\n\n.. code:: bash\n\n  $ ls -l /|pyfil -sH \'IndexError: i\' \'"{0}\\t{2}\\t{8}".format(*f)\'\n  total 32\n  lrwxrwxrwx\troot\tbin\n  drwxr-xr-x\troot\tboot/\n  drwxr-xr-x\troot\tdev/\n  drwxr-xr-x\troot\tetc/\n  drwxr-xr-x\troot\thome/\n  lrwxrwxrwx\troot\tlib\n  ...\n\nIn this case, we\'ve chosen to print line without any additional\nformatting. If other errors are encountered, it will fall back to other\nhandlers (``-S``, ``-R``, or the default). For more sophisticated error\nhandling... write a real Python script, where you can handle to your\nheart\'s content.\n\nAlso note that this case is possible to handle with a test instead of an\nexception handler because ``f`` is a special list that will return an\nempty string instead of throw an index error if the index is out of\nrange:\n\n``ls -l / | pyfil -s \'"{0}\\t{2}\\t{8}".format(*f) if f[2] else i\'``\n\nEasy-peasy.\n',
    'author': 'Aaron Christianson',
    'author_email': 'ninjaaron@gmail.com',
    'url': 'https://github.com/ninjaaron/pyfil',
    'py_modules': modules,
    'entry_points': entry_points,
    'python_requires': '>=3.4,<4.0',
}


setup(**setup_kwargs)
