Parser fix (#2769)

* Fixed parser to eliminate need for escape quotes. TODO: Fix double call to shlex, fix spaces in spec __str__

* Fixed double shlex

* cleanup

* rebased on develop

* Fixed parsing for multiple specs; broken since #360

* Revoked elimination of the `-` sigil in the syntax, and added it back into tests

* flake8

* more flake8

* Cleaned up dead code and added comments to parsing code

* bugfix for spaces in arguments; new bug found in testing

* Added unit tests for kv pairs in parsing/lexing

* Even more flake8

* ... yet another flake8

* Allow multiple specs in install

* unfathomable levels of flake8

* Updated documentation to match parser fix

lib/spack/docs/basic_usage.rst

@@ -647,8 +647,8 @@ avoid ambiguity.
 
 When spack normalizes specs, it prints them out with no spaces boolean
 variants using the backwards compatibility syntax and uses only ``~``
-for disabled boolean variants.  We allow ``-`` and spaces on the command
-line is provided for convenience and legibility.
+for disabled boolean variants.  The ``-`` and spaces on the command
+line are provided for convenience and legibility.
 
 ^^^^^^^^^^^^^^
 Compiler Flags
@@ -658,14 +658,17 @@ Compiler flags are specified using the same syntax as non-boolean variants,
 but fulfill a different purpose. While the function of a variant is set by
 the package, compiler flags are used by the compiler wrappers to inject
 flags into the compile line of the build. Additionally, compiler flags are
-inherited by dependencies. ``spack install libdwarf cppflags=\"-g\"`` will
+inherited by dependencies. ``spack install libdwarf cppflags="-g"`` will
 install both libdwarf and libelf with the ``-g`` flag injected into their
 compile line.
 
-Notice that the value of the compiler flags must be escape quoted on the
-command line. From within python files, the same spec would be specified
-``libdwarf cppflags="-g"``. This is necessary because of how the shell
-handles the quote symbols.
+Notice that the value of the compiler flags must be quoted if it
+contains any spaces. Any of ``cppflags=-O3``, ``cppflags="-O3"``,
+``cppflags='-O3'``, and ``cppflags="-O3 -fPIC"`` are acceptable, but
+``cppflags=-O3 -fPIC`` is not. Additionally, if they value of the
+compiler flags is not the last thing on the line, it must be followed
+by a space. The commmand ``spack install libelf cppflags="-O3"%intel``
+will be interpreted as an attempt to set `cppflags="-O3%intel"``.
 
 The six compiler flags are injected in the order of implicit make commands
 in GNU Autotools. If all flags are set, the order is

lib/spack/docs/features.rst

@@ -41,7 +41,7 @@ platform, all on the command line.
    $ spack install mpileaks@1.1.2 %gcc@4.7.3 +debug
 
    # Add compiler flags using the conventional names
-   $ spack install mpileaks@1.1.2 %gcc@4.7.3 cppflags=\"-O3 -floop-block\"
+   $ spack install mpileaks@1.1.2 %gcc@4.7.3 cppflags="-O3 -floop-block"
 
    # Cross-compile for a different architecture with arch=
    $ spack install mpileaks@1.1.2 arch=bgqos_0

lib/spack/docs/getting_started.rst

@@ -632,7 +632,7 @@ the command line:
 
 .. code-block:: console
 
-   $ spack install openmpi fflags=\"-mismatch\"
+   $ spack install openmpi fflags="-mismatch"
 
 Or it can be set permanently in your ``compilers.yaml``:
 

lib/spack/docs/tutorial_sc16_spack_basics.rst

@@ -184,15 +184,15 @@ compilers.
   [+] ~/spack/opt/spack/linux-redhat6-x86_64/intel-15.0.4/libelf-0.8.13-w33hrejdyqu2j2gggdswitls2zv6kdsi
 
 
-The spec syntax also includes compiler flags. Spack accepts ``cppflags``,
-``cflags``, ``cxxflags``, ``fflags``, ``ldflags``, and ``ldlibs``
-parameters.  The values of these fields must be escape-quoted with ``\"``
-on the command line. These values are injected into the compile line
-automatically by the Spack compiler wrappers.
+The spec syntax also includes compiler flags. Spack accepts
+``cppflags``, ``cflags``, ``cxxflags``, ``fflags``, ``ldflags``, and
+``ldlibs`` parameters.  The values of these fields must be quoted on
+the command line if they include spaces. These values are injected
+into the compile line automatically by the Spack compiler wrappers.
 
 .. code-block:: console
 
-  $ spack install libelf @0.8.12 cppflags=\"-O3\"
+  $ spack install libelf @0.8.12 cppflags="-O3"
   ==> Installing libelf
   ==> Trying to fetch from ~/spack/var/spack/cache/libelf/libelf-0.8.12.tar.gz
   ################################################################################################################################################################################# 100.0%
@@ -309,7 +309,7 @@ top-level package, we can also specify about a dependency using ``^``.
 
 Packages can also be referred to from the command line by their package
 hash. Using the ``spack find -lf`` command earlier we saw that the hash
-of our optimized installation of libelf (``cppflags=\"-O3\"``) began with
+of our optimized installation of libelf (``cppflags="-O3"``) began with
 ``vrv2ttb``. We can now explicitly build with that package without typing
 the entire spec, by using the ``/`` sigil to refer to it by hash. As with
 other tools like git, you do not need to specify an *entire* hash on the
@@ -1103,8 +1103,8 @@ already covered in the :ref:`basics-tutorial-install` and
 The ``spack find`` command can accept what we call "anonymous specs."
 These are expressions in spec syntax that do not contain a package
 name. For example, `spack find %intel` will return every package built
-with the intel compiler, and ``spack find cppflags=\\"-O3\\"`` will
-return every package which was built with ``cppflags=\\"-O3\\"``.
+with the intel compiler, and ``spack find cppflags="-O3"`` will
+return every package which was built with ``cppflags="-O3"``.
 
 .. code-block:: console
 
@@ -1115,7 +1115,7 @@ return every package which was built with ``cppflags=\\"-O3\\"``.
 
 
 
-  $ spack find cppflags=\"-O3\"
+  $ spack find cppflags="-O3"
   ==> 1 installed packages.
   -- linux-redhat6-x86_64 / gcc@4.4.7 -----------------------------
   libelf@0.8.12