Add environment variables to path substitution

Update documentation on config file variable substitutions and
add expansion of environment variables in config files.

lib/spack/docs/config_yaml.rst

@@ -14,27 +14,6 @@ see the default settings by looking at
 These settings can be overridden in ``etc/spack/config.yaml`` or
 ``~/.spack/config.yaml``.  See :ref:`configuration-scopes` for details.
 
-.. _config-file-variables:
-
-------------------------------
-Config file variables
-------------------------------
-
-You may notice some variables prefixed with ``$`` in the settings above.
-Spack understands several variables that can be used in values of
-configuration parameters.  They are:
-
-  * ``$spack``: path to the prefix of this spack installation
-  * ``$tempdir``: default system temporary directory (as specified in
-    Python's `tempfile.tempdir
-    <https://docs.python.org/2/library/tempfile.html#tempfile.tempdir>`_
-    variable.
-  * ``$user``: name of the current user
-
-Note that, as with shell variables, you can write these as ``$varname``
-or with braces to distinguish the variable from surrounding characters:
-``${varname}``.
-
 --------------------
 ``install_tree``
 --------------------

lib/spack/docs/configuration.rst

@@ -261,3 +261,52 @@ The merged configuration would look like this:
        - /lustre-scratch/$user
        - ~/mystage
    $ _
+
+.. _config-file-variables:
+
+------------------------------
+Config file variables
+------------------------------
+
+Spack understands several variables which can be used in config file paths
+where ever they appear. There are three sets of these variables, Spack specific 
+variables, environment variables, and user path variables. Spack specific
+variables and environment variables both are indicated by prefixing the variable
+name with ``$``. User path variables are indicated at the start of the path with
+``~`` or ``~user``. Let's discuss each in turn.
+
+^^^^^^^^^^^^^^^^^^^^^^^^
+Spack Specific Variables
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+Spack understands several special variables. These are:
+
+  * ``$spack``: path to the prefix of this spack installation
+  * ``$tempdir``: default system temporary directory (as specified in
+    Python's `tempfile.tempdir
+    <https://docs.python.org/2/library/tempfile.html#tempfile.tempdir>`_
+    variable.
+  * ``$user``: name of the current user
+
+Note that, as with shell variables, you can write these as ``$varname``
+or with braces to distinguish the variable from surrounding characters:
+``${varname}``. Their names are also case insensitive meaning that ``$SPACK``
+works just as well as ``$spack``. These special variables are also
+substituted first, so any environment variables with the same name will not
+be used.
+
+^^^^^^^^^^^^^^^^^^^^^
+Environment Variables
+^^^^^^^^^^^^^^^^^^^^^
+
+Spack then uses ``os.path.expandvars`` to expand any remaining environment
+variables.
+
+^^^^^^^^^^^^^^
+User Variables
+^^^^^^^^^^^^^^
+
+Spack also uses the ``os.path.expanduser`` function on the path to expand
+any user tilde paths such as ``~`` or ``~user``. These tilde paths must appear
+at the beginning of the path or ``os.path.expanduser`` will not properly
+expand them.

lib/spack/spack/util/path.py

@@ -65,8 +65,10 @@ def repl(match):
 
 
 def canonicalize_path(path):
-    """Substitute config vars, expand user home, take abspath."""
+    """Substitute config vars, expand environment vars,
+       expand user home, take abspath."""
     path = substitute_config_variables(path)
+    path = os.path.expandvars(path)
     path = os.path.expanduser(path)
     path = os.path.abspath(path)
     return path