zhangyiss/spack
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Use console instead of shell syntax lexer, fixes #1634

Browse Source
This commit is contained in:
Adam J. Stewart 2016-08-26 16:41:00 -05:00
parent f856952728
commit 6795f4ca12
8 changed files with 949 additions and 981 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

425
lib/spack/docs/basic_usage.rst
View File

@ -8,15 +8,18 @@ The ``spack`` command has many *subcommands*. You'll only need a
small subset of them for typical usage.
Note that Spack colorizes output. ``less -R`` should be used with
Spack to maintain this colorization. E.g.::
Spack to maintain this colorization. E.g.:
spack find | less -R
.. code-block:: console
It is recommend that the following be put in your ``.bashrc`` file::
$ spack find | less -R
It is recommend that the following be put in your ``.bashrc`` file:
.. code-block:: sh
alias less='less -R'
--------------------------
Listing available packages
--------------------------
@ -114,18 +117,18 @@ Installing and uninstalling
For example, To install the latest version of the ``mpileaks``
package, you might type this:
.. code-block:: sh
.. code-block:: console
$ spack install mpileaks
If `mpileaks` depends on other packages, Spack will install the
If ``mpileaks`` depends on other packages, Spack will install the
dependencies first. It then fetches the ``mpileaks`` tarball, expands
it, verifies that it was downloaded without errors, builds it, and
installs it in its own directory under ``$SPACK_ROOT/opt``. You'll see
a number of messages from spack, a lot of build output, and a message
that the packages is installed:
.. code-block:: sh
.. code-block:: console
$ spack install mpileaks
==> Installing mpileaks
@ -155,7 +158,7 @@ Building a specific version
Spack can also build *specific versions* of a package. To do this,
just add ``@`` after the package name, followed by a version:
.. code-block:: sh
.. code-block:: console
$ spack install mpich@3.0.4
@ -174,8 +177,8 @@ two configurations of the same version of a package, one built with boost
This can all be done on the command line using the *spec* syntax.
Spack calls the descriptor used to refer to a particular package
configuration a **spec**. In the commands above, ``mpileaks`` and
``mpileaks@3.0.4``. We'll talk more about how you can use them to
customize an installation in :ref:`sec-specs`.
``mpileaks@3.0.4`` are both valid *specs*. We'll talk more about how
you can use them to customize an installation in :ref:`sec-specs`.
.. _spack-uninstall:
@ -183,53 +186,56 @@ customize an installation in :ref:`sec-specs`.
``spack uninstall``
^^^^^^^^^^^^^^^^^^^
To uninstall a package, type ``spack uninstall <package>``. This will ask the user for
confirmation, and in case will completely remove the directory in which the package was installed.
To uninstall a package, type ``spack uninstall <package>``. This will ask
the user for confirmation, and in case will completely remove the directory
in which the package was installed.
.. code-block:: sh
.. code-block:: console
spack uninstall mpich
$ spack uninstall mpich
If there are still installed packages that depend on the package to be
uninstalled, spack will refuse to uninstall it.
To uninstall a package and every package that depends on it, you may give the
`--dependents` option.
``--dependents`` option.
.. code-block:: sh
.. code-block:: console
spack uninstall --dependents mpich
$ spack uninstall --dependents mpich
will display a list of all the packages that depend on `mpich` and, upon confirmation,
will uninstall them in the right order.
will display a list of all the packages that depend on ``mpich`` and, upon
confirmation, will uninstall them in the right order.
A line like
A command like
.. code-block:: sh
.. code-block:: console
spack uninstall mpich
$ spack uninstall mpich
may be ambiguous, if multiple ``mpich`` configurations are installed. For example, if both
``mpich@3.0.2`` and ``mpich@3.1`` are installed, ``mpich`` could refer
to either one. Because it cannot determine which one to uninstall,
Spack will ask you either to provide a version number to remove the
ambiguity or use the ``--all`` option to uninstall all of the matching packages.
may be ambiguous if multiple ``mpich`` configurations are installed.
For example, if both ``mpich@3.0.2`` and ``mpich@3.1`` are installed,
``mpich`` could refer to either one. Because it cannot determine which
one to uninstall, Spack will ask you either to provide a version number
to remove the ambiguity or use the ``--all`` option to uninstall all of
the matching packages.
You may force uninstall a package with the `--force` option
You may force uninstall a package with the ``--force`` option
.. code-block:: sh
.. code-block:: console
spack uninstall --force mpich
$ spack uninstall --force mpich
but you risk breaking other installed packages. In general, it is safer to remove dependent
packages *before* removing their dependencies or use the `--dependents` option.
but you risk breaking other installed packages. In general, it is safer to
remove dependent packages *before* removing their dependencies or use the
``--dependents`` option.
-------------------------
Seeing installed packages
-------------------------
We know that ``spack list`` shows you the names of available packages,
but how do you figure out which are installed?
but how do you figure out which are already installed?
.. _spack-find:
@ -244,7 +250,7 @@ of the same package with different specs.
Running ``spack find`` with no arguments lists installed packages:
.. code-block:: sh
.. code-block:: console
$ spack find
==> 74 installed packages.
@ -283,17 +289,18 @@ only shows the version of installed packages.
``spack find`` can filter the package list based on the package name, spec, or
a number of properties of their installation status. For example, missing
dependencies of a spec can be shown with ``-m``, packages which were
dependencies of a spec can be shown with ``--missing``, packages which were
explicitly installed with ``spack install <package>`` can be singled out with
``-e`` and those which have been pulled in only as dependencies with ``-E``.
``--explicit`` and those which have been pulled in only as dependencies with
``--implicit``.
In some cases, there may be different configurations of the *same*
version of a package installed. For example, there are two
installations of of ``libdwarf@20130729`` above. We can look at them
in more detail using ``spack find -d``, and by asking only to show
installations of ``libdwarf@20130729`` above. We can look at them
in more detail using ``spack find --deps``, and by asking only to show
``libdwarf`` packages:
.. code-block:: sh
.. code-block:: console
$ spack find --deps libdwarf
==> 2 installed packages.
@ -307,25 +314,25 @@ Now we see that the two instances of ``libdwarf`` depend on
*different* versions of ``libelf``: 0.8.12 and 0.8.13. This view can
become complicated for packages with many dependencies. If you just
want to know whether two packages' dependencies differ, you can use
``spack find -l``:
``spack find --long``:
.. code-block:: sh
.. code-block:: console
$ spack find -l libdwarf
$ spack find --long libdwarf
==> 2 installed packages.
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
libdwarf@20130729-d9b90962 libdwarf@20130729-b52fac98
Now the ``libwarf`` installs have hashes after their names. These are
Now the ``libdwarf`` installs have hashes after their names. These are
hashes over all of the dependencies of each package. If the hashes
are the same, then the packages have the same dependency configuration.
If you want to know the path where each package is installed, you can
use ``spack find -p``:
use ``spack find --paths``:
.. code-block:: sh
.. code-block:: console
$ spack find -p
$ spack find --paths
==> 74 installed packages.
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
ImageMagick@6.8.9-10 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/ImageMagick@6.8.9-10-4df950dd
@ -340,9 +347,9 @@ use ``spack find -p``:
And, finally, you can restrict your search to a particular package
by supplying its name:
.. code-block:: sh
.. code-block:: console
$ spack find -p libelf
$ spack find --paths libelf
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
libelf@0.8.11 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/libelf@0.8.11
libelf@0.8.12 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/libelf@0.8.12
@ -353,7 +360,7 @@ by supplying its name:
package. If you want to find only libelf versions greater than version
0.8.12, you could say:
.. code-block:: sh
.. code-block:: console
$ spack find libelf@0.8.12:
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
@ -362,17 +369,16 @@ package. If you want to find only libelf versions greater than version
Finding just the versions of libdwarf built with a particular version
of libelf would look like this:
.. code-block:: sh
.. code-block:: console
$ spack find -l libdwarf ^libelf@0.8.12
$ spack find --long libdwarf ^libelf@0.8.12
==> 1 installed packages.
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
libdwarf@20130729-d9b90962
We can also search for packages that have a certain attribute. For example,
``spack find -l libdwarf +debug`` will show only installations of libdwarf
with the 'debug' compile-time option enabled, while ``spack find -l +debug``
will find every installed package with a 'debug' compile-time option enabled.
``spack find libdwarf +debug`` will show only installations of libdwarf
with the 'debug' compile-time option enabled.
The full spec syntax is discussed in detail in :ref:`sec-specs`.
@ -385,7 +391,7 @@ Compiler configuration
Spack has the ability to build packages with multiple compilers and
compiler versions. Spack searches for compilers on your machine
automatically the first time it is run. It does this by inspecting
your path.
your ``PATH``.
.. _spack-compilers:
@ -394,7 +400,9 @@ your path.
^^^^^^^^^^^^^^^^^^^
You can see which compilers spack has found by running ``spack
compilers`` or ``spack compiler list``::
compilers`` or ``spack compiler list``:
.. code-block:: console
$ spack compilers
==> Available compilers
@ -432,7 +440,9 @@ An alias for ``spack compiler find``.
If you do not see a compiler in this list, but you want to use it with
Spack, you can simply run ``spack compiler find`` with the path to
where the compiler is installed. For example::
where the compiler is installed. For example:
.. code-block:: console
$ spack compiler find /usr/local/tools/ic-13.0.079
==> Added 1 new compiler to /Users/gamblin2/.spack/compilers.yaml
@ -441,7 +451,9 @@ where the compiler is installed. For example::
Or you can run ``spack compiler find`` with no arguments to force
auto-detection. This is useful if you do not know where compilers are
installed, but you know that new compilers have been added to your
``PATH``. For example, using dotkit, you might do this::
``PATH``. For example, using dotkit, you might do this:
.. code-block:: console
$ module load gcc-4.9.0
$ spack compiler find
@ -458,7 +470,9 @@ This loads the environment module for gcc-4.9.0 to add it to
^^^^^^^^^^^^^^^^^^^^^^^
If you want to see specifics on a particular compiler, you can run
``spack compiler info`` on it::
``spack compiler info`` on it:
.. code-block:: console
$ spack compiler info intel@15
intel@15.0.0:
@ -466,9 +480,11 @@ If you want to see specifics on a particular compiler, you can run
cxx = /usr/local/bin/icpc-15.0.090
f77 = /usr/local/bin/ifort-15.0.090
fc = /usr/local/bin/ifort-15.0.090
modules = []
operating system = centos6
This shows which C, C++, and Fortran compilers were detected by Spack.
Notice also that we didn\'t have to be too specific about the
Notice also that we didn't have to be too specific about the
version. We just said ``intel@15``, and information about the only
matching Intel compiler was displayed.
@ -480,9 +496,42 @@ If auto-detection fails, you can manually configure a compiler by
editing your ``~/.spack/compilers.yaml`` file. You can do this by running
``spack config edit compilers``, which will open the file in your ``$EDITOR``.
Each compiler configuration in the file looks like this::
Each compiler configuration in the file looks like this:
.. code-block:: yaml
compilers:
- compiler:
modules = []
operating_system: centos6
paths:
cc: /usr/local/bin/icc-15.0.024-beta
cxx: /usr/local/bin/icpc-15.0.024-beta
f77: /usr/local/bin/ifort-15.0.024-beta
fc: /usr/local/bin/ifort-15.0.024-beta
spec: intel@15.0.0:
For compilers, like ``clang``, that do not support Fortran, put
``None`` for ``f77`` and ``fc``:
.. code-block:: yaml
paths:
cc: /usr/bin/clang
cxx: /usr/bin/clang++
f77: None
fc: None
spec: clang@3.3svn:
Once you save the file, the configured compilers will show up in the
list displayed by ``spack compilers``.
You can also add compiler flags to manually configured compilers. The
valid flags are ``cflags``, ``cxxflags``, ``fflags``, ``cppflags``,
``ldflags``, and ``ldlibs``. For example:
.. code-block:: yaml
...
compilers:
- compiler:
modules = []
@ -492,36 +541,9 @@ Each compiler configuration in the file looks like this::
cxx: /usr/local/bin/icpc-15.0.024-beta
f77: /usr/local/bin/ifort-15.0.024-beta
fc: /usr/local/bin/ifort-15.0.024-beta
spec: intel@15.0.0:
For compilers, like ``clang``, that do not support Fortran, put
``None`` for ``f77`` and ``fc``::
clang@3.3svn:
cc: /usr/bin/clang
cxx: /usr/bin/clang++
f77: None
fc: None
Once you save the file, the configured compilers will show up in the
list displayed by ``spack compilers``.
You can also add compiler flags to manually configured compilers. The
valid flags are ``cflags``, ``cxxflags``, ``fflags``, ``cppflags``,
``ldflags``, and ``ldlibs``. For example::
...
compilers:
- compiler:
...
intel@15.0.0:
cc: /usr/local/bin/icc-15.0.024-beta
cxx: /usr/local/bin/icpc-15.0.024-beta
f77: /usr/local/bin/ifort-15.0.024-beta
fc: /usr/local/bin/ifort-15.0.024-beta
parameters:
cppflags: -O3 -fPIC
...
spec: intel@15.0.0:
These flags will be treated by spack as if they were enterred from
the command line each time this compiler is used. The compiler wrappers
@ -541,10 +563,12 @@ Spack, that descriptor is called a *spec*. Spack uses specs to refer
to a particular build configuration (or configurations) of a package.
Specs are more than a package name and a version; you can use them to
specify the compiler, compiler version, architecture, compile options,
and dependency options for a build. In this section, we\'ll go over
and dependency options for a build. In this section, we'll go over
the full syntax of specs.
Here is an example of a much longer spec than we\'ve seen thus far::
Here is an example of a much longer spec than we've seen thus far:
.. code-block:: none
mpileaks @1.2:1.4 %gcc@4.7.5 +debug -qt arch=bgq_os ^callpath @1.1 %gcc@4.7.2
@ -638,7 +662,7 @@ of dependencies within a spec is that they appear *after* the root
package. For example, these two specs represent exactly the same
configuration:
.. code-block:: sh
.. code-block:: none
mpileaks ^callpath@1.0 ^libelf@0.8.3
mpileaks ^libelf@0.8.3 ^callpath@1.0
@ -712,7 +736,7 @@ makes available. Variants can be specified using
a flexible parameter syntax ``name=<value>``. For example,
``spack install libelf debug=True`` will install libelf build with debug
flags. The names of particular variants available for a package depend on
what was provided by the package author. ``spack into <package>`` will
what was provided by the package author. ``spack info <package>`` will
provide information on what build variants are available.
For compatibility with earlier versions, variants which happen to be
@ -723,7 +747,7 @@ compile time option for the libelf package.
Depending on the package a variant may have any default value. For
``libelf`` here, ``debug`` is ``False`` by default, and we turned it on
with ``debug=True`` or ``+debug``. If a package is ``True`` by default
with ``debug=True`` or ``+debug``. If a variant is ``True`` by default
you can turn it off by either adding ``-name`` or ``~name`` to the spec.
There are two syntaxes here because, depending on context, ``~`` and
@ -774,9 +798,9 @@ command line. From within python files, the same spec would be specified
handles the quote symbols.
The six compiler flags are injected in the order of implicit make commands
in gnu autotools. If all flags are set, the order is
``$cppflags $cflags|$cxxflags $ldflags command $ldlibs`` for C and C++ and
``$fflags $cppflags $ldflags command $ldlibs`` for fortran.
in GNU Autotools. If all flags are set, the order is
``$cppflags $cflags|$cxxflags $ldflags <command> $ldlibs`` for C and C++ and
``$fflags $cppflags $ldflags <command> $ldlibs`` for Fortran.
^^^^^^^^^^^^^^^^^^^^^^^
Architecture specifiers
@ -786,14 +810,14 @@ The architecture can be specified by using the reserved
words ``target`` and/or ``os`` (``target=x86-64 os=debian7``). You can also
use the triplet form of platform, operating system and processor.
.. code-block:: sh
.. code-block:: console
spack install libelf arch=cray_xc-CNL10-haswell
$ spack install libelf arch=cray-CNL10-haswell
Users on non-Cray systems won't have to worry about specifying the architecture.
Spack will autodetect what kind of operating system is on your machine as well
as the processor. For more information on how the architecture can be
used on Cray machines, check here :ref:`cray-support`
used on Cray machines, see :ref:`cray-support`
.. _sec-virtual-dependencies:
@ -848,21 +872,27 @@ Constraining virtual packages
When installing a package that depends on a virtual package, you can
opt to specify the particular provider you want to use, or you can let
Spack pick. For example, if you just type this::
Spack pick. For example, if you just type this:
spack install mpileaks
.. code-block:: console
$ spack install mpileaks
Then spack will pick a provider for you according to site policies.
If you really want a particular version, say mpich, then you could
run this instead::
If you really want a particular version, say ``mpich``, then you could
run this instead:
spack install mpileaks ^mpich
.. code-block:: console
$ spack install mpileaks ^mpich
This forces spack to use some version of ``mpich`` for its
implementation. As always, you can be even more specific and require
a particular ``mpich`` version::
a particular ``mpich`` version:
spack install mpileaks ^mpich@3
.. code-block:: console
$ spack install mpileaks ^mpich@3
The ``mpileaks`` package in particular only needs MPI-1 commands, so
any MPI implementation will do. If another package depends on
@ -887,7 +917,7 @@ to be unique, Spack will replace the reference with the spec to which
it refers. Otherwise, it will prompt for a more qualified hash.
Note that this will not work to reinstall a depencency uninstalled by
``spack uninstall -f``.
``spack uninstall --force``.
.. _spack-providers:
@ -937,15 +967,15 @@ distributions, this can be installed from the vendor's repository:
.. code-block:: sh
yum install environment-modules # (Fedora/RHEL/CentOS)
apt-get install environment-modules # (Ubuntu/Debian)
$ yum install environment-modules # (Fedora/RHEL/CentOS)
$ apt-get install environment-modules # (Ubuntu/Debian)
If your Linux distribution does not have
Environment Modules, you can get it with Spack:
.. code-block:: sh
.. code-block:: console
spack install environment-modules
$ spack install environment-modules
In this case to activate it automatically you need to add the following two
lines to your ``.bashrc`` profile (or similar):
@ -972,7 +1002,8 @@ For ``bash`` or ``ksh``, run:
.. code-block:: sh
. ${SPACK_ROOT}/share/spack/setup-env.sh
export SPACK_ROOT=/path/to/spack
. $SPACK_ROOT/share/spack/setup-env.sh
For ``csh`` and ``tcsh`` run:
@ -995,8 +1026,8 @@ directories:
.. code-block:: sh
${SPACK_ROOT}/share/spack/modules
${SPACK_ROOT}/share/spack/dotkit
$SPACK_ROOT/share/spack/modules
$SPACK_ROOT/share/spack/dotkit
The directories are automatically added to your ``MODULEPATH`` and
``DK_NODE`` environment variables when you enable Spack's `shell
@ -1011,7 +1042,7 @@ If you have shell support enabled you should be able to run either
been installed. Here is sample output of those programs, showing lots
of installed packages.
.. code-block:: sh
.. code-block:: console
$ module avail
@ -1028,7 +1059,7 @@ of installed packages.
libNBC@1.1.1%gcc@4.9.0-27912b7b sundials@2.5.0%gcc@4.9.0-27912b7b
libdwarf@20130729%gcc@4.4.7-b52fac98
.. code-block:: sh
.. code-block:: console
$ use -l spack
@ -1054,13 +1085,13 @@ The names here should look familiar, they're the same ones from
``spack find``. You *can* use the names here directly. For example,
you could type either of these commands to load the callpath module:
.. code-block:: sh
.. code-block:: console
use callpath@1.0.1%gcc@4.4.7-5dce4318
$ use callpath@1.0.1%gcc@4.4.7-5dce4318
.. code-block:: sh
.. code-block:: console
module load callpath@1.0.1%gcc@4.4.7-5dce4318
$ module load callpath@1.0.1%gcc@4.4.7-5dce4318
Neither of these is particularly pretty, easy to remember, or
easy to type. Luckily, Spack has its own interface for using modules
@ -1077,7 +1108,7 @@ And you can use the same shortened names you use everywhere else in
Spack. For example, this will add the ``mpich`` package built with
``gcc`` to your path:
.. code-block:: sh
.. code-block:: console
$ spack install mpich %gcc@4.4.7
@ -1090,15 +1121,15 @@ Spack. For example, this will add the ``mpich`` package built with
Or, similarly with modules, you could type:
.. code-block:: sh
.. code-block:: console
$ spack load mpich %gcc@4.4.7
These commands will add appropriate directories to your ``PATH``,
``MANPATH``, ``CPATH``, and ``LD_LIBRARY_PATH``. When you no longer want to use
a package, you can type unload or unuse similarly:
``MANPATH``, ``CPATH``, and ``LD_LIBRARY_PATH``. When you no longer
want to use a package, you can type unload or unuse similarly:
.. code-block:: sh
.. code-block:: console
$ spack unload mpich %gcc@4.4.7 # modules
$ spack unuse mpich %gcc@4.4.7 # dotkit
@ -1116,7 +1147,7 @@ Ambiguous module names
If a spec used with load/unload or use/unuse is ambiguous (i.e. more
than one installed package matches it), then Spack will warn you:
.. code-block:: sh
.. code-block:: console
$ spack load libelf
==> Error: Multiple matches for spec libelf. Choose one:
@ -1129,7 +1160,7 @@ identify one package. For example, above, the key differentiator is
that one ``libelf`` is built with the Intel compiler, while the other
used ``gcc``. You could therefore just type:
.. code-block:: sh
.. code-block:: console
$ spack load libelf %intel
@ -1144,13 +1175,13 @@ and are placed in the following directories under the Spack root:
.. code-block:: sh
${SPACK_ROOT}/share/spack/modules
${SPACK_ROOT}/share/spack/dotkit
$SPACK_ROOT/share/spack/modules
$SPACK_ROOT/share/spack/dotkit
The content that gets written in each module file can be customized in two ways:
1. overriding part of the ``spack.Package`` API within a ``package.py``
2. writing dedicated configuration files
#. overriding part of the ``spack.Package`` API within a ``package.py``
#. writing dedicated configuration files
^^^^^^^^^^^^^^^^^^^^^^^^
Override ``Package`` API
@ -1179,7 +1210,7 @@ RPATH.
Modules may be loaded recursively with the ``load`` command's
``--dependencies`` or ``-r`` argument:
.. code-block:: sh
.. code-block:: console
$ spack load --dependencies <spec> ...
@ -1195,7 +1226,7 @@ commands inside a shell script. By adding the ``--shell`` flag,
``spack module find`` may also be used to generate code that can be
cut-and-pasted into a shell script. For example:
.. code-block:: sh
.. code-block:: console
$ spack module find tcl --dependencies --shell py-numpy git
# bzip2@1.0.6%gcc@4.9.3=linux-x86_64
@ -1230,7 +1261,7 @@ cut-and-pasted into a shell script. For example:
module load git-2.8.0-rc2-gcc-4.9.3-3bib4hqtnv5xjjoq5ugt3inblt4xrgkd
The script may be further edited by removing unnecessary modules.
This script may be directly executed in bash via
This script may be directly executed in bash via:
.. code-block:: sh
@ -1261,7 +1292,6 @@ overrides ``setup_dependent_environment`` in the following way:
.. code-block:: python
def setup_dependent_environment(self, spack_env, run_env, extension_spec):
# ...
if extension_spec.package.extends(self.spec):
run_env.prepend_path('PYTHONPATH', os.path.join(extension_spec.prefix, self.site_packages_dir))
@ -1355,7 +1385,8 @@ modified by your dotkit modules. Then :
dotkit:
all:
filter:
environment_blacklist: ['CPATH', 'LIBRARY_PATH'] # Exclude changes to any of these variables
# Exclude changes to any of these variables
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
will generate dotkit module files that will not contain modifications to either
``CPATH`` or ``LIBRARY_PATH`` and environment module files that instead will
@ -1448,15 +1479,12 @@ Running ``spack module refresh`` will remove the
``share/spack/modules`` and ``share/spack/dotkit`` directories, then
regenerate all module and dotkit files from scratch:
.. code-block:: sh
.. code-block:: console
$ spack module refresh
==> Regenerating tcl module files.
==> Regenerating dotkit module files.
.. _extensions:
----------------
Filesystem Views
----------------
@ -1496,10 +1524,10 @@ built by Spack.
Views can be used for a variety of purposes including:
- A central installation in a traditional layout, eg ``/usr/local`` maintained over time by the sysadmin.
- A self-contained installation area which may for the basis of a top-level atomic versioning scheme, eg ``/opt/pro`` vs ``/opt/dev``.
- Providing an atomic and monolithic binary distribution, eg for delivery as a single tarball.
- Producing ephemeral testing or developing environments.
* A central installation in a traditional layout, eg ``/usr/local`` maintained over time by the sysadmin.
* A self-contained installation area which may for the basis of a top-level atomic versioning scheme, eg ``/opt/pro`` vs ``/opt/dev``.
* Providing an atomic and monolithic binary distribution, eg for delivery as a single tarball.
* Producing ephemeral testing or developing environments.
^^^^^^^^^^^^^^^^^^^^^^
Using Filesystem Views
@ -1514,9 +1542,9 @@ The following example creates a filesystem view based
on an installed ``cmake`` package and then removes from the view the
files in the ``cmake`` package while retaining its dependencies.
.. code-block:: sh
.. code-block:: console
$ spack view -v symlink myview cmake@3.5.2
$ spack view --verbose symlink myview cmake@3.5.2
==> Linking package: "ncurses"
==> Linking package: "zlib"
==> Linking package: "openssl"
@ -1529,7 +1557,7 @@ files in the ``cmake`` package while retaining its dependencies.
captoinfo clear cpack ctest infotocap openssl tabs toe tset
ccmake cmake c_rehash infocmp ncurses6-config reset tic tput
$ spack view -v -d false rm myview cmake@3.5.2
$ spack view --verbose --dependencies false rm myview cmake@3.5.2
==> Removing package: "cmake"
$ ls myview/bin/
@ -1568,6 +1596,8 @@ particular, if two packages require the same sub-tree of dependencies,
removing one package (recursively) will remove its dependencies and
leave the other package broken.
.. _extensions:
---------------------------
Extensions & Python support
---------------------------
@ -1583,7 +1613,7 @@ Spack has support for this type of installation as well. In Spack,
a package that can live inside the prefix of another package is called
an *extension*. Suppose you have Python installed like so:
.. code-block:: sh
.. code-block:: console
$ spack find python
==> 1 installed packages.
@ -1598,7 +1628,7 @@ an *extension*. Suppose you have Python installed like so:
You can find extensions for your Python installation like this:
.. code-block:: sh
.. code-block:: console
$ spack extensions python
==> python@2.7.8%gcc@4.4.7 arch=linux-debian7-x86_64-703c7a96
@ -1623,11 +1653,11 @@ You can find extensions for your Python installation like this:
The extensions are a subset of what's returned by ``spack list``, and
they are packages like any other. They are installed into their own
prefixes, and you can see this with ``spack find -p``:
prefixes, and you can see this with ``spack find --paths``:
.. code-block:: sh
.. code-block:: console
$ spack find -p py-numpy
$ spack find --paths py-numpy
==> 1 installed packages.
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
py-numpy@1.9.1 /g/g21/gamblin2/src/spack/opt/linux-debian7-x86_64/gcc@4.4.7/py-numpy@1.9.1-66733244
@ -1635,7 +1665,7 @@ prefixes, and you can see this with ``spack find -p``:
However, even though this package is installed, you cannot use it
directly when you run ``python``:
.. code-block:: sh
.. code-block:: console
$ spack load python
$ python
@ -1659,14 +1689,14 @@ in your current shell.
For tcl modules:
.. code-block:: sh
.. code-block:: console
$ spack load python
$ spack load py-numpy
or, for dotkit:
.. code-block:: sh
.. code-block:: console
$ spack use python
$ spack use py-numpy
@ -1690,7 +1720,7 @@ installation:
``spack activate``
^^^^^^^^^^^^^^^^^^
.. code-block:: sh
.. code-block:: console
$ spack activate py-numpy
==> Activated extension py-setuptools@11.3.1%gcc@4.4.7 arch=linux-debian7-x86_64-3c74eb69 for python@2.7.8%gcc@4.4.7.
@ -1707,7 +1737,7 @@ once all dependencies were activated in the ``python`` installation,
If we run ``spack extensions`` again, we now see the three new
packages listed as activated:
.. code-block:: sh
.. code-block:: console
$ spack extensions python
==> python@2.7.8%gcc@4.4.7 arch=linux-debian7-x86_64-703c7a96
@ -1756,7 +1786,7 @@ explicitly load their preferred version.
If, for some reason, you want to activate a package *without* its
dependencies, you can use ``spack activate -f``:
.. code-block:: sh
.. code-block:: console
$ spack activate -f py-numpy
==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=linux-debian7-x86_64-66733244 for python@2.7.8%gcc@4.4.7.
@ -1775,15 +1805,17 @@ several variants:
* ``spack deactivate <extension>`` will deactivate a single
extension. If another activated extension depends on this one,
Spack will warn you and exit with an error.
* ``spack deactivate -f <extension>`` deactivates an extension
* ``spack deactivate --force <extension>`` deactivates an extension
regardless of packages that depend on it.
* ``spack deactivate -a <extension>`` deactivates an extension and
all of its dependencies. Use ``-f`` to disregard dependents.
* ``spack deactivate -a <extendee>`` deactivates *all* activated
* ``spack deactivate --all <extension>`` deactivates an extension and
all of its dependencies. Use ``--force`` to disregard dependents.
* ``spack deactivate --all <extendee>`` deactivates *all* activated
extensions of a package. For example, to deactivate *all* python
extensions, use::
extensions, use:
spack deactivate -a python
.. code-block:: console
$ spack deactivate --all python
-----------------------
Filesystem requirements
@ -1796,7 +1828,7 @@ without ``flock`` support enabled. You can determine how your
filesystems are mounted with ``mount -p``. The output for a Lustre
filesystem might look like this:
.. code-block:: sh
.. code-block:: console
$ mount -l | grep lscratch
pilsner-mds1-lnet0@o2ib100:/lsd on /p/lscratchd type lustre (rw,nosuid,noauto,_netdev,lazystatfs,flock)
@ -1808,7 +1840,7 @@ system administrator to enable ``flock``.
This issue typically manifests with the error below:
.. code-block:: sh
.. code-block:: console
$ ./spack find
Traceback (most recent call last):
@ -1817,7 +1849,7 @@ This issue typically manifests with the error below:
File "./spack", line 154,' in main
return_val = command(parser, args)
File "./spack/lib/spack/spack/cmd/find.py", line 170, in find
specs = set(spack.installed_db.query(**q_args))
specs = set(spack.installed_db.query(\**q_args))
File "./spack/lib/spack/spack/database.py", line 551, in query
with self.read_transaction():
File "./spack/lib/spack/spack/database.py", line 598, in __enter__
@ -1832,12 +1864,7 @@ This issue typically manifests with the error below:
A nicer error message is TBD in future versions of Spack.
<<<<<<< c36f13e44dae52086ca48e4f16b514642bc91106
.. _cray-support:
=======
.. _spack-cray:
>>>>>>> Fix #1604 and fix #1605, title underline problems
-------------
Spack on Cray
@ -1846,34 +1873,34 @@ Spack on Cray
Spack differs slightly when used on a Cray system. The architecture spec
can differentiate between the front-end and back-end processor and operating system.
For example, on Edison at NERSC, the back-end target processor
is \"Ivy Bridge\", so you can specify to use the back-end this way:
is "Ivy Bridge", so you can specify to use the back-end this way:
.. code-block:: sh
.. code-block:: console
spack install zlib target=ivybridge
$ spack install zlib target=ivybridge
You can also use the operating system to build against the back-end:
.. code-block:: sh
.. code-block:: console
spack install zlib os=CNL10
$ spack install zlib os=CNL10
Notice that the name includes both the operating system name and the major
version number concatenated together.
Alternatively, if you want to build something for the front-end,
you can specify the front-end target processor. The processor for a login node
on Edison is \"Sandy bridge\" so we specify on the command line like so:
on Edison is "Sandy bridge" so we specify on the command line like so:
.. code-block:: sh
.. code-block:: console
spack install zlib target=sandybridge
$ spack install zlib target=sandybridge
And the front-end operating system is:
.. code-block:: sh
.. code-block:: console
spack install zlib os=SuSE11
$ spack install zlib os=SuSE11
^^^^^^^^^^^^^^^^^^^^^^^
Cray compiler detection
@ -1888,27 +1915,27 @@ compiler wrapper names (i.e. cc, CC, ftn). During build time, Spack will load
the correct PrgEnv and compiler module and will call appropriate wrapper.
The compilers.yaml config file will also differ. There is a
modules section that is filled with the compiler\'s Programming Environment
and module name. On other systems, this field is empty []::
modules section that is filled with the compiler's Programming Environment
and module name. On other systems, this field is empty []:
.. code-block:: yaml
...
- compiler:
modules:
- PrgEnv-intel
- intel/15.0.109
...
As mentioned earlier, the compiler paths will look different on a Cray system.
Since most compilers are invoked using cc, CC and ftn, the paths for each
compiler are replaced with their respective Cray compiler wrapper names::
compiler are replaced with their respective Cray compiler wrapper names:
.. code-block:: yaml
...
paths:
cc: cc
cxx: CC
f77: ftn
fc: ftn
...
As opposed to an explicit path to the compiler executable. This allows Spack
to call the Cray compiler wrappers during build time.
@ -1917,16 +1944,16 @@ For more on compiler configuration, check out :ref:`compiler-config`.
Spack sets the default Cray link type to dynamic, to better match other
other platforms. Individual packages can enable static linking (which is the
default outside of Spack on cray systems) using the -static flag.
default outside of Spack on cray systems) using the ``-static`` flag.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Setting defaults and using Cray modules
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you want to use default compilers for each PrgEnv and also be able
to load cray external modules, you will need to set up a packages.yaml.
to load cray external modules, you will need to set up a ``packages.yaml``.
Here\'s an example of an external configuration for cray modules:
Here's an example of an external configuration for cray modules:
.. code-block:: yaml
@ -1942,7 +1969,7 @@ environment variables, libraries, etc, that are brought into the environment
via module load.
You can set the default compiler that Spack can use for each compiler type.
If you want to use the Cray defaults, then set them under *all:* in packages.yaml.
If you want to use the Cray defaults, then set them under ``all:`` in packages.yaml.
In the compiler field, set the compiler specs in your order of preference.
Whenever you build with that compiler type, Spack will concretize to that version.
@ -1987,7 +2014,7 @@ Getting Help
^^^^^^^^^^^^^^
If you don't find what you need here, the ``help`` subcommand will
print out out a list of *all* of ``spack``'s options and subcommands:
print out out a list of *all* of spack's options and subcommands:
.. command-output:: spack help
@ -1996,5 +2023,5 @@ usage information for a particular subcommand:
.. command-output:: spack help install
Alternately, you can use ``spack -h`` in place of ``spack help``, or
``spack <subcommand> -h`` to get help on a particular subcommand.
Alternately, you can use ``spack --help`` in place of ``spack help``, or
``spack <subcommand> --help`` to get help on a particular subcommand.

18
lib/spack/docs/configuration.rst
View File

@ -10,7 +10,9 @@ Configuration
Temporary space
---------------
.. warning:: Temporary space configuration will eventually be moved to
.. warning::
Temporary space configuration will eventually be moved to
configuration files, but currently these settings are in
``lib/spack/spack/__init__.py``
@ -144,7 +146,7 @@ The preferred configuration can be controlled via the
Here's an example packages.yaml file that sets preferred packages:
.. code-block:: sh
.. code-block:: yaml
packages:
opencv:
@ -205,15 +207,15 @@ supply ``-p`` to Spack on the command line, before any subcommands.
``spack -p`` output looks like this:
.. code-block:: sh
.. code-block:: console
$ spack -p graph dyninst
o dyninst
|\
| |\
| o | libdwarf
|/ /
o | libelf
\|\
\| \|\
\| o \| libdwarf
\|/ /
o \| libelf
/
o boost

24
lib/spack/docs/developer_guide.rst
View File

@ -70,7 +70,9 @@ Directory Structure
-------------------
So that you can familiarize yourself with the project, we'll start
with a high level view of Spack's directory structure::
with a high level view of Spack's directory structure:
.. code-block:: none
spack/ <- installation root
bin/
@ -109,17 +111,14 @@ Spack is designed so that it could live within a `standard UNIX
directory hierarchy <http://linux.die.net/man/7/hier>`_, so ``lib``,
``var``, and ``opt`` all contain a ``spack`` subdirectory in case
Spack is installed alongside other software. Most of the interesting
parts of Spack live in ``lib/spack``. Files under ``var`` are created
as needed, so there is no ``var`` directory when you initially clone
Spack from the repository.
parts of Spack live in ``lib/spack``.
Spack has *one* directory layout and there is no install process.
version and the source code. Most Python programs don't look like
this (they use distutils, ``setup.py``, etc.) but we wanted to make
Spack *very* easy to use. The simple layout spares users from the
need to install Spack into a Python environment. Many users don't
have write access to a Python installation, and installing an entire
new instance of Python to bootstrap Spack would be very complicated.
Most Python programs don't look like this (they use distutils, ``setup.py``,
etc.) but we wanted to make Spack *very* easy to use. The simple layout
spares users from the need to install Spack into a Python environment.
Many users don't have write access to a Python installation, and installing
an entire new instance of Python to bootstrap Spack would be very complicated.
Users should not have to install install a big, complicated package to
use the thing that's supposed to spare them from the details of big,
complicated packages. The end result is that Spack works out of the
@ -283,8 +282,7 @@ Spec objects
Package objects
---------------
Most spack commands
look something like this:
Most spack commands look something like this:
#. Parse an abstract spec (or specs) from the command line,
#. *Normalize* the spec based on information in package files,
@ -295,7 +293,7 @@ look something like this:
The information in Package files is used at all stages in this
process.
Conceptually, packages are overloaded. They contain
Conceptually, packages are overloaded. They contain:
-------------
Stage objects

26
lib/spack/docs/features.rst
View File

@ -14,7 +14,7 @@ Simple package installation
Installing the default version of a package is simple. This will install
the latest version of the ``mpileaks`` package and all of its dependencies:
.. code-block:: sh
.. code-block:: console
$ spack install mpileaks
@ -26,7 +26,7 @@ Spack allows installation to be customized. Users can specify the
version, build compiler, compile-time options, and cross-compile
platform, all on the command line.
.. code-block:: sh
.. code-block:: console
# Install a particular version by appending @
$ spack install mpileaks@1.1.2
@ -59,7 +59,7 @@ customized extensively. Suppose that ``mpileaks`` depends indirectly
on ``libelf`` and ``libdwarf``. Using ``^``, users can add custom
configurations for the dependencies:
.. code-block:: sh
.. code-block:: console
# Install mpileaks and link it with specific versions of libelf and libdwarf
$ spack install mpileaks@1.1.2 %gcc@4.7.3 +debug ^libelf@0.8.12 ^libdwarf@20130729+debug
@ -91,7 +91,7 @@ in pure Python.
For example, this command:
.. code-block:: sh
.. code-block:: console
$ spack create http://www.mr511.de/software/libelf-0.8.13.tar.gz
@ -101,16 +101,26 @@ creates a simple python file:
from spack import *
class Libelf(Package):
homepage = "http://www.example.com/"
"""FIXME: Put a proper description of your package here."""
# FIXME: Add a proper url for your package's homepage here.
homepage = "http://www.example.com"
url = "http://www.mr511.de/software/libelf-0.8.13.tar.gz"
version('0.8.13', '4136d7b4c04df68b686570afa26988ac')
def install(self, prefix):
configure("--prefix=%s" % prefix)
# FIXME: Add dependencies if required.
# depends_on('foo')
def install(self, spec, prefix):
# FIXME: Modify the configure line to suit your build system here.
configure('--prefix={0}'.format(prefix))
# FIXME: Add logic to build and install here.
make()
make("install")
make('install')
It doesn't take much python coding to get from there to a working
package:

9
lib/spack/docs/getting_started.rst
View File

@ -9,7 +9,7 @@ Download
Getting spack is easy. You can clone it from the `github repository
<https://github.com/llnl/spack>`_ using this command:
.. code-block:: sh
.. code-block:: console
$ git clone https://github.com/llnl/spack.git
@ -18,7 +18,7 @@ full path to this directory is in the ``SPACK_ROOT`` environment
variable. Add ``$SPACK_ROOT/bin`` to your path and you're ready to
go:
.. code-block:: sh
.. code-block:: console
$ export PATH=$SPACK_ROOT/bin:$PATH
$ spack install libelf
@ -26,9 +26,10 @@ go:
For a richer experience, use Spack's `shell support
<http://software.llnl.gov/spack/basic_usage.html#environment-modules>`_:
.. code-block:: sh
.. code-block:: console
# For bash users
$ export SPACK_ROOT=/path/to/spack
$ . $SPACK_ROOT/share/spack/setup-env.sh
# For tcsh or csh users (note you must set SPACK_ROOT)
@ -48,7 +49,7 @@ You may want to run it out of a prefix other than the git repository
you cloned. The ``spack bootstrap`` command provides this
functionality. To install spack in a new directory, simply type:
.. code-block:: sh
.. code-block:: console
$ spack bootstrap /my/favorite/prefix

2
lib/spack/docs/index.rst
View File

@ -28,7 +28,7 @@ Get spack from the `github repository
<https://github.com/llnl/spack>`_ and install your first
package:
.. code-block:: sh
.. code-block:: console
$ git clone https://github.com/llnl/spack.git
$ cd spack/bin

48
lib/spack/docs/mirrors.rst
View File

@ -11,7 +11,9 @@ mirror is a URL that points to a directory, either on the local
filesystem or on some server, containing tarballs for all of Spack's
packages.
Here's an example of a mirror's directory structure::
Here's an example of a mirror's directory structure:
.. code-block:: none
mirror/
cmake/
@ -57,21 +59,9 @@ contains tarballs for each package, named after each package.
----------------
Mirrors are managed with the ``spack mirror`` command. The help for
``spack mirror`` looks like this::
``spack mirror`` looks like this:
$ spack mirror -h
usage: spack mirror [-h] SUBCOMMAND ...
positional arguments:
SUBCOMMAND
create Create a directory to be used as a spack mirror, and fill
it with package archives.
add Add a mirror to Spack.
remove Remove a mirror by name.
list Print out available mirrors to the console.
optional arguments:
-h, --help show this help message and exit
.. command-output:: spack help mirror
The ``create`` command actually builds a mirror by fetching all of its
packages from the internet and checksumming them.
@ -92,7 +82,7 @@ The command will iterate through all of Spack's packages and download
the safe ones into a directory structure like the one above. Here is
what it looks like:
.. code-block:: bash
.. code-block:: console
$ spack mirror create libelf libdwarf
==> Created new mirror in spack-mirror-2014-06-24
@ -134,7 +124,9 @@ Normally, ``spack mirror create`` downloads all the archives it has
checksums for. If you want to only create a mirror for a subset of
packages, you can do that by supplying a list of package specs on the
command line after ``spack mirror create``. For example, this
command::
command:
.. code-block:: console
$ spack mirror create libelf@0.8.12: boost@1.44:
@ -146,7 +138,9 @@ Mirror files
^^^^^^^^^^^^
If you have a *very* large number of packages you want to mirror, you
can supply a file with specs in it, one per line::
can supply a file with specs in it, one per line:
.. code-block:: console
$ cat specs.txt
libdwarf
@ -168,7 +162,9 @@ your site.
Once you have a mirror, you need to let spack know about it. This is
relatively simple. First, figure out the URL for the mirror. If it's
a file, you can use a file URL like this one::
a file, you can use a file URL like this one:
.. code-block:: none
file:///Users/gamblin2/spack-mirror-2014-06-24
@ -180,7 +176,7 @@ web server, you could use a URL like this one:
Spack will use the URL as the root for all of the packages it fetches.
You can tell your Spack installation to use that mirror like this:
.. code-block:: bash
.. code-block:: console
$ spack mirror add local_filesystem file:///Users/gamblin2/spack-mirror-2014-06-24
@ -192,7 +188,9 @@ Each mirror has a name so that you can refer to it again later.
``spack mirror list``
---------------------
To see all the mirrors Spack knows about, run ``spack mirror list``::
To see all the mirrors Spack knows about, run ``spack mirror list``:
.. code-block:: console
$ spack mirror list
local_filesystem file:///Users/gamblin2/spack-mirror-2014-06-24
@ -203,7 +201,9 @@ To see all the mirrors Spack knows about, run ``spack mirror list``::
``spack mirror remove``
-----------------------
To remove a mirror by name::
To remove a mirror by name, run:
.. code-block:: console
$ spack mirror remove local_filesystem
$ spack mirror list
@ -213,7 +213,9 @@ To remove a mirror by name::
Mirror precedence
-----------------
Adding a mirror really adds a line in ``~/.spack/mirrors.yaml``::
Adding a mirror really adds a line in ``~/.spack/mirrors.yaml``:
.. code-block:: yaml
mirrors:
local_filesystem: file:///Users/gamblin2/spack-mirror-2014-06-24

474
lib/spack/docs/packaging_guide.rst
View File

File diff suppressed because it is too large Load Diff