re-merged mainline develop
This commit is contained in:
@@ -102,8 +102,8 @@ that the packages is installed:
|
||||
==> adept-utils is already installed in /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/adept-utils@1.0-5adef8da.
|
||||
==> Trying to fetch from https://github.com/hpc/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz
|
||||
######################################################################## 100.0%
|
||||
==> Staging archive: /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7=chaos_5_x86_64_ib-59f6ad23/mpileaks-1.0.tar.gz
|
||||
==> Created stage in /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7=chaos_5_x86_64_ib-59f6ad23.
|
||||
==> Staging archive: /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=chaos_5_x86_64_ib-59f6ad23/mpileaks-1.0.tar.gz
|
||||
==> Created stage in /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=chaos_5_x86_64_ib-59f6ad23.
|
||||
==> No patches needed for mpileaks.
|
||||
==> Building mpileaks.
|
||||
|
||||
@@ -132,10 +132,10 @@ sites, as installing a version that one user needs will not disrupt
|
||||
existing installations for other users.
|
||||
|
||||
In addition to different versions, Spack can customize the compiler,
|
||||
compile-time options (variants), and platform (for cross compiles) of
|
||||
an installation. Spack is unique in that it can also configure the
|
||||
*dependencies* a package is built with. For example, two
|
||||
configurations of the same version of a package, one built with boost
|
||||
compile-time options (variants), compiler flags, and platform (for
|
||||
cross compiles) of an installation. Spack is unique in that it can
|
||||
also configure the *dependencies* a package is built with. For example,
|
||||
two configurations of the same version of a package, one built with boost
|
||||
1.39.0, and the other version built with version 1.43.0, can coexist.
|
||||
|
||||
This can all be done on the command line using the *spec* syntax.
|
||||
@@ -246,6 +246,12 @@ Packages are divided into groups according to their architecture and
|
||||
compiler. Within each group, Spack tries to keep the view simple, and
|
||||
only shows the version of installed packages.
|
||||
|
||||
``spack find`` can filter the package list based on the package name, spec, or
|
||||
a number of properties of their installation status. For example, missing
|
||||
dependencies of a spec can be shown with ``-m``, packages which were
|
||||
explicitly installed with ``spack install <package>`` can be singled out with
|
||||
``-e`` and those which have been pulled in only as dependencies with ``-E``.
|
||||
|
||||
In some cases, there may be different configurations of the *same*
|
||||
version of a package installed. For example, there are two
|
||||
installations of of ``libdwarf@20130729`` above. We can look at them
|
||||
@@ -328,6 +334,11 @@ of libelf would look like this:
|
||||
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
|
||||
libdwarf@20130729-d9b90962
|
||||
|
||||
We can also search for packages that have a certain attribute. For example,
|
||||
``spack find -l libdwarf +debug`` will show only installations of libdwarf
|
||||
with the 'debug' compile-time option enabled, while ``spack find -l +debug``
|
||||
will find every installed package with a 'debug' compile-time option enabled.
|
||||
|
||||
The full spec syntax is discussed in detail in :ref:`sec-specs`.
|
||||
|
||||
|
||||
@@ -457,6 +468,26 @@ For compilers, like ``clang``, that do not support Fortran, put
|
||||
Once you save the file, the configured compilers will show up in the
|
||||
list displayed by ``spack compilers``.
|
||||
|
||||
You can also add compiler flags to manually configured compilers. The
|
||||
valid flags are ``cflags``, ``cxxflags``, ``fflags``, ``cppflags``,
|
||||
``ldflags``, and ``ldlibs``. For example,::
|
||||
|
||||
...
|
||||
chaos_5_x86_64_ib:
|
||||
...
|
||||
intel@15.0.0:
|
||||
cc: /usr/local/bin/icc-15.0.024-beta
|
||||
cxx: /usr/local/bin/icpc-15.0.024-beta
|
||||
f77: /usr/local/bin/ifort-15.0.024-beta
|
||||
fc: /usr/local/bin/ifort-15.0.024-beta
|
||||
cppflags: -O3 -fPIC
|
||||
...
|
||||
|
||||
These flags will be treated by spack as if they were enterred from
|
||||
the command line each time this compiler is used. The compiler wrappers
|
||||
then inject those flags into the compiler command. Compiler flags
|
||||
enterred from the command line will be discussed in more detail in the
|
||||
following section.
|
||||
|
||||
.. _sec-specs:
|
||||
|
||||
@@ -474,7 +505,7 @@ the full syntax of specs.
|
||||
|
||||
Here is an example of a much longer spec than we've seen thus far::
|
||||
|
||||
mpileaks @1.2:1.4 %gcc@4.7.5 +debug -qt =bgqos_0 ^callpath @1.1 %gcc@4.7.2
|
||||
mpileaks @1.2:1.4 %gcc@4.7.5 +debug -qt arch=bgq_os ^callpath @1.1 %gcc@4.7.2
|
||||
|
||||
If provided to ``spack install``, this will install the ``mpileaks``
|
||||
library at some version between ``1.2`` and ``1.4`` (inclusive),
|
||||
@@ -492,8 +523,12 @@ More formally, a spec consists of the following pieces:
|
||||
* ``%`` Optional compiler specifier, with an optional compiler version
|
||||
(``gcc`` or ``gcc@4.7.3``)
|
||||
* ``+`` or ``-`` or ``~`` Optional variant specifiers (``+debug``,
|
||||
``-qt``, or ``~qt``)
|
||||
* ``=`` Optional architecture specifier (``bgqos_0``)
|
||||
``-qt``, or ``~qt``) for boolean variants
|
||||
* ``name=<value>`` Optional variant specifiers that are not restricted to
|
||||
boolean variants
|
||||
* ``name=<value>`` Optional compiler flag specifiers. Valid flag names are
|
||||
``cflags``, ``cxxflags``, ``fflags``, ``cppflags``, ``ldflags``, and ``ldlibs``.
|
||||
* ``arch=<value>`` Optional architecture specifier (``arch=bgq_os``)
|
||||
* ``^`` Dependency specs (``^callpath@1.1``)
|
||||
|
||||
There are two things to notice here. The first is that specs are
|
||||
@@ -573,7 +608,7 @@ compilers, variants, and architectures just like any other spec.
|
||||
Specifiers are associated with the nearest package name to their left.
|
||||
For example, above, ``@1.1`` and ``%gcc@4.7.2`` associates with the
|
||||
``callpath`` package, while ``@1.2:1.4``, ``%gcc@4.7.5``, ``+debug``,
|
||||
``-qt``, and ``=bgqos_0`` all associate with the ``mpileaks`` package.
|
||||
``-qt``, and ``arch=bgq_os`` all associate with the ``mpileaks`` package.
|
||||
|
||||
In the diagram above, ``mpileaks`` depends on ``mpich`` with an
|
||||
unspecified version, but packages can depend on other packages with
|
||||
@@ -629,22 +664,25 @@ based on site policies.
|
||||
Variants
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. Note::
|
||||
|
||||
Variants are not yet supported, but will be in the next Spack
|
||||
release (0.9), due in Q2 2015.
|
||||
|
||||
Variants are named options associated with a particular package, and
|
||||
they can be turned on or off. For example, above, supplying
|
||||
``+debug`` causes ``mpileaks`` to be built with debug flags. The
|
||||
names of particular variants available for a package depend on what
|
||||
was provided by the package author. ``spack info <package>`` will
|
||||
Variants are named options associated with a particular package. They are
|
||||
optional, as each package must provide default values for each variant it
|
||||
makes available. Variants can be specified using
|
||||
a flexible parameter syntax ``name=<value>``. For example,
|
||||
``spack install libelf debug=True`` will install libelf build with debug
|
||||
flags. The names of particular variants available for a package depend on
|
||||
what was provided by the package author. ``spack into <package>`` will
|
||||
provide information on what build variants are available.
|
||||
|
||||
Depending on the package a variant may be on or off by default. For
|
||||
``mpileaks`` here, ``debug`` is off by default, and we turned it on
|
||||
with ``+debug``. If a package is on by default you can turn it off by
|
||||
either adding ``-name`` or ``~name`` to the spec.
|
||||
For compatibility with earlier versions, variants which happen to be
|
||||
boolean in nature can be specified by a syntax that represents turning
|
||||
options on and off. For example, in the previous spec we could have
|
||||
supplied ``libelf +debug`` with the same effect of enabling the debug
|
||||
compile time option for the libelf package.
|
||||
|
||||
Depending on the package a variant may have any default value. For
|
||||
``libelf`` here, ``debug`` is ``False`` by default, and we turned it on
|
||||
with ``debug=True`` or ``+debug``. If a package is ``True`` by default
|
||||
you can turn it off by either adding ``-name`` or ``~name`` to the spec.
|
||||
|
||||
There are two syntaxes here because, depending on context, ``~`` and
|
||||
``-`` may mean different things. In most shells, the following will
|
||||
@@ -656,7 +694,7 @@ result in the shell performing home directory substitution:
|
||||
mpileaks~debug # use this instead
|
||||
|
||||
If there is a user called ``debug``, the ``~`` will be incorrectly
|
||||
expanded. In this situation, you would want to write ``mpileaks
|
||||
expanded. In this situation, you would want to write ``libelf
|
||||
-debug``. However, ``-`` can be ambiguous when included after a
|
||||
package name without spaces:
|
||||
|
||||
@@ -671,12 +709,35 @@ package, not a request for ``mpileaks`` built without ``debug``
|
||||
options. In this scenario, you should write ``mpileaks~debug`` to
|
||||
avoid ambiguity.
|
||||
|
||||
When spack normalizes specs, it prints them out with no spaces and
|
||||
uses only ``~`` for disabled variants. We allow ``-`` and spaces on
|
||||
the command line is provided for convenience and legibility.
|
||||
When spack normalizes specs, it prints them out with no spaces boolean
|
||||
variants using the backwards compatibility syntax and uses only ``~``
|
||||
for disabled boolean variants. We allow ``-`` and spaces on the command
|
||||
line is provided for convenience and legibility.
|
||||
|
||||
|
||||
Architecture specifier
|
||||
Compiler Flags
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Compiler flags are specified using the same syntax as non-boolean variants,
|
||||
but fulfill a different purpose. While the function of a variant is set by
|
||||
the package, compiler flags are used by the compiler wrappers to inject
|
||||
flags into the compile line of the build. Additionally, compiler flags are
|
||||
inherited by dependencies. ``spack install libdwarf cppflags=\"-g\"`` will
|
||||
install both libdwarf and libelf with the ``-g`` flag injected into their
|
||||
compile line.
|
||||
|
||||
Notice that the value of the compiler flags must be escape quoted on the
|
||||
command line. From within python files, the same spec would be specified
|
||||
``libdwarf cppflags="-g"``. This is necessary because of how the shell
|
||||
handles the quote symbols.
|
||||
|
||||
The six compiler flags are injected in the order of implicit make commands
|
||||
in gnu autotools. If all flags are set, the order is
|
||||
``$cppflags $cflags|$cxxflags $ldflags command $ldlibs`` for C and C++ and
|
||||
``$fflags $cppflags $ldflags command $ldlibs`` for fortran.
|
||||
|
||||
|
||||
Architecture specifiers
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. Note::
|
||||
@@ -684,12 +745,9 @@ Architecture specifier
|
||||
Architecture specifiers are part of specs but are not yet
|
||||
functional. They will be in Spack version 1.0, due in Q3 2015.
|
||||
|
||||
The architecture specifier starts with a ``=`` and also comes after
|
||||
some package name within a spec. It allows a user to specify a
|
||||
particular architecture for the package to be built. This is mostly
|
||||
used for architectures that need cross-compilation, and in most cases,
|
||||
users will not need to specify the architecture when they install a
|
||||
package.
|
||||
The architecture specifier looks identical to a variant specifier for a
|
||||
non-boolean variant. The architecture can be specified only using the
|
||||
reserved name ``arch`` (``arch=bgq_os``).
|
||||
|
||||
|
||||
.. _sec-virtual-dependencies:
|
||||
@@ -767,6 +825,23 @@ any MPI implementation will do. If another package depends on
|
||||
error. Likewise, if you try to plug in some package that doesn't
|
||||
provide MPI, Spack will raise an error.
|
||||
|
||||
Specifying Specs by Hash
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Complicated specs can become cumbersome to enter on the command line,
|
||||
especially when many of the qualifications are necessary to
|
||||
distinguish between similar installs, for example when using the
|
||||
``uninstall`` command. To avoid this, when referencing an existing spec,
|
||||
Spack allows you to reference specs by their hash. We previously
|
||||
discussed the spec hash that Spack computes. In place of a spec in any
|
||||
command, substitute ``/<hash>`` where ``<hash>`` is any amount from
|
||||
the beginning of a spec hash. If the given spec hash is sufficient
|
||||
to be unique, Spack will replace the reference with the spec to which
|
||||
it refers. Otherwise, it will prompt for a more qualified hash.
|
||||
|
||||
Note that this will not work to reinstall a depencency uninstalled by
|
||||
``spack uninstall -f``.
|
||||
|
||||
.. _spack-providers:
|
||||
|
||||
``spack providers``
|
||||
@@ -996,8 +1071,8 @@ than one installed package matches it), then Spack will warn you:
|
||||
|
||||
$ spack load libelf
|
||||
==> Error: Multiple matches for spec libelf. Choose one:
|
||||
libelf@0.8.13%gcc@4.4.7=chaos_5_x86_64_ib
|
||||
libelf@0.8.13%intel@15.0.0=chaos_5_x86_64_ib
|
||||
libelf@0.8.13%gcc@4.4.7 arch=chaos_5_x86_64_ib
|
||||
libelf@0.8.13%intel@15.0.0 arch=chaos_5_x86_64_ib
|
||||
|
||||
You can either type the ``spack load`` command again with a fully
|
||||
qualified argument, or you can add just enough extra constraints to
|
||||
@@ -1276,7 +1351,7 @@ You can find extensions for your Python installation like this:
|
||||
.. code-block:: sh
|
||||
|
||||
$ spack extensions python
|
||||
==> python@2.7.8%gcc@4.4.7=chaos_5_x86_64_ib-703c7a96
|
||||
==> python@2.7.8%gcc@4.4.7 arch=chaos_5_x86_64_ib-703c7a96
|
||||
==> 36 extensions:
|
||||
geos py-ipython py-pexpect py-pyside py-sip
|
||||
py-basemap py-libxml2 py-pil py-pytz py-six
|
||||
@@ -1366,9 +1441,9 @@ installation:
|
||||
.. code-block:: sh
|
||||
|
||||
$ spack activate py-numpy
|
||||
==> Activated extension py-setuptools@11.3.1%gcc@4.4.7=chaos_5_x86_64_ib-3c74eb69 for python@2.7.8%gcc@4.4.7.
|
||||
==> Activated extension py-nose@1.3.4%gcc@4.4.7=chaos_5_x86_64_ib-5f70f816 for python@2.7.8%gcc@4.4.7.
|
||||
==> Activated extension py-numpy@1.9.1%gcc@4.4.7=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7.
|
||||
==> Activated extension py-setuptools@11.3.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-3c74eb69 for python@2.7.8%gcc@4.4.7.
|
||||
==> Activated extension py-nose@1.3.4%gcc@4.4.7 arch=chaos_5_x86_64_ib-5f70f816 for python@2.7.8%gcc@4.4.7.
|
||||
==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7.
|
||||
|
||||
Several things have happened here. The user requested that
|
||||
``py-numpy`` be activated in the ``python`` installation it was built
|
||||
@@ -1383,7 +1458,7 @@ packages listed as activated:
|
||||
.. code-block:: sh
|
||||
|
||||
$ spack extensions python
|
||||
==> python@2.7.8%gcc@4.4.7=chaos_5_x86_64_ib-703c7a96
|
||||
==> python@2.7.8%gcc@4.4.7 arch=chaos_5_x86_64_ib-703c7a96
|
||||
==> 36 extensions:
|
||||
geos py-ipython py-pexpect py-pyside py-sip
|
||||
py-basemap py-libxml2 py-pil py-pytz py-six
|
||||
@@ -1431,7 +1506,7 @@ dependencies, you can use ``spack activate -f``:
|
||||
.. code-block:: sh
|
||||
|
||||
$ spack activate -f py-numpy
|
||||
==> Activated extension py-numpy@1.9.1%gcc@4.4.7=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7.
|
||||
==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7.
|
||||
|
||||
.. _spack-deactivate:
|
||||
|
||||
|
@@ -1,6 +1,6 @@
|
||||
.. _site-configuration:
|
||||
.. _configuration:
|
||||
|
||||
Site configuration
|
||||
Configuration
|
||||
===================================
|
||||
|
||||
.. _temp-space:
|
||||
@@ -55,7 +55,7 @@ directory is.
|
||||
|
||||
|
||||
External Packages
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
----------------------------
|
||||
Spack can be configured to use externally-installed
|
||||
packages rather than building its own packages. This may be desirable
|
||||
if machines ship with system packages, such as a customized MPI
|
||||
@@ -70,15 +70,15 @@ directory. Here's an example of an external configuration:
|
||||
packages:
|
||||
openmpi:
|
||||
paths:
|
||||
openmpi@1.4.3%gcc@4.4.7=chaos_5_x86_64_ib: /opt/openmpi-1.4.3
|
||||
openmpi@1.4.3%gcc@4.4.7=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug
|
||||
openmpi@1.6.5%intel@10.1=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel
|
||||
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib: /opt/openmpi-1.4.3
|
||||
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug
|
||||
openmpi@1.6.5%intel@10.1 arch=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel
|
||||
|
||||
This example lists three installations of OpenMPI, one built with gcc,
|
||||
one built with gcc and debug information, and another built with Intel.
|
||||
If Spack is asked to build a package that uses one of these MPIs as a
|
||||
dependency, it will use the the pre-installed OpenMPI in
|
||||
the given directory.
|
||||
the given directory.
|
||||
|
||||
Each ``packages.yaml`` begins with a ``packages:`` token, followed
|
||||
by a list of package names. To specify externals, add a ``paths``
|
||||
@@ -108,9 +108,9 @@ be:
|
||||
packages:
|
||||
openmpi:
|
||||
paths:
|
||||
openmpi@1.4.3%gcc@4.4.7=chaos_5_x86_64_ib: /opt/openmpi-1.4.3
|
||||
openmpi@1.4.3%gcc@4.4.7=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug
|
||||
openmpi@1.6.5%intel@10.1=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel
|
||||
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib: /opt/openmpi-1.4.3
|
||||
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug
|
||||
openmpi@1.6.5%intel@10.1 arch=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel
|
||||
buildable: False
|
||||
|
||||
The addition of the ``buildable`` flag tells Spack that it should never build
|
||||
@@ -118,13 +118,73 @@ its own version of OpenMPI, and it will instead always rely on a pre-built
|
||||
OpenMPI. Similar to ``paths``, ``buildable`` is specified as a property under
|
||||
a package name.
|
||||
|
||||
The ``buildable`` does not need to be paired with external packages.
|
||||
It could also be used alone to forbid packages that may be
|
||||
The ``buildable`` does not need to be paired with external packages.
|
||||
It could also be used alone to forbid packages that may be
|
||||
buggy or otherwise undesirable.
|
||||
|
||||
|
||||
Concretization Preferences
|
||||
--------------------------------
|
||||
|
||||
Spack can be configured to prefer certain compilers, package
|
||||
versions, depends_on, and variants during concretization.
|
||||
The preferred configuration can be controlled via the
|
||||
``~/.spack/packages.yaml`` file for user configuations, or the
|
||||
``etc/spack/packages.yaml`` site configuration.
|
||||
|
||||
|
||||
Here's an example packages.yaml file that sets preferred packages:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
packages:
|
||||
dyninst:
|
||||
compiler: [gcc@4.9]
|
||||
variants: +debug
|
||||
gperftools:
|
||||
version: [2.2, 2.4, 2.3]
|
||||
all:
|
||||
compiler: [gcc@4.4.7, gcc@4.6:, intel, clang, pgi]
|
||||
providers:
|
||||
mpi: [mvapich, mpich, openmpi]
|
||||
|
||||
|
||||
At a high level, this example is specifying how packages should be
|
||||
concretized. The dyninst package should prefer using gcc 4.9 and
|
||||
be built with debug options. The gperftools package should prefer version
|
||||
2.2 over 2.4. Every package on the system should prefer mvapich for
|
||||
its MPI and gcc 4.4.7 (except for Dyninst, which overrides this by preferring gcc 4.9).
|
||||
These options are used to fill in implicit defaults. Any of them can be overwritten
|
||||
on the command line if explicitly requested.
|
||||
|
||||
Each packages.yaml file begins with the string ``packages:`` and
|
||||
package names are specified on the next level. The special string ``all``
|
||||
applies settings to each package. Underneath each package name is
|
||||
one or more components: ``compiler``, ``variants``, ``version``,
|
||||
or ``providers``. Each component has an ordered list of spec
|
||||
``constraints``, with earlier entries in the list being preferred over
|
||||
later entries.
|
||||
|
||||
Sometimes a package installation may have constraints that forbid
|
||||
the first concretization rule, in which case Spack will use the first
|
||||
legal concretization rule. Going back to the example, if a user
|
||||
requests gperftools 2.3 or later, then Spack will install version 2.4
|
||||
as the 2.4 version of gperftools is preferred over 2.3.
|
||||
|
||||
An explicit concretization rule in the preferred section will always
|
||||
take preference over unlisted concretizations. In the above example,
|
||||
xlc isn't listed in the compiler list. Every listed compiler from
|
||||
gcc to pgi will thus be preferred over the xlc compiler.
|
||||
|
||||
The syntax for the ``provider`` section differs slightly from other
|
||||
concretization rules. A provider lists a value that packages may
|
||||
``depend_on`` (e.g, mpi) and a list of rules for fulfilling that
|
||||
dependency.
|
||||
|
||||
|
||||
|
||||
Profiling
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
------------------
|
||||
|
||||
Spack has some limited built-in support for profiling, and can report
|
||||
statistics using standard Python timing tools. To use this feature,
|
||||
@@ -133,7 +193,7 @@ supply ``-p`` to Spack on the command line, before any subcommands.
|
||||
.. _spack-p:
|
||||
|
||||
``spack -p``
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
``spack -p`` output looks like this:
|
||||
|
@@ -31,14 +31,21 @@ platform, all on the command line.
|
||||
# Specify a compiler (and its version), with %
|
||||
$ spack install mpileaks@1.1.2 %gcc@4.7.3
|
||||
|
||||
# Add special compile-time options with +
|
||||
# Add special compile-time options by name
|
||||
$ spack install mpileaks@1.1.2 %gcc@4.7.3 debug=True
|
||||
|
||||
# Add special boolean compile-time options with +
|
||||
$ spack install mpileaks@1.1.2 %gcc@4.7.3 +debug
|
||||
|
||||
# Cross-compile for a different architecture with =
|
||||
$ spack install mpileaks@1.1.2 =bgqos_0
|
||||
# Add compiler flags using the conventional names
|
||||
$ spack install mpileaks@1.1.2 %gcc@4.7.3 cppflags=\"-O3 -floop-block\"
|
||||
|
||||
Users can specify as many or few options as they care about. Spack
|
||||
will fill in the unspecified values with sensible defaults.
|
||||
# Cross-compile for a different architecture with arch=
|
||||
$ spack install mpileaks@1.1.2 arch=bgqos_0
|
||||
|
||||
Users can specify as many or few options as they care about. Spack
|
||||
will fill in the unspecified values with sensible defaults. The two listed
|
||||
syntaxes for variants are identical when the value is boolean.
|
||||
|
||||
|
||||
Customize dependencies
|
||||
|
@@ -47,7 +47,7 @@ Table of Contents
|
||||
basic_usage
|
||||
packaging_guide
|
||||
mirrors
|
||||
site_configuration
|
||||
configuration
|
||||
developer_guide
|
||||
command_index
|
||||
package_list
|
||||
|
@@ -1221,11 +1221,13 @@ just as easily provide a version range:
|
||||
|
||||
depends_on("libelf@0.8.2:0.8.4:")
|
||||
|
||||
Or a requirement for a particular variant:
|
||||
Or a requirement for a particular variant or compiler flags:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
depends_on("libelf@0.8+debug")
|
||||
depends_on('libelf debug=True')
|
||||
depends_on('libelf cppflags="-fPIC")
|
||||
|
||||
Both users *and* package authors can use the same spec syntax to refer
|
||||
to different package configurations. Users use the spec syntax on the
|
||||
@@ -1623,21 +1625,21 @@ the user runs ``spack install`` and the time the ``install()`` method
|
||||
is called. The concretized version of the spec above might look like
|
||||
this::
|
||||
|
||||
mpileaks@2.3%gcc@4.7.3=linux-ppc64
|
||||
^callpath@1.0%gcc@4.7.3+debug=linux-ppc64
|
||||
^dyninst@8.1.2%gcc@4.7.3=linux-ppc64
|
||||
^libdwarf@20130729%gcc@4.7.3=linux-ppc64
|
||||
^libelf@0.8.11%gcc@4.7.3=linux-ppc64
|
||||
^mpich@3.0.4%gcc@4.7.3=linux-ppc64
|
||||
mpileaks@2.3%gcc@4.7.3 arch=linux-ppc64
|
||||
^callpath@1.0%gcc@4.7.3+debug arch=linux-ppc64
|
||||
^dyninst@8.1.2%gcc@4.7.3 arch=linux-ppc64
|
||||
^libdwarf@20130729%gcc@4.7.3 arch=linux-ppc64
|
||||
^libelf@0.8.11%gcc@4.7.3 arch=linux-ppc64
|
||||
^mpich@3.0.4%gcc@4.7.3 arch=linux-ppc64
|
||||
|
||||
.. graphviz::
|
||||
|
||||
digraph {
|
||||
"mpileaks@2.3\n%gcc@4.7.3\n=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n=linux-ppc64"
|
||||
"mpileaks@2.3\n%gcc@4.7.3\n=linux-ppc64" -> "callpath@1.0\n%gcc@4.7.3+debug\n=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n=linux-ppc64"
|
||||
"callpath@1.0\n%gcc@4.7.3+debug\n=linux-ppc64" -> "dyninst@8.1.2\n%gcc@4.7.3\n=linux-ppc64"
|
||||
"dyninst@8.1.2\n%gcc@4.7.3\n=linux-ppc64" -> "libdwarf@20130729\n%gcc@4.7.3\n=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n=linux-ppc64"
|
||||
"dyninst@8.1.2\n%gcc@4.7.3\n=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n=linux-ppc64"
|
||||
"mpileaks@2.3\n%gcc@4.7.3\n arch=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n arch=linux-ppc64"
|
||||
"mpileaks@2.3\n%gcc@4.7.3\n arch=linux-ppc64" -> "callpath@1.0\n%gcc@4.7.3+debug\n arch=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n arch=linux-ppc64"
|
||||
"callpath@1.0\n%gcc@4.7.3+debug\n arch=linux-ppc64" -> "dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-ppc64"
|
||||
"dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-ppc64" -> "libdwarf@20130729\n%gcc@4.7.3\n arch=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n arch=linux-ppc64"
|
||||
"dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n arch=linux-ppc64"
|
||||
}
|
||||
|
||||
Here, all versions, compilers, and platforms are filled in, and there
|
||||
@@ -1648,8 +1650,8 @@ point will Spack call the ``install()`` method for your package.
|
||||
Concretization in Spack is based on certain selection policies that
|
||||
tell Spack how to select, e.g., a version, when one is not specified
|
||||
explicitly. Concretization policies are discussed in more detail in
|
||||
:ref:`site-configuration`. Sites using Spack can customize them to
|
||||
match the preferences of their own users.
|
||||
:ref:`configuration`. Sites using Spack can customize them to match
|
||||
the preferences of their own users.
|
||||
|
||||
.. _spack-spec:
|
||||
|
||||
@@ -1666,9 +1668,9 @@ running ``spack spec``. For example:
|
||||
^libdwarf
|
||||
^libelf
|
||||
|
||||
dyninst@8.0.1%gcc@4.7.3=linux-ppc64
|
||||
^libdwarf@20130729%gcc@4.7.3=linux-ppc64
|
||||
^libelf@0.8.13%gcc@4.7.3=linux-ppc64
|
||||
dyninst@8.0.1%gcc@4.7.3 arch=linux-ppc64
|
||||
^libdwarf@20130729%gcc@4.7.3 arch=linux-ppc64
|
||||
^libelf@0.8.13%gcc@4.7.3 arch=linux-ppc64
|
||||
|
||||
This is useful when you want to know exactly what Spack will do when
|
||||
you ask for a particular spec.
|
||||
@@ -1682,60 +1684,8 @@ be concretized on their system. For example, one user may prefer packages
|
||||
built with OpenMPI and the Intel compiler. Another user may prefer
|
||||
packages be built with MVAPICH and GCC.
|
||||
|
||||
Spack can be configured to prefer certain compilers, package
|
||||
versions, depends_on, and variants during concretization.
|
||||
The preferred configuration can be controlled via the
|
||||
``~/.spack/packages.yaml`` file for user configuations, or the
|
||||
``etc/spack/packages.yaml`` site configuration.
|
||||
|
||||
|
||||
Here's an example packages.yaml file that sets preferred packages:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
packages:
|
||||
dyninst:
|
||||
compiler: [gcc@4.9]
|
||||
variants: +debug
|
||||
gperftools:
|
||||
version: [2.2, 2.4, 2.3]
|
||||
all:
|
||||
compiler: [gcc@4.4.7, gcc@4.6:, intel, clang, pgi]
|
||||
providers:
|
||||
mpi: [mvapich, mpich, openmpi]
|
||||
|
||||
|
||||
At a high level, this example is specifying how packages should be
|
||||
concretized. The dyninst package should prefer using gcc 4.9 and
|
||||
be built with debug options. The gperftools package should prefer version
|
||||
2.2 over 2.4. Every package on the system should prefer mvapich for
|
||||
its MPI and gcc 4.4.7 (except for Dyninst, which overrides this by preferring gcc 4.9).
|
||||
These options are used to fill in implicit defaults. Any of them can be overwritten
|
||||
on the command line if explicitly requested.
|
||||
|
||||
Each packages.yaml file begins with the string ``packages:`` and
|
||||
package names are specified on the next level. The special string ``all``
|
||||
applies settings to each package. Underneath each package name is
|
||||
one or more components: ``compiler``, ``variants``, ``version``,
|
||||
or ``providers``. Each component has an ordered list of spec
|
||||
``constraints``, with earlier entries in the list being preferred over
|
||||
later entries.
|
||||
|
||||
Sometimes a package installation may have constraints that forbid
|
||||
the first concretization rule, in which case Spack will use the first
|
||||
legal concretization rule. Going back to the example, if a user
|
||||
requests gperftools 2.3 or later, then Spack will install version 2.4
|
||||
as the 2.4 version of gperftools is preferred over 2.3.
|
||||
|
||||
An explicit concretization rule in the preferred section will always
|
||||
take preference over unlisted concretizations. In the above example,
|
||||
xlc isn't listed in the compiler list. Every listed compiler from
|
||||
gcc to pgi will thus be preferred over the xlc compiler.
|
||||
|
||||
The syntax for the ``provider`` section differs slightly from other
|
||||
concretization rules. A provider lists a value that packages may
|
||||
``depend_on`` (e.g, mpi) and a list of rules for fulfilling that
|
||||
dependency.
|
||||
See the `documentation in the config section <concretization-preferences_>`_
|
||||
for more details.
|
||||
|
||||
.. _install-method:
|
||||
|
||||
@@ -1960,6 +1910,12 @@ the command line.
|
||||
``$rpath_flag`` can be overriden on a compiler specific basis in
|
||||
``lib/spack/spack/compilers/$compiler.py``.
|
||||
|
||||
The compiler wrappers also pass the compiler flags specified by the user from
|
||||
the command line (``cflags``, ``cxxflags``, ``fflags``, ``cppflags``, ``ldflags``,
|
||||
and/or ``ldlibs``). They do not override the canonical autotools flags with the
|
||||
same names (but in ALL-CAPS) that may be passed into the build by particularly
|
||||
challenging package scripts.
|
||||
|
||||
Compiler flags
|
||||
~~~~~~~~~~~~~~
|
||||
In rare circumstances such as compiling and running small unit tests, a package
|
||||
@@ -2206,12 +2162,12 @@ example:
|
||||
def install(self, prefix):
|
||||
# Do default install
|
||||
|
||||
@when('=chaos_5_x86_64_ib')
|
||||
@when('arch=chaos_5_x86_64_ib')
|
||||
def install(self, prefix):
|
||||
# This will be executed instead of the default install if
|
||||
# the package's sys_type() is chaos_5_x86_64_ib.
|
||||
|
||||
@when('=bgqos_0")
|
||||
@when('arch=bgqos_0")
|
||||
def install(self, prefix):
|
||||
# This will be executed if the package's sys_type is bgqos_0
|
||||
|
||||
@@ -2801,11 +2757,11 @@ build it:
|
||||
$ spack stage libelf
|
||||
==> Trying to fetch from http://www.mr511.de/software/libelf-0.8.13.tar.gz
|
||||
######################################################################## 100.0%
|
||||
==> Staging archive: /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3=linux-ppc64/libelf-0.8.13.tar.gz
|
||||
==> Created stage in /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3=linux-ppc64.
|
||||
==> Staging archive: /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-ppc64/libelf-0.8.13.tar.gz
|
||||
==> Created stage in /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-ppc64.
|
||||
$ spack cd libelf
|
||||
$ pwd
|
||||
/Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3=linux-ppc64/libelf-0.8.13
|
||||
/Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-ppc64/libelf-0.8.13
|
||||
|
||||
``spack cd`` here changed he current working directory to the
|
||||
directory containing the expanded ``libelf`` source code. There are a
|
||||
|
@@ -38,71 +38,59 @@
|
||||
|
||||
def setup_parser(subparser):
|
||||
format_group = subparser.add_mutually_exclusive_group()
|
||||
format_group.add_argument('-s',
|
||||
'--short',
|
||||
format_group.add_argument('-s', '--short',
|
||||
action='store_const',
|
||||
dest='mode',
|
||||
const='short',
|
||||
help='Show only specs (default)')
|
||||
format_group.add_argument('-p',
|
||||
'--paths',
|
||||
format_group.add_argument('-p', '--paths',
|
||||
action='store_const',
|
||||
dest='mode',
|
||||
const='paths',
|
||||
help='Show paths to package install directories')
|
||||
format_group.add_argument(
|
||||
'-d',
|
||||
'--deps',
|
||||
'-d', '--deps',
|
||||
action='store_const',
|
||||
dest='mode',
|
||||
const='deps',
|
||||
help='Show full dependency DAG of installed packages')
|
||||
|
||||
subparser.add_argument('-l',
|
||||
'--long',
|
||||
subparser.add_argument('-l', '--long',
|
||||
action='store_true',
|
||||
dest='long',
|
||||
help='Show dependency hashes as well as versions.')
|
||||
subparser.add_argument('-L',
|
||||
'--very-long',
|
||||
subparser.add_argument('-L', '--very-long',
|
||||
action='store_true',
|
||||
dest='very_long',
|
||||
help='Show dependency hashes as well as versions.')
|
||||
subparser.add_argument('-f',
|
||||
'--show-flags',
|
||||
subparser.add_argument('-f', '--show-flags',
|
||||
action='store_true',
|
||||
dest='show_flags',
|
||||
help='Show spec compiler flags.')
|
||||
|
||||
subparser.add_argument(
|
||||
'-e',
|
||||
'--explicit',
|
||||
'-e', '--explicit',
|
||||
action='store_true',
|
||||
help='Show only specs that were installed explicitly')
|
||||
subparser.add_argument(
|
||||
'-E',
|
||||
'--implicit',
|
||||
'-E', '--implicit',
|
||||
action='store_true',
|
||||
help='Show only specs that were installed as dependencies')
|
||||
subparser.add_argument(
|
||||
'-u',
|
||||
'--unknown',
|
||||
'-u', '--unknown',
|
||||
action='store_true',
|
||||
dest='unknown',
|
||||
help='Show only specs Spack does not have a package for.')
|
||||
subparser.add_argument(
|
||||
'-m',
|
||||
'--missing',
|
||||
'-m', '--missing',
|
||||
action='store_true',
|
||||
dest='missing',
|
||||
help='Show missing dependencies as well as installed specs.')
|
||||
subparser.add_argument('-M',
|
||||
'--only-missing',
|
||||
subparser.add_argument('-M', '--only-missing',
|
||||
action='store_true',
|
||||
dest='only_missing',
|
||||
help='Show only missing dependencies.')
|
||||
subparser.add_argument('-N',
|
||||
'--namespace',
|
||||
subparser.add_argument('-N', '--namespace',
|
||||
action='store_true',
|
||||
help='Show fully qualified package names.')
|
||||
|
||||
@@ -188,7 +176,32 @@ def fmt(s):
|
||||
print(hsh + spec.format(format_string, color=True) + '\n')
|
||||
|
||||
else:
|
||||
raise ValueError("Invalid mode for display_specs: %s. Must be one of (paths, deps, short)." % mode) # NOQA: ignore=E501
|
||||
raise ValueError(
|
||||
"Invalid mode for display_specs: %s. Must be one of (paths,"
|
||||
"deps, short)." % mode) # NOQA: ignore=E501
|
||||
|
||||
|
||||
def query_arguments(args):
|
||||
# Check arguments
|
||||
if args.explicit and args.implicit:
|
||||
tty.error('You can\'t pass -E and -e options simultaneously.')
|
||||
raise SystemExit(1)
|
||||
|
||||
# Set up query arguments.
|
||||
installed, known = True, any
|
||||
if args.only_missing:
|
||||
installed = False
|
||||
elif args.missing:
|
||||
installed = any
|
||||
if args.unknown:
|
||||
known = False
|
||||
explicit = any
|
||||
if args.explicit:
|
||||
explicit = True
|
||||
if args.implicit:
|
||||
explicit = False
|
||||
q_args = {'installed': installed, 'known': known, "explicit": explicit}
|
||||
return q_args
|
||||
|
||||
|
||||
def find(parser, args):
|
||||
@@ -205,22 +218,7 @@ def find(parser, args):
|
||||
if not query_specs:
|
||||
return
|
||||
|
||||
# Set up query arguments.
|
||||
installed, known = True, any
|
||||
if args.only_missing:
|
||||
installed = False
|
||||
elif args.missing:
|
||||
installed = any
|
||||
if args.unknown:
|
||||
known = False
|
||||
|
||||
explicit = any
|
||||
if args.explicit:
|
||||
explicit = False
|
||||
if args.implicit:
|
||||
explicit = True
|
||||
|
||||
q_args = {'installed': installed, 'known': known, "explicit": explicit}
|
||||
q_args = query_arguments(args)
|
||||
|
||||
# Get all the specs the user asked for
|
||||
if not query_specs:
|
||||
|
@@ -40,7 +40,6 @@
|
||||
|
||||
"""
|
||||
import os
|
||||
import time
|
||||
import socket
|
||||
|
||||
import yaml
|
||||
@@ -56,6 +55,7 @@
|
||||
from spack.error import SpackError
|
||||
from spack.repository import UnknownPackageError
|
||||
|
||||
|
||||
# DB goes in this directory underneath the root
|
||||
_db_dirname = '.spack-db'
|
||||
|
||||
@@ -69,10 +69,12 @@
|
||||
def _autospec(function):
|
||||
"""Decorator that automatically converts the argument of a single-arg
|
||||
function to a Spec."""
|
||||
|
||||
def converter(self, spec_like, *args, **kwargs):
|
||||
if not isinstance(spec_like, spack.spec.Spec):
|
||||
spec_like = spack.spec.Spec(spec_like)
|
||||
return function(self, spec_like, *args, **kwargs)
|
||||
|
||||
return converter
|
||||
|
||||
|
||||
@@ -92,6 +94,7 @@ class InstallRecord(object):
|
||||
dependents left.
|
||||
|
||||
"""
|
||||
|
||||
def __init__(self, spec, path, installed, ref_count=0, explicit=False):
|
||||
self.spec = spec
|
||||
self.path = str(path)
|
||||
@@ -100,16 +103,19 @@ def __init__(self, spec, path, installed, ref_count=0, explicit=False):
|
||||
self.explicit = explicit
|
||||
|
||||
def to_dict(self):
|
||||
return { 'spec' : self.spec.to_node_dict(),
|
||||
'path' : self.path,
|
||||
'installed' : self.installed,
|
||||
'ref_count' : self.ref_count,
|
||||
'explicit' : self.explicit }
|
||||
return {
|
||||
'spec': self.spec.to_node_dict(),
|
||||
'path': self.path,
|
||||
'installed': self.installed,
|
||||
'ref_count': self.ref_count,
|
||||
'explicit': self.explicit
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, spec, dictionary):
|
||||
d = dictionary
|
||||
return InstallRecord(spec, d['path'], d['installed'], d['ref_count'], d.get('explicit', False))
|
||||
return InstallRecord(spec, d['path'], d['installed'], d['ref_count'],
|
||||
d.get('explicit', False))
|
||||
|
||||
|
||||
class Database(object):
|
||||
@@ -144,7 +150,7 @@ def __init__(self, root, db_dir=None):
|
||||
|
||||
# Set up layout of database files within the db dir
|
||||
self._index_path = join_path(self._db_dir, 'index.yaml')
|
||||
self._lock_path = join_path(self._db_dir, 'lock')
|
||||
self._lock_path = join_path(self._db_dir, 'lock')
|
||||
|
||||
# Create needed directories and files
|
||||
if not os.path.exists(self._db_dir):
|
||||
@@ -157,17 +163,14 @@ def __init__(self, root, db_dir=None):
|
||||
self.lock = Lock(self._lock_path)
|
||||
self._data = {}
|
||||
|
||||
|
||||
def write_transaction(self, timeout=_db_lock_timeout):
|
||||
"""Get a write lock context manager for use in a `with` block."""
|
||||
return WriteTransaction(self, self._read, self._write, timeout)
|
||||
|
||||
|
||||
def read_transaction(self, timeout=_db_lock_timeout):
|
||||
"""Get a read lock context manager for use in a `with` block."""
|
||||
return ReadTransaction(self, self._read, None, timeout)
|
||||
|
||||
|
||||
def _write_to_yaml(self, stream):
|
||||
"""Write out the databsae to a YAML file.
|
||||
|
||||
@@ -183,9 +186,9 @@ def _write_to_yaml(self, stream):
|
||||
# different paths, it can't differentiate.
|
||||
# TODO: fix this before we support multiple install locations.
|
||||
database = {
|
||||
'database' : {
|
||||
'installs' : installs,
|
||||
'version' : str(_db_version)
|
||||
'database': {
|
||||
'installs': installs,
|
||||
'version': str(_db_version)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -194,15 +197,11 @@ def _write_to_yaml(self, stream):
|
||||
except YAMLError as e:
|
||||
raise SpackYAMLError("error writing YAML database:", str(e))
|
||||
|
||||
|
||||
def _read_spec_from_yaml(self, hash_key, installs, parent_key=None):
|
||||
"""Recursively construct a spec from a hash in a YAML database.
|
||||
|
||||
Does not do any locking.
|
||||
"""
|
||||
if hash_key not in installs:
|
||||
parent = read_spec(installs[parent_key]['path'])
|
||||
|
||||
spec_dict = installs[hash_key]['spec']
|
||||
|
||||
# Install records don't include hash with spec, so we add it in here
|
||||
@@ -224,7 +223,6 @@ def _read_spec_from_yaml(self, hash_key, installs, parent_key=None):
|
||||
spec._mark_concrete()
|
||||
return spec
|
||||
|
||||
|
||||
def _read_from_yaml(self, stream):
|
||||
"""
|
||||
Fill database from YAML, do not maintain old data
|
||||
@@ -246,15 +244,15 @@ def _read_from_yaml(self, stream):
|
||||
return
|
||||
|
||||
def check(cond, msg):
|
||||
if not cond: raise CorruptDatabaseError(self._index_path, msg)
|
||||
if not cond:
|
||||
raise CorruptDatabaseError(self._index_path, msg)
|
||||
|
||||
check('database' in yfile, "No 'database' attribute in YAML.")
|
||||
|
||||
# High-level file checks
|
||||
db = yfile['database']
|
||||
check('installs' in db, "No 'installs' in YAML DB.")
|
||||
check('version' in db, "No 'version' in YAML DB.")
|
||||
|
||||
check('version' in db, "No 'version' in YAML DB.")
|
||||
|
||||
installs = db['installs']
|
||||
|
||||
@@ -277,25 +275,25 @@ def check(cond, msg):
|
||||
# hashes are the same.
|
||||
spec_hash = spec.dag_hash()
|
||||
if not spec_hash == hash_key:
|
||||
tty.warn("Hash mismatch in database: %s -> spec with hash %s"
|
||||
% (hash_key, spec_hash))
|
||||
continue # TODO: is skipping the right thing to do?
|
||||
tty.warn(
|
||||
"Hash mismatch in database: %s -> spec with hash %s" %
|
||||
(hash_key, spec_hash))
|
||||
continue # TODO: is skipping the right thing to do?
|
||||
|
||||
# Insert the brand new spec in the database. Each
|
||||
# spec has its own copies of its dependency specs.
|
||||
# TODO: would a more immmutable spec implementation simplify this?
|
||||
# TODO: would a more immmutable spec implementation simplify
|
||||
# this?
|
||||
data[hash_key] = InstallRecord.from_dict(spec, rec)
|
||||
|
||||
except Exception as e:
|
||||
tty.warn("Invalid database reecord:",
|
||||
"file: %s" % self._index_path,
|
||||
"hash: %s" % hash_key,
|
||||
"cause: %s" % str(e))
|
||||
"hash: %s" % hash_key, "cause: %s" % str(e))
|
||||
raise
|
||||
|
||||
self._data = data
|
||||
|
||||
|
||||
def reindex(self, directory_layout):
|
||||
"""Build database index from scratch based from a directory layout.
|
||||
|
||||
@@ -320,7 +318,6 @@ def reindex(self, directory_layout):
|
||||
self._data = old_data
|
||||
raise
|
||||
|
||||
|
||||
def _check_ref_counts(self):
|
||||
"""Ensure consistency of reference counts in the DB.
|
||||
|
||||
@@ -342,9 +339,8 @@ def _check_ref_counts(self):
|
||||
found = rec.ref_count
|
||||
if not expected == found:
|
||||
raise AssertionError(
|
||||
"Invalid ref_count: %s: %d (expected %d), in DB %s"
|
||||
% (key, found, expected, self._index_path))
|
||||
|
||||
"Invalid ref_count: %s: %d (expected %d), in DB %s" %
|
||||
(key, found, expected, self._index_path))
|
||||
|
||||
def _write(self):
|
||||
"""Write the in-memory database index to its file path.
|
||||
@@ -366,7 +362,6 @@ def _write(self):
|
||||
os.remove(temp_file)
|
||||
raise
|
||||
|
||||
|
||||
def _read(self):
|
||||
"""Re-read Database from the data in the set location.
|
||||
|
||||
@@ -381,7 +376,6 @@ def _read(self):
|
||||
# reindex() takes its own write lock, so no lock here.
|
||||
self.reindex(spack.install_layout)
|
||||
|
||||
|
||||
def _add(self, spec, path, directory_layout=None, explicit=False):
|
||||
"""Add an install record for spec at path to the database.
|
||||
|
||||
@@ -404,11 +398,11 @@ def _add(self, spec, path, directory_layout=None, explicit=False):
|
||||
rec.path = path
|
||||
|
||||
else:
|
||||
self._data[key] = InstallRecord(spec, path, True, explicit=explicit)
|
||||
self._data[key] = InstallRecord(spec, path, True,
|
||||
explicit=explicit)
|
||||
for dep in spec.dependencies.values():
|
||||
self._increment_ref_count(dep, directory_layout)
|
||||
|
||||
|
||||
def _increment_ref_count(self, spec, directory_layout=None):
|
||||
"""Recursively examine dependencies and update their DB entries."""
|
||||
key = spec.dag_hash()
|
||||
@@ -438,28 +432,25 @@ def add(self, spec, path, explicit=False):
|
||||
with self.write_transaction():
|
||||
self._add(spec, path, explicit=explicit)
|
||||
|
||||
|
||||
def _get_matching_spec_key(self, spec, **kwargs):
|
||||
"""Get the exact spec OR get a single spec that matches."""
|
||||
key = spec.dag_hash()
|
||||
if not key in self._data:
|
||||
if key not in self._data:
|
||||
match = self.query_one(spec, **kwargs)
|
||||
if match:
|
||||
return match.dag_hash()
|
||||
raise KeyError("No such spec in database! %s" % spec)
|
||||
return key
|
||||
|
||||
|
||||
@_autospec
|
||||
def get_record(self, spec, **kwargs):
|
||||
key = self._get_matching_spec_key(spec, **kwargs)
|
||||
return self._data[key]
|
||||
|
||||
|
||||
def _decrement_ref_count(self, spec):
|
||||
key = spec.dag_hash()
|
||||
|
||||
if not key in self._data:
|
||||
if key not in self._data:
|
||||
# TODO: print something here? DB is corrupt, but
|
||||
# not much we can do.
|
||||
return
|
||||
@@ -472,7 +463,6 @@ def _decrement_ref_count(self, spec):
|
||||
for dep in spec.dependencies.values():
|
||||
self._decrement_ref_count(dep)
|
||||
|
||||
|
||||
def _remove(self, spec):
|
||||
"""Non-locking version of remove(); does real work.
|
||||
"""
|
||||
@@ -491,7 +481,6 @@ def _remove(self, spec):
|
||||
# query spec was passed in.
|
||||
return rec.spec
|
||||
|
||||
|
||||
@_autospec
|
||||
def remove(self, spec):
|
||||
"""Removes a spec from the database. To be called on uninstall.
|
||||
@@ -508,7 +497,6 @@ def remove(self, spec):
|
||||
with self.write_transaction():
|
||||
return self._remove(spec)
|
||||
|
||||
|
||||
@_autospec
|
||||
def installed_extensions_for(self, extendee_spec):
|
||||
"""
|
||||
@@ -519,12 +507,11 @@ def installed_extensions_for(self, extendee_spec):
|
||||
try:
|
||||
if s.package.extends(extendee_spec):
|
||||
yield s.package
|
||||
except UnknownPackageError as e:
|
||||
except UnknownPackageError:
|
||||
continue
|
||||
# skips unknown packages
|
||||
# TODO: conditional way to do this instead of catching exceptions
|
||||
|
||||
|
||||
def query(self, query_spec=any, known=any, installed=True, explicit=any):
|
||||
"""Run a query on the database.
|
||||
|
||||
@@ -567,14 +554,14 @@ def query(self, query_spec=any, known=any, installed=True, explicit=any):
|
||||
continue
|
||||
if explicit is not any and rec.explicit != explicit:
|
||||
continue
|
||||
if known is not any and spack.repo.exists(rec.spec.name) != known:
|
||||
if known is not any and spack.repo.exists(
|
||||
rec.spec.name) != known:
|
||||
continue
|
||||
if query_spec is any or rec.spec.satisfies(query_spec):
|
||||
results.append(rec.spec)
|
||||
|
||||
return sorted(results)
|
||||
|
||||
|
||||
def query_one(self, query_spec, known=any, installed=True):
|
||||
"""Query for exactly one spec that matches the query spec.
|
||||
|
||||
@@ -586,10 +573,9 @@ def query_one(self, query_spec, known=any, installed=True):
|
||||
assert len(concrete_specs) <= 1
|
||||
return concrete_specs[0] if concrete_specs else None
|
||||
|
||||
|
||||
def missing(self, spec):
|
||||
with self.read_transaction():
|
||||
key = spec.dag_hash()
|
||||
key = spec.dag_hash()
|
||||
return key in self._data and not self._data[key].installed
|
||||
|
||||
|
||||
@@ -601,7 +587,10 @@ class _Transaction(object):
|
||||
|
||||
Timeout for lock is customizable.
|
||||
"""
|
||||
def __init__(self, db, acquire_fn=None, release_fn=None,
|
||||
|
||||
def __init__(self, db,
|
||||
acquire_fn=None,
|
||||
release_fn=None,
|
||||
timeout=_db_lock_timeout):
|
||||
self._db = db
|
||||
self._timeout = timeout
|
||||
@@ -636,11 +625,11 @@ def _exit(self):
|
||||
class CorruptDatabaseError(SpackError):
|
||||
def __init__(self, path, msg=''):
|
||||
super(CorruptDatabaseError, self).__init__(
|
||||
"Spack database is corrupt: %s. %s" %(path, msg))
|
||||
"Spack database is corrupt: %s. %s" % (path, msg))
|
||||
|
||||
|
||||
class InvalidDatabaseVersionError(SpackError):
|
||||
def __init__(self, expected, found):
|
||||
super(InvalidDatabaseVersionError, self).__init__(
|
||||
"Expected database version %s but found version %s"
|
||||
% (expected, found))
|
||||
"Expected database version %s but found version %s" %
|
||||
(expected, found))
|
||||
|
@@ -1,4 +1,4 @@
|
||||
##############################################################################
|
||||
#
|
||||
# Copyright (c) 2013-2016, Lawrence Livermore National Security, LLC.
|
||||
# Produced at the Lawrence Livermore National Laboratory.
|
||||
#
|
||||
@@ -21,7 +21,7 @@
|
||||
# You should have received a copy of the GNU Lesser General Public
|
||||
# License along with this program; if not, write to the Free Software
|
||||
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
|
||||
##############################################################################
|
||||
#
|
||||
"""
|
||||
Fetch strategies are used to download source code into a staging area
|
||||
in order to build it. They need to define the following methods:
|
||||
@@ -75,11 +75,13 @@ def wrapper(self, *args, **kwargs):
|
||||
|
||||
|
||||
class FetchStrategy(object):
|
||||
|
||||
"""Superclass of all fetch strategies."""
|
||||
enabled = False # Non-abstract subclasses should be enabled.
|
||||
required_attributes = None # Attributes required in version() args.
|
||||
|
||||
class __metaclass__(type):
|
||||
|
||||
"""This metaclass registers all fetch strategies in a list."""
|
||||
|
||||
def __init__(cls, name, bases, dict):
|
||||
@@ -126,6 +128,7 @@ def matches(cls, args):
|
||||
|
||||
@pattern.composite(interface=FetchStrategy)
|
||||
class FetchStrategyComposite(object):
|
||||
|
||||
"""
|
||||
Composite for a FetchStrategy object. Implements the GoF composite pattern.
|
||||
"""
|
||||
@@ -134,6 +137,7 @@ class FetchStrategyComposite(object):
|
||||
|
||||
|
||||
class URLFetchStrategy(FetchStrategy):
|
||||
|
||||
"""FetchStrategy that pulls source code from a URL for an archive,
|
||||
checks the archive against a checksum,and decompresses the archive.
|
||||
"""
|
||||
@@ -235,12 +239,13 @@ def fetch(self):
|
||||
# redirects properly.
|
||||
content_types = re.findall(r'Content-Type:[^\r\n]+', headers)
|
||||
if content_types and 'text/html' in content_types[-1]:
|
||||
tty.warn(
|
||||
"The contents of " + self.archive_file + " look like HTML.",
|
||||
"The checksum will likely be bad. If it is, you can use",
|
||||
"'spack clean <package>' to remove the bad archive, then fix",
|
||||
"your internet gateway issue and install again.")
|
||||
|
||||
tty.warn("The contents of ",
|
||||
(self.archive_file if self.archive_file is not None
|
||||
else "the archive"),
|
||||
" look like HTML.",
|
||||
"The checksum will likely be bad. If it is, you can use",
|
||||
"'spack clean <package>' to remove the bad archive, then",
|
||||
"fix your internet gateway issue and install again.")
|
||||
if save_file:
|
||||
os.rename(partial_file, save_file)
|
||||
|
||||
@@ -353,6 +358,7 @@ def __str__(self):
|
||||
|
||||
|
||||
class VCSFetchStrategy(FetchStrategy):
|
||||
|
||||
def __init__(self, name, *rev_types, **kwargs):
|
||||
super(VCSFetchStrategy, self).__init__()
|
||||
self.name = name
|
||||
@@ -407,6 +413,7 @@ def __repr__(self):
|
||||
|
||||
|
||||
class GoFetchStrategy(VCSFetchStrategy):
|
||||
|
||||
"""
|
||||
Fetch strategy that employs the `go get` infrastructure
|
||||
Use like this in a package:
|
||||
@@ -466,6 +473,7 @@ def __str__(self):
|
||||
|
||||
|
||||
class GitFetchStrategy(VCSFetchStrategy):
|
||||
|
||||
"""
|
||||
Fetch strategy that gets source code from a git repository.
|
||||
Use like this in a package:
|
||||
@@ -586,6 +594,7 @@ def __str__(self):
|
||||
|
||||
|
||||
class SvnFetchStrategy(VCSFetchStrategy):
|
||||
|
||||
"""Fetch strategy that gets source code from a subversion repository.
|
||||
Use like this in a package:
|
||||
|
||||
@@ -662,6 +671,7 @@ def __str__(self):
|
||||
|
||||
|
||||
class HgFetchStrategy(VCSFetchStrategy):
|
||||
|
||||
"""
|
||||
Fetch strategy that gets source code from a Mercurial repository.
|
||||
Use like this in a package:
|
||||
@@ -806,11 +816,13 @@ def for_package_version(pkg, version):
|
||||
|
||||
|
||||
class FetchError(spack.error.SpackError):
|
||||
|
||||
def __init__(self, msg, long_msg=None):
|
||||
super(FetchError, self).__init__(msg, long_msg)
|
||||
|
||||
|
||||
class FailedDownloadError(FetchError):
|
||||
|
||||
"""Raised wen a download fails."""
|
||||
|
||||
def __init__(self, url, msg=""):
|
||||
@@ -820,16 +832,19 @@ def __init__(self, url, msg=""):
|
||||
|
||||
|
||||
class NoArchiveFileError(FetchError):
|
||||
|
||||
def __init__(self, msg, long_msg):
|
||||
super(NoArchiveFileError, self).__init__(msg, long_msg)
|
||||
|
||||
|
||||
class NoDigestError(FetchError):
|
||||
|
||||
def __init__(self, msg, long_msg=None):
|
||||
super(NoDigestError, self).__init__(msg, long_msg)
|
||||
|
||||
|
||||
class InvalidArgsError(FetchError):
|
||||
|
||||
def __init__(self, pkg, version):
|
||||
msg = ("Could not construct a fetch strategy for package %s at "
|
||||
"version %s")
|
||||
@@ -838,6 +853,7 @@ def __init__(self, pkg, version):
|
||||
|
||||
|
||||
class ChecksumError(FetchError):
|
||||
|
||||
"""Raised when archive fails to checksum."""
|
||||
|
||||
def __init__(self, message, long_msg=None):
|
||||
@@ -845,6 +861,7 @@ def __init__(self, message, long_msg=None):
|
||||
|
||||
|
||||
class NoStageError(FetchError):
|
||||
|
||||
"""Raised when fetch operations are called before set_stage()."""
|
||||
|
||||
def __init__(self, method):
|
||||
|
File diff suppressed because it is too large
Load Diff
@@ -39,7 +39,7 @@
|
||||
'svn_fetch', 'hg_fetch', 'mirror', 'modules', 'url_extrapolate',
|
||||
'cc', 'link_tree', 'spec_yaml', 'optional_deps',
|
||||
'make_executable', 'configure_guess', 'lock', 'database',
|
||||
'namespace_trie', 'yaml', 'sbang', 'environment',
|
||||
'namespace_trie', 'yaml', 'sbang', 'environment', 'cmd.find',
|
||||
'cmd.uninstall', 'cmd.test_install']
|
||||
|
||||
|
||||
|
60
lib/spack/spack/test/cmd/find.py
Normal file
60
lib/spack/spack/test/cmd/find.py
Normal file
@@ -0,0 +1,60 @@
|
||||
##############################################################################
|
||||
# Copyright (c) 2013-2016, Lawrence Livermore National Security, LLC.
|
||||
# Produced at the Lawrence Livermore National Laboratory.
|
||||
#
|
||||
# This file is part of Spack.
|
||||
# Created by Todd Gamblin, tgamblin@llnl.gov, All rights reserved.
|
||||
# LLNL-CODE-647188
|
||||
#
|
||||
# For details, see https://github.com/llnl/spack
|
||||
# Please also see the LICENSE file for our notice and the LGPL.
|
||||
#
|
||||
# This program is free software; you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Lesser General Public License (as
|
||||
# published by the Free Software Foundation) version 2.1, February 1999.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful, but
|
||||
# WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the terms and
|
||||
# conditions of the GNU Lesser General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU Lesser General Public
|
||||
# License along with this program; if not, write to the Free Software
|
||||
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
|
||||
##############################################################################
|
||||
|
||||
|
||||
import spack.cmd.find
|
||||
import unittest
|
||||
|
||||
|
||||
class Bunch(object):
|
||||
|
||||
def __init__(self, **kwargs):
|
||||
self.__dict__.update(kwargs)
|
||||
|
||||
|
||||
class FindTest(unittest.TestCase):
|
||||
|
||||
def test_query_arguments(self):
|
||||
query_arguments = spack.cmd.find.query_arguments
|
||||
# Default arguments
|
||||
args = Bunch(only_missing=False, missing=False,
|
||||
unknown=False, explicit=False, implicit=False)
|
||||
q_args = query_arguments(args)
|
||||
self.assertTrue('installed' in q_args)
|
||||
self.assertTrue('known' in q_args)
|
||||
self.assertTrue('explicit' in q_args)
|
||||
self.assertEqual(q_args['installed'], True)
|
||||
self.assertEqual(q_args['known'], any)
|
||||
self.assertEqual(q_args['explicit'], any)
|
||||
# Check that explicit works correctly
|
||||
args.explicit = True
|
||||
q_args = query_arguments(args)
|
||||
self.assertEqual(q_args['explicit'], True)
|
||||
args.explicit = False
|
||||
args.implicit = True
|
||||
q_args = query_arguments(args)
|
||||
self.assertEqual(q_args['explicit'], False)
|
||||
args.explicit = True
|
||||
self.assertRaises(SystemExit, query_arguments, args)
|
Reference in New Issue
Block a user