zhangyiss/spack
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update docs for YAML configuration files and preferred concretization

Browse Source
This commit is contained in:
Matthew LeGendre
2015-07-07 16:27:13 -07:00
parent ee68a76a19
commit 987cd9e78f
4 changed files with 91 additions and 104 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
lib/spack/docs/basic_usage.rst
View File

@@ -357,7 +357,7 @@ Spack, you can simply run ``spack compiler add`` with the path to
where the compiler is installed. For example::
$ spack compiler add /usr/local/tools/ic-13.0.079
==> Added 1 new compiler to /Users/gamblin2/.spackconfig
==> Added 1 new compiler to /Users/gamblin2/.spack/compilers.yaml
intel@13.0.079
Or you can run ``spack compiler add`` with no arguments to force
@@ -367,7 +367,7 @@ installed, but you know that new compilers have been added to your
$ module load gcc-4.9.0
$ spack compiler add
==> Added 1 new compiler to /Users/gamblin2/.spackconfig
==> Added 1 new compiler to /Users/gamblin2/.spack/compilers.yaml
gcc@4.9.0
This loads the environment module for gcc-4.9.0 to get it into the
@@ -398,27 +398,34 @@ Manual compiler configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If auto-detection fails, you can manually configure a compiler by
editing your ``~/.spackconfig`` file. You can do this by running
``spack config edit``, which will open the file in your ``$EDITOR``.
editing your ``~/.spack/compilers.yaml`` file. You can do this by running
``spack config edit compilers``, which will open the file in your ``$EDITOR``.
Each compiler configuration in the file looks like this::
...
[compiler "intel@15.0.0"]
cc = /usr/local/bin/icc-15.0.024-beta
cxx = /usr/local/bin/icpc-15.0.024-beta
f77 = /usr/local/bin/ifort-15.0.024-beta
fc = /usr/local/bin/ifort-15.0.024-beta
...
chaos_5_x86_64_ib:
...
intel@15.0.0:
cc: /usr/local/bin/icc-15.0.024-beta
cxx: /usr/local/bin/icpc-15.0.024-beta
f77: /usr/local/bin/ifort-15.0.024-beta
fc: /usr/local/bin/ifort-15.0.024-beta
...
The chaos_5_x86_64_ib string is an architecture string, and multiple
compilers can be listed underneath an architecture. The architecture
string may be replaced with the string 'all' to signify compilers that
work on all architectures.
For compilers, like ``clang``, that do not support Fortran, put
``None`` for ``f77`` and ``fc``::
[compiler "clang@3.3svn"]
cc = /usr/bin/clang
cxx = /usr/bin/clang++
f77 = None
fc = None
clang@3.3svn:
cc: /usr/bin/clang
cxx: /usr/bin/clang++
f77: None
fc: None
Once you save the file, the configured compilers will show up in the
list displayed by ``spack compilers``.

9
lib/spack/docs/mirrors.rst
View File

@@ -205,12 +205,11 @@ And, if you want to remove a mirror, just remove it by name::
Mirror precedence
----------------------------
Adding a mirror really just adds a section in ``~/.spackconfig``::
Adding a mirror really just adds a section in ``~/.spack/mirrors.yaml``::
[mirror "local_filesystem"]
url = file:///Users/gamblin2/spack-mirror-2014-06-24
[mirror "remote_server"]
url = https://example.com/some/web-hosted/directory/spack-mirror-2014-06-24
mirrors:
- local_filesystem: file:///Users/gamblin2/spack-mirror-2014-06-24
- remote_server: https://example.com/some/web-hosted/directory/spack-mirror-2014-06-24
If you want to change the order in which mirrors are searched for
packages, you can edit this file and reorder the sections. Spack will

66
lib/spack/docs/packaging_guide.rst
View File

@@ -632,7 +632,7 @@ Default
revision instead.
Revisions
Add ``hg`` and ``revision``parameters:
Add ``hg`` and ``revision`` parameters:
.. code-block:: python
@@ -1524,6 +1524,70 @@ This is useful when you want to know exactly what Spack will do when
you ask for a particular spec.
``Concretization Policies``
~~~~~~~~~~~~~~~~~~~~~~~~~~~
A user may have certain perferrences for how packages should
be concretized on their system. For example, one user may prefer packages
built with OpenMPI and the Intel compiler. Another user may prefer
packages be built with MVAPICH and GCC.
Spack's ``preferred`` configuration can be used to set defaults for sites or users.
Spack uses this configuration to make decisions about which compilers, package
versions, depends_on, and variants it should prefer during concretization.
The preferred configuration can be controlled by editing the
``~/.spack/preferred.yaml`` file for user configuations, or the
Here's an example preferred.yaml file:
.. code-block:: sh
preferred:
dyninst:
compiler: gcc@4.9
variants: +debug
gperftools:
version: 2.2, 2.4, 2.3
all:
compiler: gcc@4.4.7, gcc@4.6:, intel, clang, pgi
providers:
mpi: mvapich, mpich, openmpi
At a high level, this example is specifying how packages should be
concretized. The dyninst package should prefer using gcc 4.9 and
be built with debug options. The gperftools package should prefer version
2.2 over 2.4. Every package on the system should prefer mvapich for
its MPI and gcc 4.4.7 (except for Dyninst, which perfers gcc 4.9).
These options are used to fill in implicit defaults. Any of them can be overwritten
on the command line if explicitly requested.
Each preferred.yaml file begin with the string ``preferred:`` and
each subsequent entry is indented underneath it. The next layer contains
package names or the special string ``all`` (which applies to
every package). Underneath each package name is
one or more components: ``compiler``, ``variants``, ``version``,
or ``providers``. Each component has an ordered list of spec
``constraints``, with earlier entries in the list being prefered over
latter entries.
Sometimes a package installation may have constraints that forbid
the first concretization rule, in which case Spack will use the first
legal concretization rule. Going back to the example, if a user
requests gperftools 2.3 or latter, then Spack will install version 2.4
as the 2.4 version of gperftools is preferred over 2.3.
An explicit concretization rule in the preferred section will always
take preference over unlisted concretizations. In the above example,
xlc isn't listed in the compiler list. Every listed compiler from
gcc to pgi will thus be preferred over the xlc compiler.
The syntax for the ``providers`` section differs slightly from other
concretization rules. A provider lists a value that packages may
``depend_on`` (e.g, mpi) and a list of rules for fulfilling that
dependency.
.. _install-method:
Implementing the ``install`` method

83
lib/spack/docs/site_configuration.rst
View File

@@ -54,89 +54,6 @@ more elements to the list to indicate where your own site's temporary
directory is.
.. _concretization-policies:
Concretization policies
----------------------------
When a user asks for a package like ``mpileaks`` to be installed,
Spack has to make decisions like what version should be installed,
what compiler to use, and how its dependencies should be configured.
This process is called *concretization*, and it's covered in detail in
:ref:`its own section <abstract-and-concrete>`.
The default concretization policies are in the
:py:mod:`spack.concretize` module, specifically in the
:py:class:`spack.concretize.DefaultConcretizer` class. These are the
important methods used in the concretization process:
* :py:meth:`concretize_version(self, spec) <spack.concretize.DefaultConcretizer.concretize_version>`
* :py:meth:`concretize_architecture(self, spec) <spack.concretize.DefaultConcretizer.concretize_architecture>`
* :py:meth:`concretize_compiler(self, spec) <spack.concretize.DefaultConcretizer.concretize_compiler>`
* :py:meth:`choose_provider(self, spec, providers) <spack.concretize.DefaultConcretizer.choose_provider>`
The first three take a :py:class:`Spec <spack.spec.Spec>` object and
modify it by adding constraints for the version. For example, if the
input spec had a version range like `1.0:5.0.3`, then the
``concretize_version`` method should set the spec's version to a
*single* version in that range. Likewise, ``concretize_architecture``
selects an architecture when the input spec does not have one, and
``concretize_compiler`` needs to set both a concrete compiler and a
concrete compiler version.
``choose_provider()`` affects how concrete implementations are chosen
based on a virtual dependency spec. The input spec is some virtual
dependency and the ``providers`` index is a :py:class:`ProviderIndex
<spack.packages.ProviderIndex>` object. The ``ProviderIndex`` maps
the virtual spec to specs for possible implementations, and
``choose_provider()`` should simply choose one of these. The
``concretize_*`` methods will be called on the chosen implementation
later, so there is no need to fully concretize the spec when returning
it.
The ``DefaultConcretizer`` is intended to provide sensible defaults
for each policy, but there are certain choices that it can't know
about. For example, one site might prefer ``OpenMPI`` over ``MPICH``,
or another might prefer an old version of some packages. These types
of special cases can be integrated with custom concretizers.
Writing a custom concretizer
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To write your own concretizer, you need only subclass
``DefaultConcretizer`` and override the methods you want to change.
For example, you might write a class like this to change *only* the
``concretize_version()`` behavior:
.. code-block:: python
from spack.concretize import DefaultConcretizer
class MyConcretizer(DefaultConcretizer):
def concretize_version(self, spec):
# implement custom logic here.
Once you have written your custom concretizer, you can make Spack use
it by editing ``globals.py``. Find this part of the file:
.. code-block:: python
#
# This controls how things are concretized in spack.
# Replace it with a subclass if you want different
# policies.
#
concretizer = DefaultConcretizer()
Set concretizer to *your own* class instead of the default:
.. code-block:: python
concretizer = MyConcretizer()
The next time you run Spack, your changes should take effect.
Profiling
~~~~~~~~~~~~~~~~~~~~~