zhangyiss/spack
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #1125 from NERSC/docs/cray

Browse Source 
Update documentation for use on Cray
This commit is contained in:
Todd Gamblin 2016-07-06 15:06:20 -07:00 committed by GitHub
commit b9148b1751
3 changed files with 237 additions and 94 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

268
lib/spack/docs/basic_usage.rst
View File

@ -114,13 +114,13 @@ that the packages is installed:
$ spack install mpileaks $ spack install mpileaks
==> Installing mpileaks ==> Installing mpileaks
==> mpich is already installed in /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/mpich@3.0.4. ==> mpich is already installed in /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/mpich@3.0.4.
==> callpath is already installed in /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/callpath@1.0.2-5dce4318. ==> callpath is already installed in /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/callpath@1.0.2-5dce4318.
==> adept-utils is already installed in /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/adept-utils@1.0-5adef8da. ==> adept-utils is already installed in /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/adept-utils@1.0-5adef8da.
==> Trying to fetch from https://github.com/hpc/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz ==> Trying to fetch from https://github.com/hpc/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz
######################################################################## 100.0% ######################################################################## 100.0%
==> Staging archive: /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=chaos_5_x86_64_ib-59f6ad23/mpileaks-1.0.tar.gz ==> Staging archive: /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=linux-debian7-x86_64-59f6ad23/mpileaks-1.0.tar.gz
==> Created stage in /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=chaos_5_x86_64_ib-59f6ad23. ==> Created stage in /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=linux-debian7-x86_64-59f6ad23.
==> No patches needed for mpileaks. ==> No patches needed for mpileaks.
==> Building mpileaks. ==> Building mpileaks.
@ -128,7 +128,7 @@ that the packages is installed:
==> Successfully installed mpileaks. ==> Successfully installed mpileaks.
Fetch: 2.16s. Build: 9.82s. Total: 11.98s. Fetch: 2.16s. Build: 9.82s. Total: 11.98s.
[+] /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/mpileaks@1.0-59f6ad23 [+] /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/mpileaks@1.0-59f6ad23
The last line, with the ``[+]``, indicates where the package is The last line, with the ``[+]``, indicates where the package is
installed. installed.
@ -230,7 +230,7 @@ Running ``spack find`` with no arguments lists installed packages:
$ spack find $ spack find
==> 74 installed packages. ==> 74 installed packages.
-- chaos_5_x86_64_ib / gcc@4.4.7 -------------------------------- -- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
ImageMagick@6.8.9-10 libdwarf@20130729 py-dateutil@2.4.0 ImageMagick@6.8.9-10 libdwarf@20130729 py-dateutil@2.4.0
adept-utils@1.0 libdwarf@20130729 py-ipython@2.3.1 adept-utils@1.0 libdwarf@20130729 py-ipython@2.3.1
atk@2.14.0 libelf@0.8.12 py-matplotlib@1.4.2 atk@2.14.0 libelf@0.8.12 py-matplotlib@1.4.2
@ -256,7 +256,7 @@ Running ``spack find`` with no arguments lists installed packages:
lcms@2.6 pixman@0.32.6 xz@5.2.0 lcms@2.6 pixman@0.32.6 xz@5.2.0
libdrm@2.4.33 py-dateutil@2.4.0 zlib@1.2.8 libdrm@2.4.33 py-dateutil@2.4.0 zlib@1.2.8
-- chaos_5_x86_64_ib / gcc@4.9.2 -------------------------------- -- linux-debian7-x86_64 / gcc@4.9.2 --------------------------------
libelf@0.8.10 mpich@3.0.4 libelf@0.8.10 mpich@3.0.4
Packages are divided into groups according to their architecture and Packages are divided into groups according to their architecture and
@ -279,7 +279,7 @@ in more detail using ``spack find -d``, and by asking only to show
$ spack find --deps libdwarf $ spack find --deps libdwarf
==> 2 installed packages. ==> 2 installed packages.
-- chaos_5_x86_64_ib / gcc@4.4.7 -------------------------------- -- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
libdwarf@20130729-d9b90962 libdwarf@20130729-d9b90962
^libelf@0.8.12 ^libelf@0.8.12
libdwarf@20130729-b52fac98 libdwarf@20130729-b52fac98
@ -295,7 +295,7 @@ want to know whether two packages' dependencies differ, you can use
$ spack find -l libdwarf $ spack find -l libdwarf
==> 2 installed packages. ==> 2 installed packages.
-- chaos_5_x86_64_ib / gcc@4.4.7 -------------------------------- -- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
libdwarf@20130729-d9b90962 libdwarf@20130729-b52fac98 libdwarf@20130729-d9b90962 libdwarf@20130729-b52fac98
Now the ``libwarf`` installs have hashes after their names. These are Now the ``libwarf`` installs have hashes after their names. These are
@ -309,14 +309,14 @@ use ``spack find -p``:
$ spack find -p $ spack find -p
==> 74 installed packages. ==> 74 installed packages.
-- chaos_5_x86_64_ib / gcc@4.4.7 -------------------------------- -- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
ImageMagick@6.8.9-10 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/ImageMagick@6.8.9-10-4df950dd ImageMagick@6.8.9-10 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/ImageMagick@6.8.9-10-4df950dd
adept-utils@1.0 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/adept-utils@1.0-5adef8da adept-utils@1.0 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/adept-utils@1.0-5adef8da
atk@2.14.0 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/atk@2.14.0-3d09ac09 atk@2.14.0 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/atk@2.14.0-3d09ac09
boost@1.55.0 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/boost@1.55.0 boost@1.55.0 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/boost@1.55.0
bzip2@1.0.6 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/bzip2@1.0.6 bzip2@1.0.6 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/bzip2@1.0.6
cairo@1.14.0 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/cairo@1.14.0-fcc2ab44 cairo@1.14.0 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/cairo@1.14.0-fcc2ab44
callpath@1.0.2 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/callpath@1.0.2-5dce4318 callpath@1.0.2 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/callpath@1.0.2-5dce4318
... ...
And, finally, you can restrict your search to a particular package And, finally, you can restrict your search to a particular package
@ -325,10 +325,10 @@ by supplying its name:
.. code-block:: sh .. code-block:: sh
$ spack find -p libelf $ spack find -p libelf
-- chaos_5_x86_64_ib / gcc@4.4.7 -------------------------------- -- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
libelf@0.8.11 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/libelf@0.8.11 libelf@0.8.11 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/libelf@0.8.11
libelf@0.8.12 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/libelf@0.8.12 libelf@0.8.12 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/libelf@0.8.12
libelf@0.8.13 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/libelf@0.8.13 libelf@0.8.13 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/libelf@0.8.13
``spack find`` actually does a lot more than this. You can use ``spack find`` actually does a lot more than this. You can use
*specs* to query for specific configurations and builds of each *specs* to query for specific configurations and builds of each
@ -338,7 +338,7 @@ package. If you want to find only libelf versions greater than version
.. code-block:: sh .. code-block:: sh
$ spack find libelf@0.8.12: $ spack find libelf@0.8.12:
-- chaos_5_x86_64_ib / gcc@4.4.7 -------------------------------- -- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
libelf@0.8.12 libelf@0.8.13 libelf@0.8.12 libelf@0.8.13
Finding just the versions of libdwarf built with a particular version Finding just the versions of libdwarf built with a particular version
@ -348,7 +348,7 @@ of libelf would look like this:
$ spack find -l libdwarf ^libelf@0.8.12 $ spack find -l libdwarf ^libelf@0.8.12
==> 1 installed packages. ==> 1 installed packages.
-- chaos_5_x86_64_ib / gcc@4.4.7 -------------------------------- -- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
libdwarf@20130729-d9b90962 libdwarf@20130729-d9b90962
We can also search for packages that have a certain attribute. For example, We can also search for packages that have a certain attribute. For example,
@ -359,6 +359,7 @@ will find every installed package with a 'debug' compile-time option enabled.
The full spec syntax is discussed in detail in :ref:`sec-specs`. The full spec syntax is discussed in detail in :ref:`sec-specs`.
.. _compiler-config:
Compiler configuration Compiler configuration
----------------------------------- -----------------------------------
@ -445,7 +446,7 @@ If you want to see specifics on a particular compiler, you can run
fc = /usr/local/bin/ifort-15.0.090 fc = /usr/local/bin/ifort-15.0.090
This shows which C, C++, and Fortran compilers were detected by Spack. This shows which C, C++, and Fortran compilers were detected by Spack.
Notice also that we didn't have to be too specific about the Notice also that we didn\'t have to be too specific about the
version. We just said ``intel@15``, and information about the only version. We just said ``intel@15``, and information about the only
matching Intel compiler was displayed. matching Intel compiler was displayed.
@ -460,19 +461,17 @@ editing your ``~/.spack/compilers.yaml`` file. You can do this by running
Each compiler configuration in the file looks like this:: Each compiler configuration in the file looks like this::
... ...
chaos_5_x86_64_ib: compilers:
... - compiler:
intel@15.0.0: modules = []
operating_system: OS
paths:
cc: /usr/local/bin/icc-15.0.024-beta cc: /usr/local/bin/icc-15.0.024-beta
cxx: /usr/local/bin/icpc-15.0.024-beta cxx: /usr/local/bin/icpc-15.0.024-beta
f77: /usr/local/bin/ifort-15.0.024-beta f77: /usr/local/bin/ifort-15.0.024-beta
fc: /usr/local/bin/ifort-15.0.024-beta fc: /usr/local/bin/ifort-15.0.024-beta
...
The chaos_5_x86_64_ib string is an architecture string, and multiple spec: intel@15.0.0:
compilers can be listed underneath an architecture. The architecture
string may be replaced with the string 'all' to signify compilers that
work on all architectures.
For compilers, like ``clang``, that do not support Fortran, put For compilers, like ``clang``, that do not support Fortran, put
``None`` for ``f77`` and ``fc``:: ``None`` for ``f77`` and ``fc``::
@ -488,10 +487,11 @@ list displayed by ``spack compilers``.
You can also add compiler flags to manually configured compilers. The You can also add compiler flags to manually configured compilers. The
valid flags are ``cflags``, ``cxxflags``, ``fflags``, ``cppflags``, valid flags are ``cflags``, ``cxxflags``, ``fflags``, ``cppflags``,
``ldflags``, and ``ldlibs``. For example,:: ``ldflags``, and ``ldlibs``. For example::
... ...
chaos_5_x86_64_ib: compilers:
- compiler:
... ...
intel@15.0.0: intel@15.0.0:
cc: /usr/local/bin/icc-15.0.024-beta cc: /usr/local/bin/icc-15.0.024-beta
@ -518,10 +518,10 @@ Spack, that descriptor is called a *spec*. Spack uses specs to refer
to a particular build configuration (or configurations) of a package. to a particular build configuration (or configurations) of a package.
Specs are more than a package name and a version; you can use them to Specs are more than a package name and a version; you can use them to
specify the compiler, compiler version, architecture, compile options, specify the compiler, compiler version, architecture, compile options,
and dependency options for a build. In this section, we'll go over and dependency options for a build. In this section, we\'ll go over
the full syntax of specs. the full syntax of specs.
Here is an example of a much longer spec than we've seen thus far:: Here is an example of a much longer spec than we\'ve seen thus far::
mpileaks @1.2:1.4 %gcc@4.7.5 +debug -qt arch=bgq_os ^callpath @1.1 %gcc@4.7.2 mpileaks @1.2:1.4 %gcc@4.7.5 +debug -qt arch=bgq_os ^callpath @1.1 %gcc@4.7.2
@ -546,8 +546,8 @@ More formally, a spec consists of the following pieces:
boolean variants boolean variants
* ``name=<value>`` Optional compiler flag specifiers. Valid flag names are * ``name=<value>`` Optional compiler flag specifiers. Valid flag names are
``cflags``, ``cxxflags``, ``fflags``, ``cppflags``, ``ldflags``, and ``ldlibs``. ``cflags``, ``cxxflags``, ``fflags``, ``cppflags``, ``ldflags``, and ``ldlibs``.
* ``arch=<value>`` Optional architecture specifier (``arch=bgq_os``) * ``target=<value> os=<value>`` Optional architecture specifier
* ``^`` Dependency specs (``^callpath@1.1``) (``target=haswell os=CNL10``) * ``^`` Dependency specs (``^callpath@1.1``)
There are two things to notice here. The first is that specs are There are two things to notice here. The first is that specs are
recursively defined. That is, each dependency after ``^`` is a spec recursively defined. That is, each dependency after ``^`` is a spec
@ -626,7 +626,7 @@ compilers, variants, and architectures just like any other spec.
Specifiers are associated with the nearest package name to their left. Specifiers are associated with the nearest package name to their left.
For example, above, ``@1.1`` and ``%gcc@4.7.2`` associates with the For example, above, ``@1.1`` and ``%gcc@4.7.2`` associates with the
``callpath`` package, while ``@1.2:1.4``, ``%gcc@4.7.5``, ``+debug``, ``callpath`` package, while ``@1.2:1.4``, ``%gcc@4.7.5``, ``+debug``,
``-qt``, and ``arch=bgq_os`` all associate with the ``mpileaks`` package. ``-qt``, and ``target=haswell os=CNL10`` all associate with the ``mpileaks`` package.
In the diagram above, ``mpileaks`` depends on ``mpich`` with an In the diagram above, ``mpileaks`` depends on ``mpich`` with an
unspecified version, but packages can depend on other packages with unspecified version, but packages can depend on other packages with
@ -758,14 +758,18 @@ in gnu autotools. If all flags are set, the order is
Architecture specifiers Architecture specifiers
~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~
.. Note:: The architecture can be specified by using the reserved
words ``target`` and/or ``os`` (``target=x86-64 os=debian7``). You can also
use the triplet form of platform, operating system and processor.
Architecture specifiers are part of specs but are not yet .. code-block:: sh
functional. They will be in Spack version 1.0, due in Q3 2015.
spack install libelf arch=cray_xc-CNL10-haswell
The architecture specifier looks identical to a variant specifier for a Users on non-Cray systems won't have to worry about specifying the architecture.
non-boolean variant. The architecture can be specified only using the Spack will autodetect what kind of operating system is on your machine as well
reserved name ``arch`` (``arch=bgq_os``). as the processor. For more information on how the architecture can be
used on Cray machines, check here :ref:`spack-cray`
.. _sec-virtual-dependencies: .. _sec-virtual-dependencies:
@ -985,7 +989,7 @@ of installed packages.
$ module avail $ module avail
------- /home/gamblin2/spack/share/spack/modules/chaos_5_x86_64_ib -------- ------- /home/gamblin2/spack/share/spack/modules/linux-debian7-x86_64 --------
adept-utils@1.0%gcc@4.4.7-5adef8da libelf@0.8.13%gcc@4.4.7 adept-utils@1.0%gcc@4.4.7-5adef8da libelf@0.8.13%gcc@4.4.7
automaded@1.0%gcc@4.4.7-d9691bb0 libelf@0.8.13%intel@15.0.0 automaded@1.0%gcc@4.4.7-d9691bb0 libelf@0.8.13%intel@15.0.0
boost@1.55.0%gcc@4.4.7 mpc@1.0.2%gcc@4.4.7-559607f5 boost@1.55.0%gcc@4.4.7 mpc@1.0.2%gcc@4.4.7-559607f5
@ -1056,7 +1060,7 @@ Spack. For example, this will add the ``mpich`` package built with
$ spack use mpich %gcc@4.4.7 $ spack use mpich %gcc@4.4.7
Prepending: mpich@3.0.4%gcc@4.4.7 (ok) Prepending: mpich@3.0.4%gcc@4.4.7 (ok)
$ which mpicc $ which mpicc
~/src/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/mpich@3.0.4/bin/mpicc ~/src/spack/opt/linux-debian7-x86_64/gcc@4.4.7/mpich@3.0.4/bin/mpicc
Or, similarly with modules, you could type: Or, similarly with modules, you could type:
@ -1089,8 +1093,8 @@ than one installed package matches it), then Spack will warn you:
$ spack load libelf $ spack load libelf
==> Error: Multiple matches for spec libelf. Choose one: ==> Error: Multiple matches for spec libelf. Choose one:
libelf@0.8.13%gcc@4.4.7 arch=chaos_5_x86_64_ib libelf@0.8.13%gcc@4.4.7 arch=linux-debian7-x86_64
libelf@0.8.13%intel@15.0.0 arch=chaos_5_x86_64_ib libelf@0.8.13%intel@15.0.0 arch=linux-debian7-x86_64
You can either type the ``spack load`` command again with a fully You can either type the ``spack load`` command again with a fully
qualified argument, or you can add just enough extra constraints to qualified argument, or you can add just enough extra constraints to
@ -1543,7 +1547,7 @@ an *extension*. Suppose you have Python installed like so:
$ spack find python $ spack find python
==> 1 installed packages. ==> 1 installed packages.
-- chaos_5_x86_64_ib / gcc@4.4.7 -------------------------------- -- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
python@2.7.8 python@2.7.8
.. _spack-extensions: .. _spack-extensions:
@ -1556,7 +1560,7 @@ You can find extensions for your Python installation like this:
.. code-block:: sh .. code-block:: sh
$ spack extensions python $ spack extensions python
==> python@2.7.8%gcc@4.4.7 arch=chaos_5_x86_64_ib-703c7a96 ==> python@2.7.8%gcc@4.4.7 arch=linux-debian7-x86_64-703c7a96
==> 36 extensions: ==> 36 extensions:
geos py-ipython py-pexpect py-pyside py-sip geos py-ipython py-pexpect py-pyside py-sip
py-basemap py-libxml2 py-pil py-pytz py-six py-basemap py-libxml2 py-pil py-pytz py-six
@ -1568,7 +1572,7 @@ You can find extensions for your Python installation like this:
py-h5py py-numpy py-pyqt py-shiboken py-h5py py-numpy py-pyqt py-shiboken
==> 12 installed: ==> 12 installed:
-- chaos_5_x86_64_ib / gcc@4.4.7 -------------------------------- -- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
py-dateutil@2.4.0 py-nose@1.3.4 py-pyside@1.2.2 py-dateutil@2.4.0 py-nose@1.3.4 py-pyside@1.2.2
py-dateutil@2.4.0 py-numpy@1.9.1 py-pytz@2014.10 py-dateutil@2.4.0 py-numpy@1.9.1 py-pytz@2014.10
py-ipython@2.3.1 py-pygments@2.0.1 py-setuptools@11.3.1 py-ipython@2.3.1 py-pygments@2.0.1 py-setuptools@11.3.1
@ -1584,8 +1588,8 @@ prefixes, and you can see this with ``spack find -p``:
$ spack find -p py-numpy $ spack find -p py-numpy
==> 1 installed packages. ==> 1 installed packages.
-- chaos_5_x86_64_ib / gcc@4.4.7 -------------------------------- -- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
py-numpy@1.9.1 /g/g21/gamblin2/src/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/py-numpy@1.9.1-66733244 py-numpy@1.9.1 /g/g21/gamblin2/src/spack/opt/linux-debian7-x86_64/gcc@4.4.7/py-numpy@1.9.1-66733244
However, even though this package is installed, you cannot use it However, even though this package is installed, you cannot use it
directly when you run ``python``: directly when you run ``python``:
@ -1646,9 +1650,9 @@ installation:
.. code-block:: sh .. code-block:: sh
$ spack activate py-numpy $ spack activate py-numpy
==> Activated extension py-setuptools@11.3.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-3c74eb69 for python@2.7.8%gcc@4.4.7. ==> Activated extension py-setuptools@11.3.1%gcc@4.4.7 arch=linux-debian7-x86_64-3c74eb69 for python@2.7.8%gcc@4.4.7.
==> Activated extension py-nose@1.3.4%gcc@4.4.7 arch=chaos_5_x86_64_ib-5f70f816 for python@2.7.8%gcc@4.4.7. ==> Activated extension py-nose@1.3.4%gcc@4.4.7 arch=linux-debian7-x86_64-5f70f816 for python@2.7.8%gcc@4.4.7.
==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7. ==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=linux-debian7-x86_64-66733244 for python@2.7.8%gcc@4.4.7.
Several things have happened here. The user requested that Several things have happened here. The user requested that
``py-numpy`` be activated in the ``python`` installation it was built ``py-numpy`` be activated in the ``python`` installation it was built
@ -1663,7 +1667,7 @@ packages listed as activated:
.. code-block:: sh .. code-block:: sh
$ spack extensions python $ spack extensions python
==> python@2.7.8%gcc@4.4.7 arch=chaos_5_x86_64_ib-703c7a96 ==> python@2.7.8%gcc@4.4.7 arch=linux-debian7-x86_64-703c7a96
==> 36 extensions: ==> 36 extensions:
geos py-ipython py-pexpect py-pyside py-sip geos py-ipython py-pexpect py-pyside py-sip
py-basemap py-libxml2 py-pil py-pytz py-six py-basemap py-libxml2 py-pil py-pytz py-six
@ -1675,14 +1679,14 @@ packages listed as activated:
py-h5py py-numpy py-pyqt py-shiboken py-h5py py-numpy py-pyqt py-shiboken
==> 12 installed: ==> 12 installed:
-- chaos_5_x86_64_ib / gcc@4.4.7 -------------------------------- -- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
py-dateutil@2.4.0 py-nose@1.3.4 py-pyside@1.2.2 py-dateutil@2.4.0 py-nose@1.3.4 py-pyside@1.2.2
py-dateutil@2.4.0 py-numpy@1.9.1 py-pytz@2014.10 py-dateutil@2.4.0 py-numpy@1.9.1 py-pytz@2014.10
py-ipython@2.3.1 py-pygments@2.0.1 py-setuptools@11.3.1 py-ipython@2.3.1 py-pygments@2.0.1 py-setuptools@11.3.1
py-matplotlib@1.4.2 py-pyparsing@2.0.3 py-six@1.9.0 py-matplotlib@1.4.2 py-pyparsing@2.0.3 py-six@1.9.0
==> 3 currently activated: ==> 3 currently activated:
-- chaos_5_x86_64_ib / gcc@4.4.7 -------------------------------- -- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
py-nose@1.3.4 py-numpy@1.9.1 py-setuptools@11.3.1 py-nose@1.3.4 py-numpy@1.9.1 py-setuptools@11.3.1
@ -1711,7 +1715,7 @@ dependencies, you can use ``spack activate -f``:
.. code-block:: sh .. code-block:: sh
$ spack activate -f py-numpy $ spack activate -f py-numpy
==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7. ==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=linux-debian7-x86_64-66733244 for python@2.7.8%gcc@4.4.7.
.. _spack-deactivate: .. _spack-deactivate:
@ -1743,7 +1747,7 @@ Spack currently needs to be run from a filesystem that supports
``flock`` locking semantics. Nearly all local filesystems and recent ``flock`` locking semantics. Nearly all local filesystems and recent
versions of NFS support this, but parallel filesystems may be mounted versions of NFS support this, but parallel filesystems may be mounted
without ``flock`` support enabled. You can determine how your without ``flock`` support enabled. You can determine how your
filesystems are mounted with ``mount -p``. The output for a Lustre filesystems are mounted with ``mount -p``. The output for a Lustre
filesystem might look like this: filesystem might look like this:
.. code-block:: sh .. code-block:: sh
@ -1764,7 +1768,7 @@ This issue typically manifests with the error below:
Traceback (most recent call last): Traceback (most recent call last):
File "./spack", line 176, in <module> File "./spack", line 176, in <module>
main() main()
File "./spack", line 154, in main File "./spack", line 154,' in main
return_val = command(parser, args) return_val = command(parser, args)
File "./spack/lib/spack/spack/cmd/find.py", line 170, in find File "./spack/lib/spack/spack/cmd/find.py", line 170, in find
specs = set(spack.installed_db.query(**q_args)) specs = set(spack.installed_db.query(**q_args))
@ -1782,6 +1786,142 @@ This issue typically manifests with the error below:
A nicer error message is TBD in future versions of Spack. A nicer error message is TBD in future versions of Spack.
.. _spack-cray:
Spack on Cray
-----------------------------
Spack differs slightly when used on a Cray system. The architecture spec
can differentiate between the front-end and back-end processor and operating system.
For example, on Edison at NERSC, the back-end target processor
is \"Ivy Bridge\", so you can specify to use the back-end this way:
.. code-block:: sh
spack install zlib target=ivybridge
You can also use the operating system to build against the back-end:
.. code-block:: sh
spack install zlib os=CNL10
Notice that the name includes both the operating system name and the major
version number concatenated together.
Alternatively, if you want to build something for the front-end,
you can specify the front-end target processor. The processor for a login node
on Edison is \"Sandy bridge\" so we specify on the command line like so:
.. code-block:: sh
spack install zlib target=sandybridge
And the front-end operating system is:
.. code-block:: sh
spack install zlib os=SuSE11
Cray compiler detection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Spack can detect compilers using two methods. For the front-end, we treat
everything the same. The difference lies in back-end compiler detection.
Back-end compiler detection is made via the Tcl module avail command.
Once it detects the compiler it writes the appropriate PrgEnv and compiler
module name to compilers.yaml and sets the paths to each compiler with Cray\'s
compiler wrapper names (i.e. cc, CC, ftn). During build time, Spack will load
the correct PrgEnv and compiler module and will call appropriate wrapper.
The compilers.yaml config file will also differ. There is a
modules section that is filled with the compiler\'s Programming Environment
and module name. On other systems, this field is empty []::
...
- compiler:
modules:
- PrgEnv-intel
- intel/15.0.109
...
As mentioned earlier, the compiler paths will look different on a Cray system.
Since most compilers are invoked using cc, CC and ftn, the paths for each
compiler are replaced with their respective Cray compiler wrapper names::
...
paths:
cc: cc
cxx: CC
f77: ftn
fc: ftn
...
As opposed to an explicit path to the compiler executable. This allows Spack
to call the Cray compiler wrappers during build time.
For more on compiler configuration, check out :ref:`compiler-config`.
Setting defaults and using Cray modules
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you want to use default compilers for each PrgEnv and also be able
to load cray external modules, you will need to set up a packages.yaml.
Here\'s an example of an external configuration for cray modules:
.. code-block:: yaml
packages:
mpi:
modules:
mpich@7.3.1%gcc@5.2.0 arch=cray_xc-haswell-CNL10: cray-mpich
mpich@7.3.1%intel@16.0.0.109 arch=cray_xc-haswell-CNL10: cray-mpich
This tells Spack that for whatever package that depends on mpi, load the
cray-mpich module into the environment. You can then be able to use whatever
environment variables, libraries, etc, that are brought into the environment
via module load.
You can set the default compiler that Spack can use for each compiler type.
If you want to use the Cray defaults, then set them under *all:* in packages.yaml.
In the compiler field, set the compiler specs in your order of preference.
Whenever you build with that compiler type, Spack will concretize to that version.
Here is an example of a full packages.yaml used at NERSC
.. code-block:: yaml
packages:
mpi:
modules:
mpich@7.3.1%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge: cray-mpich
mpich@7.3.1%intel@16.0.0.109 arch=cray_xc-SuSE11-ivybridge: cray-mpich
buildable: False
netcdf:
modules:
netcdf@4.3.3.1%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge: cray-netcdf
netcdf@4.3.3.1%intel@16.0.0.109 arch=cray_xc-CNL10-ivybridge: cray-netcdf
buildable: False
hdf5:
modules:
hdf5@1.8.14%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge: cray-hdf5
hdf5@1.8.14%intel@16.0.0.109 arch=cray_xc-CNL10-ivybridge: cray-hdf5
buildable: False
all:
compiler: [gcc@5.2.0, intel@16.0.0.109]
Here we tell spack that whenever we want to build with gcc use version 5.2.0 or
if we want to build with intel compilers, use version 16.0.0.109. We add a spec
for each compiler type for each cray modules. This ensures that for each
compiler on our system we can use that external module.
For more on external packages check out the section :ref:`sec-external_packages`.
Getting Help Getting Help
----------------------- -----------------------

23
lib/spack/docs/configuration.rst
View File

@ -53,6 +53,7 @@ in the first directory it finds to which it has write access. Add
more elements to the list to indicate where your own site's temporary more elements to the list to indicate where your own site's temporary
directory is. directory is.
.. _sec-external_packages:
External Packages External Packages
---------------------------- ----------------------------
@ -70,20 +71,20 @@ directory. Here's an example of an external configuration:
packages: packages:
openmpi: openmpi:
paths: paths:
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib: /opt/openmpi-1.4.3 openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug
openmpi@1.6.5%intel@10.1 arch=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel
This example lists three installations of OpenMPI, one built with gcc, This example lists three installations of OpenMPI, one built with gcc,
one built with gcc and debug information, and another built with Intel. one built with gcc and debug information, and another built with Intel.
If Spack is asked to build a package that uses one of these MPIs as a If Spack is asked to build a package that uses one of these MPIs as a
dependency, it will use the the pre-installed OpenMPI in dependency, it will use the the pre-installed OpenMPI in
the given directory. the given directory. Packages.yaml can also be used to specify modules
Each ``packages.yaml`` begins with a ``packages:`` token, followed Each ``packages.yaml`` begins with a ``packages:`` token, followed
by a list of package names. To specify externals, add a ``paths`` by a list of package names. To specify externals, add a ``paths`` or ``modules``
token under the package name, which lists externals in a token under the package name, which lists externals in a
``spec : /path`` format. Each spec should be as ``spec: /path`` or ``spec: module-name`` format. Each spec should be as
well-defined as reasonably possible. If a well-defined as reasonably possible. If a
package lacks a spec component, such as missing a compiler or package lacks a spec component, such as missing a compiler or
package version, then Spack will guess the missing component based package version, then Spack will guess the missing component based
@ -108,9 +109,9 @@ be:
packages: packages:
openmpi: openmpi:
paths: paths:
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib: /opt/openmpi-1.4.3 openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug
openmpi@1.6.5%intel@10.1 arch=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel
buildable: False buildable: False
The addition of the ``buildable`` flag tells Spack that it should never build The addition of the ``buildable`` flag tells Spack that it should never build
@ -118,6 +119,9 @@ its own version of OpenMPI, and it will instead always rely on a pre-built
OpenMPI. Similar to ``paths``, ``buildable`` is specified as a property under OpenMPI. Similar to ``paths``, ``buildable`` is specified as a property under
a package name. a package name.
If an external module is specified as not buildable, then Spack will load the
external module into the build environment which can be used for linking.
The ``buildable`` does not need to be paired with external packages. The ``buildable`` does not need to be paired with external packages.
It could also be used alone to forbid packages that may be It could also be used alone to forbid packages that may be
buggy or otherwise undesirable. buggy or otherwise undesirable.
@ -181,7 +185,6 @@ concretization rules. A provider lists a value that packages may
dependency. dependency.
Profiling Profiling
------------------ ------------------

40
lib/spack/docs/packaging_guide.rst
View File

@ -1660,21 +1660,21 @@ the user runs ``spack install`` and the time the ``install()`` method
is called. The concretized version of the spec above might look like is called. The concretized version of the spec above might look like
this:: this::
mpileaks@2.3%gcc@4.7.3 arch=linux-ppc64 mpileaks@2.3%gcc@4.7.3 arch=linux-debian7-x86_64
^callpath@1.0%gcc@4.7.3+debug arch=linux-ppc64 ^callpath@1.0%gcc@4.7.3+debug arch=linux-debian7-x86_64
^dyninst@8.1.2%gcc@4.7.3 arch=linux-ppc64 ^dyninst@8.1.2%gcc@4.7.3 arch=linux-debian7-x86_64
^libdwarf@20130729%gcc@4.7.3 arch=linux-ppc64 ^libdwarf@20130729%gcc@4.7.3 arch=linux-debian7-x86_64
^libelf@0.8.11%gcc@4.7.3 arch=linux-ppc64 ^libelf@0.8.11%gcc@4.7.3 arch=linux-debian7-x86_64
^mpich@3.0.4%gcc@4.7.3 arch=linux-ppc64 ^mpich@3.0.4%gcc@4.7.3 arch=linux-debian7-x86_64
.. graphviz:: .. graphviz::
digraph { digraph {
"mpileaks@2.3\n%gcc@4.7.3\n arch=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n arch=linux-ppc64" "mpileaks@2.3\n%gcc@4.7.3\n arch=linux-debian7-x86_64" -> "mpich@3.0.4\n%gcc@4.7.3\n arch=linux-debian7-x86_64"
"mpileaks@2.3\n%gcc@4.7.3\n arch=linux-ppc64" -> "callpath@1.0\n%gcc@4.7.3+debug\n arch=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n arch=linux-ppc64" "mpileaks@2.3\n%gcc@4.7.3\n arch=linux-debian7-x86_64" -> "callpath@1.0\n%gcc@4.7.3+debug\n arch=linux-debian7-x86_64" -> "mpich@3.0.4\n%gcc@4.7.3\n arch=linux-debian7-x86_64"
"callpath@1.0\n%gcc@4.7.3+debug\n arch=linux-ppc64" -> "dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-ppc64" "callpath@1.0\n%gcc@4.7.3+debug\n arch=linux-debian7-x86_64" -> "dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-debian7-x86_64"
"dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-ppc64" -> "libdwarf@20130729\n%gcc@4.7.3\n arch=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n arch=linux-ppc64" "dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-debian7-x86_64" -> "libdwarf@20130729\n%gcc@4.7.3\n arch=linux-debian7-x86_64" -> "libelf@0.8.11\n%gcc@4.7.3\n arch=linux-debian7-x86_64"
"dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n arch=linux-ppc64" "dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-debian7-x86_64" -> "libelf@0.8.11\n%gcc@4.7.3\n arch=linux-debian7-x86_64"
} }
Here, all versions, compilers, and platforms are filled in, and there Here, all versions, compilers, and platforms are filled in, and there
@ -1703,9 +1703,9 @@ running ``spack spec``. For example:
^libdwarf ^libdwarf
^libelf ^libelf
dyninst@8.0.1%gcc@4.7.3 arch=linux-ppc64 dyninst@8.0.1%gcc@4.7.3 arch=linux-debian7-x86_64
^libdwarf@20130729%gcc@4.7.3 arch=linux-ppc64 ^libdwarf@20130729%gcc@4.7.3 arch=linux-debian7-x86_64
^libelf@0.8.13%gcc@4.7.3 arch=linux-ppc64 ^libelf@0.8.13%gcc@4.7.3 arch=linux-debian7-x86_64
This is useful when you want to know exactly what Spack will do when This is useful when you want to know exactly what Spack will do when
you ask for a particular spec. you ask for a particular spec.
@ -2210,12 +2210,12 @@ example:
def install(self, prefix): def install(self, prefix):
# Do default install # Do default install
@when('arch=chaos_5_x86_64_ib') @when('arch=linux-debian7-x86_64')
def install(self, prefix): def install(self, prefix):
# This will be executed instead of the default install if # This will be executed instead of the default install if
# the package's sys_type() is chaos_5_x86_64_ib. # the package's sys_type() is chaos_5_x86_64_ib.
@when('arch=bgqos_0") @when('arch=linux-debian7-x86_64")
def install(self, prefix): def install(self, prefix):
# This will be executed if the package's sys_type is bgqos_0 # This will be executed if the package's sys_type is bgqos_0
@ -2343,7 +2343,7 @@ build system.
.. _sanity-checks: .. _sanity-checks:
Sanity checking an intallation Sanity checking an installation
-------------------------------- --------------------------------
By default, Spack assumes that a build has failed if nothing is By default, Spack assumes that a build has failed if nothing is
@ -2863,11 +2863,11 @@ build it:
$ spack stage libelf $ spack stage libelf
==> Trying to fetch from http://www.mr511.de/software/libelf-0.8.13.tar.gz ==> Trying to fetch from http://www.mr511.de/software/libelf-0.8.13.tar.gz
######################################################################## 100.0% ######################################################################## 100.0%
==> Staging archive: /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-ppc64/libelf-0.8.13.tar.gz ==> Staging archive: /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-debian7-x86_64/libelf-0.8.13.tar.gz
==> Created stage in /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-ppc64. ==> Created stage in /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-debian7-x86_64.
$ spack cd libelf $ spack cd libelf
$ pwd $ pwd
/Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-ppc64/libelf-0.8.13 /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-debian7-x86_64/libelf-0.8.13
``spack cd`` here changed he current working directory to the ``spack cd`` here changed he current working directory to the
directory containing the expanded ``libelf`` source code. There are a directory containing the expanded ``libelf`` source code. There are a