Last minute edits to the Configuration Tutorial (#6271)
This commit is contained in:
parent
b77acd0896
commit
9db7eaade8
@ -133,7 +133,7 @@ We will start by opening the compilers configuration file
|
|||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
spack config edit compilers
|
$ spack config edit compilers
|
||||||
|
|
||||||
|
|
||||||
.. code-block:: yaml
|
.. code-block:: yaml
|
||||||
@ -514,58 +514,13 @@ okay.
|
|||||||
|
|
||||||
Concretized
|
Concretized
|
||||||
--------------------------------
|
--------------------------------
|
||||||
hdf5@1.10.1%clang@3.8.0-2ubuntu4+cxx~debug~fortran~hl+mpi+pic~shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64
|
hdf5@1.10.1%gcc@5.4.0~cxx~debug~fortran~hl~mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64
|
||||||
^zlib@1.2.11%clang@3.8.0-2ubuntu4+pic~shared arch=linux-ubuntu16.04-x86_64
|
^zlib@1.2.8%gcc@5.4.0+optimize+pic~shared arch=linux-ubuntu16.04-x86_64
|
||||||
|
|
||||||
|
|
||||||
You'll notice that this did not use our external zlib. Why?
|
You'll notice that Spack is now using the external zlib installation,
|
||||||
Because Spack concretized zlib to a different Spec than the
|
but the compiler used to build zlib is now overriding our compiler
|
||||||
one we provided. There are two ways we could get Spack to
|
preference of clang. If we explicitly specify clang:
|
||||||
build with our external zlib. One would be to explicitly ask
|
|
||||||
for that spec on the command line. The other is to tell Spack
|
|
||||||
it's not allowed to build its own zlib. We'll go with the latter.
|
|
||||||
|
|
||||||
.. code-block:: yaml
|
|
||||||
|
|
||||||
packages:
|
|
||||||
all:
|
|
||||||
compiler: [clang, gcc, intel, pgi, xl, nag]
|
|
||||||
providers:
|
|
||||||
mpi: [mpich, openmpi]
|
|
||||||
variants: ~shared
|
|
||||||
hdf5:
|
|
||||||
variants: ~mpi
|
|
||||||
zlib:
|
|
||||||
paths:
|
|
||||||
zlib@1.2.11%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64: /usr
|
|
||||||
buildable: False
|
|
||||||
|
|
||||||
|
|
||||||
Now Spack will be forced to choose the external zlib.
|
|
||||||
|
|
||||||
.. code-block:: console
|
|
||||||
|
|
||||||
$ spack spec hdf5
|
|
||||||
Input spec
|
|
||||||
--------------------------------
|
|
||||||
hdf5
|
|
||||||
|
|
||||||
Normalized
|
|
||||||
--------------------------------
|
|
||||||
hdf5
|
|
||||||
^zlib@1.1.2:
|
|
||||||
|
|
||||||
Concretized
|
|
||||||
--------------------------------
|
|
||||||
hdf5@1.10.1%gcc@5.4.0+cxx~debug~fortran~hl+mpi+pic~shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64
|
|
||||||
^zlib@1.2.11%gcc@5.4.0+pic+shared arch=linux-ubuntu16.04-x86_64
|
|
||||||
|
|
||||||
|
|
||||||
Note that Spack now concretizes the entire DAG to use the gcc
|
|
||||||
compiler. Because we did not specify a build using the clang compiler
|
|
||||||
(only expressed a preference) Spack used the gcc compiler specified by
|
|
||||||
the zlib spec. If we want to use clang for the rest of the build, we
|
|
||||||
have to specify it.
|
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
@ -581,8 +536,50 @@ have to specify it.
|
|||||||
|
|
||||||
Concretized
|
Concretized
|
||||||
--------------------------------
|
--------------------------------
|
||||||
hdf5@1.10.1%clang@3.8.0-2ubuntu4+cxx~debug~fortran~hl+mpi+pic~shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64
|
hdf5@1.10.1%clang@3.8.0-2ubuntu4~cxx~debug~fortran~hl~mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64
|
||||||
^zlib@1.2.11%gcc@5.4.0+pic+shared arch=linux-ubuntu16.04-x86_64
|
^zlib@1.2.11%clang@3.8.0-2ubuntu4+optimize+pic~shared arch=linux-ubuntu16.04-x86_64
|
||||||
|
|
||||||
|
|
||||||
|
Spack concretizes to both hdf5 and zlib being built with clang.
|
||||||
|
This has a side-effect of rebuilding zlib. If we want to force
|
||||||
|
Spack to use the system zlib, we have two choices. We can either
|
||||||
|
specify it on the command line, or we can tell Spack that it's
|
||||||
|
not allowed to build its own zlib. We'll go with the latter.
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
packages:
|
||||||
|
all:
|
||||||
|
compiler: [clang, gcc, intel, pgi, xl, nag]
|
||||||
|
providers:
|
||||||
|
mpi: [mpich, openmpi]
|
||||||
|
variants: ~shared
|
||||||
|
hdf5:
|
||||||
|
variants: ~mpi
|
||||||
|
zlib:
|
||||||
|
paths:
|
||||||
|
zlib@1.2.8%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64: /usr
|
||||||
|
buildable: False
|
||||||
|
|
||||||
|
|
||||||
|
Now Spack will be forced to choose the external zlib.
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ spack spec hdf5 %clang
|
||||||
|
Input spec
|
||||||
|
--------------------------------
|
||||||
|
hdf5%clang
|
||||||
|
|
||||||
|
Normalized
|
||||||
|
--------------------------------
|
||||||
|
hdf5%clang
|
||||||
|
^zlib@1.1.2:
|
||||||
|
|
||||||
|
Concretized
|
||||||
|
--------------------------------
|
||||||
|
hdf5@1.10.1%clang@3.8.0-2ubuntu4~cxx~debug~fortran~hl~mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64
|
||||||
|
^zlib@1.2.8%gcc@5.4.0+optimize+pic~shared arch=linux-ubuntu16.04-x86_64
|
||||||
|
|
||||||
|
|
||||||
This gets slightly more complicated with virtual dependencies. Suppose
|
This gets slightly more complicated with virtual dependencies. Suppose
|
||||||
@ -601,7 +598,7 @@ of hdf5? Well, fortunately we have mpich installed on these systems.
|
|||||||
variants: ~mpi
|
variants: ~mpi
|
||||||
zlib:
|
zlib:
|
||||||
paths:
|
paths:
|
||||||
zlib@1.2.11%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64: /usr
|
zlib@1.2.8%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64: /usr
|
||||||
buildable: False
|
buildable: False
|
||||||
mpich:
|
mpich:
|
||||||
paths:
|
paths:
|
||||||
@ -638,7 +635,7 @@ build with an alternate MPI implementation.
|
|||||||
^util-macros@1.19.1%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64
|
^util-macros@1.19.1%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64
|
||||||
^libxml2@2.9.4%clang@3.8.0-2ubuntu4~python arch=linux-ubuntu16.04-x86_64
|
^libxml2@2.9.4%clang@3.8.0-2ubuntu4~python arch=linux-ubuntu16.04-x86_64
|
||||||
^xz@5.2.3%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64
|
^xz@5.2.3%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64
|
||||||
^zlib@1.2.11%gcc@5.4.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64
|
^zlib@1.2.8%gcc@5.4.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64
|
||||||
|
|
||||||
|
|
||||||
We have only expressed a preference for mpich over other MPI
|
We have only expressed a preference for mpich over other MPI
|
||||||
@ -662,7 +659,7 @@ again.
|
|||||||
variants: ~shared
|
variants: ~shared
|
||||||
zlib:
|
zlib:
|
||||||
paths:
|
paths:
|
||||||
zlib@1.2.11%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64: /usr
|
zlib@1.2.8%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64: /usr
|
||||||
buildable: False
|
buildable: False
|
||||||
mpich:
|
mpich:
|
||||||
paths:
|
paths:
|
||||||
@ -699,17 +696,22 @@ providers for MPI we can try again.
|
|||||||
--------------------------------
|
--------------------------------
|
||||||
hdf5@1.10.1%clang@3.8.0-2ubuntu4+cxx~debug~fortran~hl+mpi+pic~shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64
|
hdf5@1.10.1%clang@3.8.0-2ubuntu4+cxx~debug~fortran~hl+mpi+pic~shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64
|
||||||
^mpich@3.2%gcc@5.4.0 device=ch3 +hydra netmod=tcp +pmi+romio~verbs arch=linux-ubuntu16.04-x86_64
|
^mpich@3.2%gcc@5.4.0 device=ch3 +hydra netmod=tcp +pmi+romio~verbs arch=linux-ubuntu16.04-x86_64
|
||||||
^zlib@1.2.11%gcc@5.4.0+pic+shared arch=linux-ubuntu16.04-x86_64
|
^zlib@1.2.8%gcc@5.4.0+pic+shared arch=linux-ubuntu16.04-x86_64
|
||||||
|
|
||||||
Now that we have hdf5 configured to install exactly as we want it, we
|
|
||||||
can install it. We've now minimized the command line effort necessary
|
|
||||||
to get exactly the hdf5 installation we want, and we can now build
|
|
||||||
hdf5 against our external installations of zlib and mpich.
|
|
||||||
|
|
||||||
.. code-block:: console
|
By configuring most of our package preferences in ``packages.yaml``,
|
||||||
|
we can cut down on the amount of work we need to do when specifying
|
||||||
|
a spec on the command line. In addition to compiler and variant
|
||||||
|
preferences, we can specify version preferences as well. Anything
|
||||||
|
that you can specify on the command line can be specified in
|
||||||
|
``packages.yaml`` with the exact same spec syntax.
|
||||||
|
|
||||||
$ spack install hdf5 %clang
|
.. warning::
|
||||||
...
|
|
||||||
|
Make sure to delete or move the ``packages.yaml`` you have been
|
||||||
|
editing up to this point. Otherwise, it will change the hashes
|
||||||
|
of your packages, leading to differences in the output of later
|
||||||
|
tutorial sections.
|
||||||
|
|
||||||
|
|
||||||
-----------------
|
-----------------
|
||||||
@ -757,11 +759,11 @@ the build environment with the ``dirty`` setting:
|
|||||||
However, this is strongly discouraged, as it can pull unwanted libraries
|
However, this is strongly discouraged, as it can pull unwanted libraries
|
||||||
into the build.
|
into the build.
|
||||||
|
|
||||||
One last setting that may be of interest to users building on a shared
|
One last setting that may be of interest to many users is the ability
|
||||||
login node is the ability to customize the parallelism of Spack builds.
|
to customize the parallelism of Spack builds. By default, Spack
|
||||||
By default, Spack installs all packages in parallel with the number of
|
installs all packages in parallel with the number of jobs equal to the
|
||||||
jobs equal to the number of cores on the node. For example, on a node
|
number of cores on the node. For example, on a node with 36 cores,
|
||||||
with 36 cores, this will look like:
|
this will look like:
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
@ -839,10 +841,3 @@ If we uninstall and reinstall zlib, we see that it now uses only 4 cores:
|
|||||||
Obviously, if you want to build everything in serial for whatever reason,
|
Obviously, if you want to build everything in serial for whatever reason,
|
||||||
you would set ``build_jobs`` to 1.
|
you would set ``build_jobs`` to 1.
|
||||||
|
|
||||||
.. warning::
|
|
||||||
|
|
||||||
At this point, make sure you delete or move the ``packages.yaml`` and
|
|
||||||
``config.yaml`` you have been editing up to this point. Otherwise, they
|
|
||||||
will change the hashes of your packages, leading to differences in the
|
|
||||||
output of later tutorial sections.
|
|
||||||
|
|
||||||
|
Loading…
Reference in New Issue
Block a user