zhangyiss/spack
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Use console instead of shell syntax lexer, fixes #1634

Browse Source
This commit is contained in:
Adam J. Stewart 2016-08-26 16:41:00 -05:00
parent f856952728
commit 6795f4ca12
8 changed files with 949 additions and 981 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

815
lib/spack/docs/basic_usage.rst
View File

File diff suppressed because it is too large Load Diff

62
lib/spack/docs/configuration.rst
View File

@ -10,7 +10,9 @@ Configuration
Temporary space
---------------
.. warning:: Temporary space configuration will eventually be moved to
.. warning::
Temporary space configuration will eventually be moved to
configuration files, but currently these settings are in
``lib/spack/spack/__init__.py``
@ -73,11 +75,11 @@ directory. Here's an example of an external configuration:
.. code-block:: yaml
packages:
openmpi:
paths:
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug
openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel
openmpi:
paths:
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug
openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel
This example lists three installations of OpenMPI, one built with gcc,
one built with gcc and debug information, and another built with Intel.
@ -110,13 +112,13 @@ be:
.. code-block:: yaml
packages:
openmpi:
paths:
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug
openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel
buildable: False
packages:
openmpi:
paths:
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug
openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel
buildable: False
The addition of the ``buildable`` flag tells Spack that it should never build
its own version of OpenMPI, and it will instead always rely on a pre-built
@ -144,18 +146,18 @@ The preferred configuration can be controlled via the
Here's an example packages.yaml file that sets preferred packages:
.. code-block:: sh
.. code-block:: yaml
packages:
opencv:
compiler: [gcc@4.9]
variants: +debug
gperftools:
version: [2.2, 2.4, 2.3]
all:
compiler: [gcc@4.4.7, gcc@4.6:, intel, clang, pgi]
providers:
mpi: [mvapich, mpich, openmpi]
packages:
opencv:
compiler: [gcc@4.9]
variants: +debug
gperftools:
version: [2.2, 2.4, 2.3]
all:
compiler: [gcc@4.4.7, gcc@4.6:, intel, clang, pgi]
providers:
mpi: [mvapich, mpich, openmpi]
At a high level, this example is specifying how packages should be
concretized. The opencv package should prefer using gcc 4.9 and
@ -205,15 +207,15 @@ supply ``-p`` to Spack on the command line, before any subcommands.
``spack -p`` output looks like this:
.. code-block:: sh
.. code-block:: console
$ spack -p graph dyninst
o dyninst
|\
| |\
| o | libdwarf
|/ /
o | libelf
\|\
\| \|\
\| o \| libdwarf
\|/ /
o \| libelf
/
o boost

104
lib/spack/docs/developer_guide.rst
View File

@ -18,12 +18,12 @@ Overview
Spack is designed with three separate roles in mind:
#. **Users**, who need to install software *without* knowing all the
details about how it is built.
#. **Packagers** who know how a particular software package is
built and encode this information in package files.
#. **Developers** who work on Spack, add new features, and try to
make the jobs of packagers and users easier.
#. **Users**, who need to install software *without* knowing all the
details about how it is built.
#. **Packagers** who know how a particular software package is
built and encode this information in package files.
#. **Developers** who work on Spack, add new features, and try to
make the jobs of packagers and users easier.
Users could be end users installing software in their home directory,
or administrators installing software to a shared directory on a
@ -43,9 +43,9 @@ specification.
This gets us to the two key concepts in Spack's software design:
#. **Specs**: expressions for describing builds of software, and
#. **Packages**: Python modules that build software according to a
spec.
#. **Specs**: expressions for describing builds of software, and
#. **Packages**: Python modules that build software according to a
spec.
A package is a template for building particular software, and a spec
as a descriptor for one or more instances of that template. Users
@ -70,56 +70,55 @@ Directory Structure
-------------------
So that you can familiarize yourself with the project, we'll start
with a high level view of Spack's directory structure::
with a high level view of Spack's directory structure:
spack/ <- installation root
bin/
spack <- main spack executable
.. code-block:: none
etc/
spack/ <- Spack config files.
Can be overridden by files in ~/.spack.
spack/ <- installation root
bin/
spack <- main spack executable
var/
spack/ <- build & stage directories
repos/ <- contains package repositories
builtin/ <- pkg repository that comes with Spack
repo.yaml <- descriptor for the builtin repository
packages/ <- directories under here contain packages
cache/ <- saves resources downloaded during installs
etc/
spack/ <- Spack config files.
Can be overridden by files in ~/.spack.
opt/
spack/ <- packages are installed here
var/
spack/ <- build & stage directories
repos/ <- contains package repositories
builtin/ <- pkg repository that comes with Spack
repo.yaml <- descriptor for the builtin repository
packages/ <- directories under here contain packages
cache/ <- saves resources downloaded during installs
lib/
spack/
docs/ <- source for this documentation
env/ <- compiler wrappers for build environment
opt/
spack/ <- packages are installed here
external/ <- external libs included in Spack distro
llnl/ <- some general-use libraries
lib/
spack/
docs/ <- source for this documentation
env/ <- compiler wrappers for build environment
spack/ <- spack module; contains Python code
cmd/ <- each file in here is a spack subcommand
compilers/ <- compiler description files
test/ <- unit test modules
util/ <- common code
external/ <- external libs included in Spack distro
llnl/ <- some general-use libraries
spack/ <- spack module; contains Python code
cmd/ <- each file in here is a spack subcommand
compilers/ <- compiler description files
test/ <- unit test modules
util/ <- common code
Spack is designed so that it could live within a `standard UNIX
directory hierarchy <http://linux.die.net/man/7/hier>`_, so ``lib``,
``var``, and ``opt`` all contain a ``spack`` subdirectory in case
Spack is installed alongside other software. Most of the interesting
parts of Spack live in ``lib/spack``. Files under ``var`` are created
as needed, so there is no ``var`` directory when you initially clone
Spack from the repository.
parts of Spack live in ``lib/spack``.
Spack has *one* directory layout and there is no install process.
version and the source code. Most Python programs don't look like
this (they use distutils, ``setup.py``, etc.) but we wanted to make
Spack *very* easy to use. The simple layout spares users from the
need to install Spack into a Python environment. Many users don't
have write access to a Python installation, and installing an entire
new instance of Python to bootstrap Spack would be very complicated.
Most Python programs don't look like this (they use distutils, ``setup.py``,
etc.) but we wanted to make Spack *very* easy to use. The simple layout
spares users from the need to install Spack into a Python environment.
Many users don't have write access to a Python installation, and installing
an entire new instance of Python to bootstrap Spack would be very complicated.
Users should not have to install install a big, complicated package to
use the thing that's supposed to spare them from the details of big,
complicated packages. The end result is that Spack works out of the
@ -283,19 +282,18 @@ Spec objects
Package objects
---------------
Most spack commands
look something like this:
Most spack commands look something like this:
#. Parse an abstract spec (or specs) from the command line,
#. *Normalize* the spec based on information in package files,
#. *Concretize* the spec according to some customizable policies,
#. Instantiate a package based on the spec, and
#. Call methods (e.g., ``install()``) on the package object.
#. Parse an abstract spec (or specs) from the command line,
#. *Normalize* the spec based on information in package files,
#. *Concretize* the spec according to some customizable policies,
#. Instantiate a package based on the spec, and
#. Call methods (e.g., ``install()``) on the package object.
The information in Package files is used at all stages in this
process.
Conceptually, packages are overloaded. They contain
Conceptually, packages are overloaded. They contain:
-------------
Stage objects

26
lib/spack/docs/features.rst
View File

@ -14,7 +14,7 @@ Simple package installation
Installing the default version of a package is simple. This will install
the latest version of the ``mpileaks`` package and all of its dependencies:
.. code-block:: sh
.. code-block:: console
$ spack install mpileaks
@ -26,7 +26,7 @@ Spack allows installation to be customized. Users can specify the
version, build compiler, compile-time options, and cross-compile
platform, all on the command line.
.. code-block:: sh
.. code-block:: console
# Install a particular version by appending @
$ spack install mpileaks@1.1.2
@ -59,7 +59,7 @@ customized extensively. Suppose that ``mpileaks`` depends indirectly
on ``libelf`` and ``libdwarf``. Using ``^``, users can add custom
configurations for the dependencies:
.. code-block:: sh
.. code-block:: console
# Install mpileaks and link it with specific versions of libelf and libdwarf
$ spack install mpileaks@1.1.2 %gcc@4.7.3 +debug ^libelf@0.8.12 ^libdwarf@20130729+debug
@ -91,7 +91,7 @@ in pure Python.
For example, this command:
.. code-block:: sh
.. code-block:: console
$ spack create http://www.mr511.de/software/libelf-0.8.13.tar.gz
@ -101,16 +101,26 @@ creates a simple python file:
from spack import *
class Libelf(Package):
homepage = "http://www.example.com/"
"""FIXME: Put a proper description of your package here."""
# FIXME: Add a proper url for your package's homepage here.
homepage = "http://www.example.com"
url = "http://www.mr511.de/software/libelf-0.8.13.tar.gz"
version('0.8.13', '4136d7b4c04df68b686570afa26988ac')
def install(self, prefix):
configure("--prefix=%s" % prefix)
# FIXME: Add dependencies if required.
# depends_on('foo')
def install(self, spec, prefix):
# FIXME: Modify the configure line to suit your build system here.
configure('--prefix={0}'.format(prefix))
# FIXME: Add logic to build and install here.
make()
make("install")
make('install')
It doesn't take much python coding to get from there to a working
package:

11
lib/spack/docs/getting_started.rst
View File

@ -9,7 +9,7 @@ Download
Getting spack is easy. You can clone it from the `github repository
<https://github.com/llnl/spack>`_ using this command:
.. code-block:: sh
.. code-block:: console
$ git clone https://github.com/llnl/spack.git
@ -18,7 +18,7 @@ full path to this directory is in the ``SPACK_ROOT`` environment
variable. Add ``$SPACK_ROOT/bin`` to your path and you're ready to
go:
.. code-block:: sh
.. code-block:: console
$ export PATH=$SPACK_ROOT/bin:$PATH
$ spack install libelf
@ -26,9 +26,10 @@ go:
For a richer experience, use Spack's `shell support
<http://software.llnl.gov/spack/basic_usage.html#environment-modules>`_:
.. code-block:: sh
.. code-block:: console
# For bash users
$ export SPACK_ROOT=/path/to/spack
$ . $SPACK_ROOT/share/spack/setup-env.sh
# For tcsh or csh users (note you must set SPACK_ROOT)
@ -48,9 +49,9 @@ You may want to run it out of a prefix other than the git repository
you cloned. The ``spack bootstrap`` command provides this
functionality. To install spack in a new directory, simply type:
.. code-block:: sh
.. code-block:: console
$ spack bootstrap /my/favorite/prefix
$ spack bootstrap /my/favorite/prefix
This will install a new spack script in ``/my/favorite/prefix/bin``,
which you can use just like you would the regular spack script. Each

2
lib/spack/docs/index.rst
View File

@ -28,7 +28,7 @@ Get spack from the `github repository
<https://github.com/llnl/spack>`_ and install your first
package:
.. code-block:: sh
.. code-block:: console
$ git clone https://github.com/llnl/spack.git
$ cd spack/bin

102
lib/spack/docs/mirrors.rst
View File

@ -11,27 +11,29 @@ mirror is a URL that points to a directory, either on the local
filesystem or on some server, containing tarballs for all of Spack's
packages.
Here's an example of a mirror's directory structure::
Here's an example of a mirror's directory structure:
mirror/
cmake/
cmake-2.8.10.2.tar.gz
dyninst/
dyninst-8.1.1.tgz
dyninst-8.1.2.tgz
libdwarf/
libdwarf-20130126.tar.gz
libdwarf-20130207.tar.gz
libdwarf-20130729.tar.gz
libelf/
libelf-0.8.12.tar.gz
libelf-0.8.13.tar.gz
libunwind/
libunwind-1.1.tar.gz
mpich/
mpich-3.0.4.tar.gz
mvapich2/
mvapich2-1.9.tgz
.. code-block:: none
mirror/
cmake/
cmake-2.8.10.2.tar.gz
dyninst/
dyninst-8.1.1.tgz
dyninst-8.1.2.tgz
libdwarf/
libdwarf-20130126.tar.gz
libdwarf-20130207.tar.gz
libdwarf-20130729.tar.gz
libelf/
libelf-0.8.12.tar.gz
libelf-0.8.13.tar.gz
libunwind/
libunwind-1.1.tar.gz
mpich/
mpich-3.0.4.tar.gz
mvapich2/
mvapich2-1.9.tgz
The structure is very simple. There is a top-level directory. The
second level directories are named after packages, and the third level
@ -57,21 +59,9 @@ contains tarballs for each package, named after each package.
----------------
Mirrors are managed with the ``spack mirror`` command. The help for
``spack mirror`` looks like this::
``spack mirror`` looks like this:
$ spack mirror -h
usage: spack mirror [-h] SUBCOMMAND ...
positional arguments:
SUBCOMMAND
create Create a directory to be used as a spack mirror, and fill
it with package archives.
add Add a mirror to Spack.
remove Remove a mirror by name.
list Print out available mirrors to the console.
optional arguments:
-h, --help show this help message and exit
.. command-output:: spack help mirror
The ``create`` command actually builds a mirror by fetching all of its
packages from the internet and checksumming them.
@ -92,7 +82,7 @@ The command will iterate through all of Spack's packages and download
the safe ones into a directory structure like the one above. Here is
what it looks like:
.. code-block:: bash
.. code-block:: console
$ spack mirror create libelf libdwarf
==> Created new mirror in spack-mirror-2014-06-24
@ -134,9 +124,11 @@ Normally, ``spack mirror create`` downloads all the archives it has
checksums for. If you want to only create a mirror for a subset of
packages, you can do that by supplying a list of package specs on the
command line after ``spack mirror create``. For example, this
command::
command:
$ spack mirror create libelf@0.8.12: boost@1.44:
.. code-block:: console
$ spack mirror create libelf@0.8.12: boost@1.44:
Will create a mirror for libelf versions greater than or equal to
0.8.12 and boost versions greater than or equal to 1.44.
@ -146,7 +138,9 @@ Mirror files
^^^^^^^^^^^^
If you have a *very* large number of packages you want to mirror, you
can supply a file with specs in it, one per line::
can supply a file with specs in it, one per line:
.. code-block:: console
$ cat specs.txt
libdwarf
@ -168,19 +162,21 @@ your site.
Once you have a mirror, you need to let spack know about it. This is
relatively simple. First, figure out the URL for the mirror. If it's
a file, you can use a file URL like this one::
a file, you can use a file URL like this one:
file:///Users/gamblin2/spack-mirror-2014-06-24
.. code-block:: none
file:///Users/gamblin2/spack-mirror-2014-06-24
That points to the directory on the local filesystem. If it were on a
web server, you could use a URL like this one:
https://example.com/some/web-hosted/directory/spack-mirror-2014-06-24
https://example.com/some/web-hosted/directory/spack-mirror-2014-06-24
Spack will use the URL as the root for all of the packages it fetches.
You can tell your Spack installation to use that mirror like this:
.. code-block:: bash
.. code-block:: console
$ spack mirror add local_filesystem file:///Users/gamblin2/spack-mirror-2014-06-24
@ -192,7 +188,9 @@ Each mirror has a name so that you can refer to it again later.
``spack mirror list``
---------------------
To see all the mirrors Spack knows about, run ``spack mirror list``::
To see all the mirrors Spack knows about, run ``spack mirror list``:
.. code-block:: console
$ spack mirror list
local_filesystem file:///Users/gamblin2/spack-mirror-2014-06-24
@ -203,7 +201,9 @@ To see all the mirrors Spack knows about, run ``spack mirror list``::
``spack mirror remove``
-----------------------
To remove a mirror by name::
To remove a mirror by name, run:
.. code-block:: console
$ spack mirror remove local_filesystem
$ spack mirror list
@ -213,7 +213,9 @@ To remove a mirror by name::
Mirror precedence
-----------------
Adding a mirror really adds a line in ``~/.spack/mirrors.yaml``::
Adding a mirror really adds a line in ``~/.spack/mirrors.yaml``:
.. code-block:: yaml
mirrors:
local_filesystem: file:///Users/gamblin2/spack-mirror-2014-06-24
@ -232,12 +234,12 @@ Local Default Cache
Spack caches resources that are downloaded as part of installs. The cache is
a valid spack mirror: it uses the same directory structure and naming scheme
as other Spack mirrors (so it can be copied anywhere and referenced with a URL
like other mirrors). The mirror is maintained locally (within the Spack
installation directory) at :file:`var/spack/cache/`. It is always enabled (and
is always searched first when attempting to retrieve files for an installation)
like other mirrors). The mirror is maintained locally (within the Spack
installation directory) at :file:`var/spack/cache/`. It is always enabled (and
is always searched first when attempting to retrieve files for an installation)
but can be cleared with :ref:`purge <spack-purge>`; the cache directory can also
be deleted manually without issue.
be deleted manually without issue.
Caching includes retrieved tarball archives and source control repositories, but
only resources with an associated digest or commit ID (e.g. a revision number
only resources with an associated digest or commit ID (e.g. a revision number
for SVN) will be cached.

808
lib/spack/docs/packaging_guide.rst
View File

File diff suppressed because it is too large Load Diff