zhangyiss/spack
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Use console instead of shell syntax lexer, fixes #1634

Browse Source
This commit is contained in:
Adam J. Stewart 2016-08-26 16:41:00 -05:00
parent f856952728
commit 6795f4ca12
8 changed files with 949 additions and 981 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

815
lib/spack/docs/basic_usage.rst
View File

File diff suppressed because it is too large Load Diff

62
lib/spack/docs/configuration.rst
View File

@ -10,7 +10,9 @@ Configuration
Temporary space Temporary space
--------------- ---------------
.. warning:: Temporary space configuration will eventually be moved to .. warning::
Temporary space configuration will eventually be moved to
configuration files, but currently these settings are in configuration files, but currently these settings are in
``lib/spack/spack/__init__.py`` ``lib/spack/spack/__init__.py``
@ -73,11 +75,11 @@ directory. Here's an example of an external configuration:
.. code-block:: yaml .. code-block:: yaml
packages: packages:
openmpi: openmpi:
paths: paths:
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3 openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug
openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel
This example lists three installations of OpenMPI, one built with gcc, This example lists three installations of OpenMPI, one built with gcc,
one built with gcc and debug information, and another built with Intel. one built with gcc and debug information, and another built with Intel.
@ -110,13 +112,13 @@ be:
.. code-block:: yaml .. code-block:: yaml
packages: packages:
openmpi: openmpi:
paths: paths:
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3 openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug
openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel
buildable: False buildable: False
The addition of the ``buildable`` flag tells Spack that it should never build The addition of the ``buildable`` flag tells Spack that it should never build
its own version of OpenMPI, and it will instead always rely on a pre-built its own version of OpenMPI, and it will instead always rely on a pre-built
@ -144,18 +146,18 @@ The preferred configuration can be controlled via the
Here's an example packages.yaml file that sets preferred packages: Here's an example packages.yaml file that sets preferred packages:
.. code-block:: sh .. code-block:: yaml
packages: packages:
opencv: opencv:
compiler: [gcc@4.9] compiler: [gcc@4.9]
variants: +debug variants: +debug
gperftools: gperftools:
version: [2.2, 2.4, 2.3] version: [2.2, 2.4, 2.3]
all: all:
compiler: [gcc@4.4.7, gcc@4.6:, intel, clang, pgi] compiler: [gcc@4.4.7, gcc@4.6:, intel, clang, pgi]
providers: providers:
mpi: [mvapich, mpich, openmpi] mpi: [mvapich, mpich, openmpi]
At a high level, this example is specifying how packages should be At a high level, this example is specifying how packages should be
concretized. The opencv package should prefer using gcc 4.9 and concretized. The opencv package should prefer using gcc 4.9 and
@ -205,15 +207,15 @@ supply ``-p`` to Spack on the command line, before any subcommands.
``spack -p`` output looks like this: ``spack -p`` output looks like this:
.. code-block:: sh .. code-block:: console
$ spack -p graph dyninst $ spack -p graph dyninst
o dyninst o dyninst
|\ \|\
| |\ \| \|\
| o | libdwarf \| o \| libdwarf
|/ / \|/ /
o | libelf o \| libelf
/ /
o boost o boost

104
lib/spack/docs/developer_guide.rst
View File

@ -18,12 +18,12 @@ Overview
Spack is designed with three separate roles in mind: Spack is designed with three separate roles in mind:
#. **Users**, who need to install software *without* knowing all the #. **Users**, who need to install software *without* knowing all the
details about how it is built. details about how it is built.
#. **Packagers** who know how a particular software package is #. **Packagers** who know how a particular software package is
built and encode this information in package files. built and encode this information in package files.
#. **Developers** who work on Spack, add new features, and try to #. **Developers** who work on Spack, add new features, and try to
make the jobs of packagers and users easier. make the jobs of packagers and users easier.
Users could be end users installing software in their home directory, Users could be end users installing software in their home directory,
or administrators installing software to a shared directory on a or administrators installing software to a shared directory on a
@ -43,9 +43,9 @@ specification.
This gets us to the two key concepts in Spack's software design: This gets us to the two key concepts in Spack's software design:
#. **Specs**: expressions for describing builds of software, and #. **Specs**: expressions for describing builds of software, and
#. **Packages**: Python modules that build software according to a #. **Packages**: Python modules that build software according to a
spec. spec.
A package is a template for building particular software, and a spec A package is a template for building particular software, and a spec
as a descriptor for one or more instances of that template. Users as a descriptor for one or more instances of that template. Users
@ -70,56 +70,55 @@ Directory Structure
------------------- -------------------
So that you can familiarize yourself with the project, we'll start So that you can familiarize yourself with the project, we'll start
with a high level view of Spack's directory structure:: with a high level view of Spack's directory structure:
spack/ <- installation root .. code-block:: none
bin/
spack <- main spack executable
etc/ spack/ <- installation root
spack/ <- Spack config files. bin/
Can be overridden by files in ~/.spack. spack <- main spack executable
var/ etc/
spack/ <- build & stage directories spack/ <- Spack config files.
repos/ <- contains package repositories Can be overridden by files in ~/.spack.
builtin/ <- pkg repository that comes with Spack
repo.yaml <- descriptor for the builtin repository
packages/ <- directories under here contain packages
cache/ <- saves resources downloaded during installs
opt/ var/
spack/ <- packages are installed here spack/ <- build & stage directories
repos/ <- contains package repositories
builtin/ <- pkg repository that comes with Spack
repo.yaml <- descriptor for the builtin repository
packages/ <- directories under here contain packages
cache/ <- saves resources downloaded during installs
lib/ opt/
spack/ spack/ <- packages are installed here
docs/ <- source for this documentation
env/ <- compiler wrappers for build environment
external/ <- external libs included in Spack distro lib/
llnl/ <- some general-use libraries spack/
docs/ <- source for this documentation
env/ <- compiler wrappers for build environment
spack/ <- spack module; contains Python code external/ <- external libs included in Spack distro
cmd/ <- each file in here is a spack subcommand llnl/ <- some general-use libraries
compilers/ <- compiler description files
test/ <- unit test modules spack/ <- spack module; contains Python code
util/ <- common code cmd/ <- each file in here is a spack subcommand
compilers/ <- compiler description files
test/ <- unit test modules
util/ <- common code
Spack is designed so that it could live within a `standard UNIX Spack is designed so that it could live within a `standard UNIX
directory hierarchy <http://linux.die.net/man/7/hier>`_, so ``lib``, directory hierarchy <http://linux.die.net/man/7/hier>`_, so ``lib``,
``var``, and ``opt`` all contain a ``spack`` subdirectory in case ``var``, and ``opt`` all contain a ``spack`` subdirectory in case
Spack is installed alongside other software. Most of the interesting Spack is installed alongside other software. Most of the interesting
parts of Spack live in ``lib/spack``. Files under ``var`` are created parts of Spack live in ``lib/spack``.
as needed, so there is no ``var`` directory when you initially clone
Spack from the repository.
Spack has *one* directory layout and there is no install process. Spack has *one* directory layout and there is no install process.
version and the source code. Most Python programs don't look like Most Python programs don't look like this (they use distutils, ``setup.py``,
this (they use distutils, ``setup.py``, etc.) but we wanted to make etc.) but we wanted to make Spack *very* easy to use. The simple layout
Spack *very* easy to use. The simple layout spares users from the spares users from the need to install Spack into a Python environment.
need to install Spack into a Python environment. Many users don't Many users don't have write access to a Python installation, and installing
have write access to a Python installation, and installing an entire an entire new instance of Python to bootstrap Spack would be very complicated.
new instance of Python to bootstrap Spack would be very complicated.
Users should not have to install install a big, complicated package to Users should not have to install install a big, complicated package to
use the thing that's supposed to spare them from the details of big, use the thing that's supposed to spare them from the details of big,
complicated packages. The end result is that Spack works out of the complicated packages. The end result is that Spack works out of the
@ -283,19 +282,18 @@ Spec objects
Package objects Package objects
--------------- ---------------
Most spack commands Most spack commands look something like this:
look something like this:
#. Parse an abstract spec (or specs) from the command line, #. Parse an abstract spec (or specs) from the command line,
#. *Normalize* the spec based on information in package files, #. *Normalize* the spec based on information in package files,
#. *Concretize* the spec according to some customizable policies, #. *Concretize* the spec according to some customizable policies,
#. Instantiate a package based on the spec, and #. Instantiate a package based on the spec, and
#. Call methods (e.g., ``install()``) on the package object. #. Call methods (e.g., ``install()``) on the package object.
The information in Package files is used at all stages in this The information in Package files is used at all stages in this
process. process.
Conceptually, packages are overloaded. They contain Conceptually, packages are overloaded. They contain:
------------- -------------
Stage objects Stage objects

26
lib/spack/docs/features.rst
View File

@ -14,7 +14,7 @@ Simple package installation
Installing the default version of a package is simple. This will install Installing the default version of a package is simple. This will install
the latest version of the ``mpileaks`` package and all of its dependencies: the latest version of the ``mpileaks`` package and all of its dependencies:
.. code-block:: sh .. code-block:: console
$ spack install mpileaks $ spack install mpileaks
@ -26,7 +26,7 @@ Spack allows installation to be customized. Users can specify the
version, build compiler, compile-time options, and cross-compile version, build compiler, compile-time options, and cross-compile
platform, all on the command line. platform, all on the command line.
.. code-block:: sh .. code-block:: console
# Install a particular version by appending @ # Install a particular version by appending @
$ spack install mpileaks@1.1.2 $ spack install mpileaks@1.1.2
@ -59,7 +59,7 @@ customized extensively. Suppose that ``mpileaks`` depends indirectly
on ``libelf`` and ``libdwarf``. Using ``^``, users can add custom on ``libelf`` and ``libdwarf``. Using ``^``, users can add custom
configurations for the dependencies: configurations for the dependencies:
.. code-block:: sh .. code-block:: console
# Install mpileaks and link it with specific versions of libelf and libdwarf # Install mpileaks and link it with specific versions of libelf and libdwarf
$ spack install mpileaks@1.1.2 %gcc@4.7.3 +debug ^libelf@0.8.12 ^libdwarf@20130729+debug $ spack install mpileaks@1.1.2 %gcc@4.7.3 +debug ^libelf@0.8.12 ^libdwarf@20130729+debug
@ -91,7 +91,7 @@ in pure Python.
For example, this command: For example, this command:
.. code-block:: sh .. code-block:: console
$ spack create http://www.mr511.de/software/libelf-0.8.13.tar.gz $ spack create http://www.mr511.de/software/libelf-0.8.13.tar.gz
@ -101,16 +101,26 @@ creates a simple python file:
from spack import * from spack import *
class Libelf(Package): class Libelf(Package):
homepage = "http://www.example.com/" """FIXME: Put a proper description of your package here."""
# FIXME: Add a proper url for your package's homepage here.
homepage = "http://www.example.com"
url = "http://www.mr511.de/software/libelf-0.8.13.tar.gz" url = "http://www.mr511.de/software/libelf-0.8.13.tar.gz"
version('0.8.13', '4136d7b4c04df68b686570afa26988ac') version('0.8.13', '4136d7b4c04df68b686570afa26988ac')
def install(self, prefix): # FIXME: Add dependencies if required.
configure("--prefix=%s" % prefix) # depends_on('foo')
def install(self, spec, prefix):
# FIXME: Modify the configure line to suit your build system here.
configure('--prefix={0}'.format(prefix))
# FIXME: Add logic to build and install here.
make() make()
make("install") make('install')
It doesn't take much python coding to get from there to a working It doesn't take much python coding to get from there to a working
package: package:

11
lib/spack/docs/getting_started.rst
View File

@ -9,7 +9,7 @@ Download
Getting spack is easy. You can clone it from the `github repository Getting spack is easy. You can clone it from the `github repository
<https://github.com/llnl/spack>`_ using this command: <https://github.com/llnl/spack>`_ using this command:
.. code-block:: sh .. code-block:: console
$ git clone https://github.com/llnl/spack.git $ git clone https://github.com/llnl/spack.git
@ -18,7 +18,7 @@ full path to this directory is in the ``SPACK_ROOT`` environment
variable. Add ``$SPACK_ROOT/bin`` to your path and you're ready to variable. Add ``$SPACK_ROOT/bin`` to your path and you're ready to
go: go:
.. code-block:: sh .. code-block:: console
$ export PATH=$SPACK_ROOT/bin:$PATH $ export PATH=$SPACK_ROOT/bin:$PATH
$ spack install libelf $ spack install libelf
@ -26,9 +26,10 @@ go:
For a richer experience, use Spack's `shell support For a richer experience, use Spack's `shell support
<http://software.llnl.gov/spack/basic_usage.html#environment-modules>`_: <http://software.llnl.gov/spack/basic_usage.html#environment-modules>`_:
.. code-block:: sh .. code-block:: console
# For bash users # For bash users
$ export SPACK_ROOT=/path/to/spack
$ . $SPACK_ROOT/share/spack/setup-env.sh $ . $SPACK_ROOT/share/spack/setup-env.sh
# For tcsh or csh users (note you must set SPACK_ROOT) # For tcsh or csh users (note you must set SPACK_ROOT)
@ -48,9 +49,9 @@ You may want to run it out of a prefix other than the git repository
you cloned. The ``spack bootstrap`` command provides this you cloned. The ``spack bootstrap`` command provides this
functionality. To install spack in a new directory, simply type: functionality. To install spack in a new directory, simply type:
.. code-block:: sh .. code-block:: console
$ spack bootstrap /my/favorite/prefix $ spack bootstrap /my/favorite/prefix
This will install a new spack script in ``/my/favorite/prefix/bin``, This will install a new spack script in ``/my/favorite/prefix/bin``,
which you can use just like you would the regular spack script. Each which you can use just like you would the regular spack script. Each

2
lib/spack/docs/index.rst
View File

@ -28,7 +28,7 @@ Get spack from the `github repository
<https://github.com/llnl/spack>`_ and install your first <https://github.com/llnl/spack>`_ and install your first
package: package:
.. code-block:: sh .. code-block:: console
$ git clone https://github.com/llnl/spack.git $ git clone https://github.com/llnl/spack.git
$ cd spack/bin $ cd spack/bin

102
lib/spack/docs/mirrors.rst
View File

@ -11,27 +11,29 @@ mirror is a URL that points to a directory, either on the local
filesystem or on some server, containing tarballs for all of Spack's filesystem or on some server, containing tarballs for all of Spack's
packages. packages.
Here's an example of a mirror's directory structure:: Here's an example of a mirror's directory structure:
mirror/ .. code-block:: none
cmake/
cmake-2.8.10.2.tar.gz mirror/
dyninst/ cmake/
dyninst-8.1.1.tgz cmake-2.8.10.2.tar.gz
dyninst-8.1.2.tgz dyninst/
libdwarf/ dyninst-8.1.1.tgz
libdwarf-20130126.tar.gz dyninst-8.1.2.tgz
libdwarf-20130207.tar.gz libdwarf/
libdwarf-20130729.tar.gz libdwarf-20130126.tar.gz
libelf/ libdwarf-20130207.tar.gz
libelf-0.8.12.tar.gz libdwarf-20130729.tar.gz
libelf-0.8.13.tar.gz libelf/
libunwind/ libelf-0.8.12.tar.gz
libunwind-1.1.tar.gz libelf-0.8.13.tar.gz
mpich/ libunwind/
mpich-3.0.4.tar.gz libunwind-1.1.tar.gz
mvapich2/ mpich/
mvapich2-1.9.tgz mpich-3.0.4.tar.gz
mvapich2/
mvapich2-1.9.tgz
The structure is very simple. There is a top-level directory. The The structure is very simple. There is a top-level directory. The
second level directories are named after packages, and the third level second level directories are named after packages, and the third level
@ -57,21 +59,9 @@ contains tarballs for each package, named after each package.
---------------- ----------------
Mirrors are managed with the ``spack mirror`` command. The help for Mirrors are managed with the ``spack mirror`` command. The help for
``spack mirror`` looks like this:: ``spack mirror`` looks like this:
$ spack mirror -h .. command-output:: spack help mirror
usage: spack mirror [-h] SUBCOMMAND ...
positional arguments:
SUBCOMMAND
create Create a directory to be used as a spack mirror, and fill
it with package archives.
add Add a mirror to Spack.
remove Remove a mirror by name.
list Print out available mirrors to the console.
optional arguments:
-h, --help show this help message and exit
The ``create`` command actually builds a mirror by fetching all of its The ``create`` command actually builds a mirror by fetching all of its
packages from the internet and checksumming them. packages from the internet and checksumming them.
@ -92,7 +82,7 @@ The command will iterate through all of Spack's packages and download
the safe ones into a directory structure like the one above. Here is the safe ones into a directory structure like the one above. Here is
what it looks like: what it looks like:
.. code-block:: bash .. code-block:: console
$ spack mirror create libelf libdwarf $ spack mirror create libelf libdwarf
==> Created new mirror in spack-mirror-2014-06-24 ==> Created new mirror in spack-mirror-2014-06-24
@ -134,9 +124,11 @@ Normally, ``spack mirror create`` downloads all the archives it has
checksums for. If you want to only create a mirror for a subset of checksums for. If you want to only create a mirror for a subset of
packages, you can do that by supplying a list of package specs on the packages, you can do that by supplying a list of package specs on the
command line after ``spack mirror create``. For example, this command line after ``spack mirror create``. For example, this
command:: command:
$ spack mirror create libelf@0.8.12: boost@1.44: .. code-block:: console
$ spack mirror create libelf@0.8.12: boost@1.44:
Will create a mirror for libelf versions greater than or equal to Will create a mirror for libelf versions greater than or equal to
0.8.12 and boost versions greater than or equal to 1.44. 0.8.12 and boost versions greater than or equal to 1.44.
@ -146,7 +138,9 @@ Mirror files
^^^^^^^^^^^^ ^^^^^^^^^^^^
If you have a *very* large number of packages you want to mirror, you If you have a *very* large number of packages you want to mirror, you
can supply a file with specs in it, one per line:: can supply a file with specs in it, one per line:
.. code-block:: console
$ cat specs.txt $ cat specs.txt
libdwarf libdwarf
@ -168,19 +162,21 @@ your site.
Once you have a mirror, you need to let spack know about it. This is Once you have a mirror, you need to let spack know about it. This is
relatively simple. First, figure out the URL for the mirror. If it's relatively simple. First, figure out the URL for the mirror. If it's
a file, you can use a file URL like this one:: a file, you can use a file URL like this one:
file:///Users/gamblin2/spack-mirror-2014-06-24 .. code-block:: none
file:///Users/gamblin2/spack-mirror-2014-06-24
That points to the directory on the local filesystem. If it were on a That points to the directory on the local filesystem. If it were on a
web server, you could use a URL like this one: web server, you could use a URL like this one:
https://example.com/some/web-hosted/directory/spack-mirror-2014-06-24 https://example.com/some/web-hosted/directory/spack-mirror-2014-06-24
Spack will use the URL as the root for all of the packages it fetches. Spack will use the URL as the root for all of the packages it fetches.
You can tell your Spack installation to use that mirror like this: You can tell your Spack installation to use that mirror like this:
.. code-block:: bash .. code-block:: console
$ spack mirror add local_filesystem file:///Users/gamblin2/spack-mirror-2014-06-24 $ spack mirror add local_filesystem file:///Users/gamblin2/spack-mirror-2014-06-24
@ -192,7 +188,9 @@ Each mirror has a name so that you can refer to it again later.
``spack mirror list`` ``spack mirror list``
--------------------- ---------------------
To see all the mirrors Spack knows about, run ``spack mirror list``:: To see all the mirrors Spack knows about, run ``spack mirror list``:
.. code-block:: console
$ spack mirror list $ spack mirror list
local_filesystem file:///Users/gamblin2/spack-mirror-2014-06-24 local_filesystem file:///Users/gamblin2/spack-mirror-2014-06-24
@ -203,7 +201,9 @@ To see all the mirrors Spack knows about, run ``spack mirror list``::
``spack mirror remove`` ``spack mirror remove``
----------------------- -----------------------
To remove a mirror by name:: To remove a mirror by name, run:
.. code-block:: console
$ spack mirror remove local_filesystem $ spack mirror remove local_filesystem
$ spack mirror list $ spack mirror list
@ -213,7 +213,9 @@ To remove a mirror by name::
Mirror precedence Mirror precedence
----------------- -----------------
Adding a mirror really adds a line in ``~/.spack/mirrors.yaml``:: Adding a mirror really adds a line in ``~/.spack/mirrors.yaml``:
.. code-block:: yaml
mirrors: mirrors:
local_filesystem: file:///Users/gamblin2/spack-mirror-2014-06-24 local_filesystem: file:///Users/gamblin2/spack-mirror-2014-06-24
@ -232,12 +234,12 @@ Local Default Cache
Spack caches resources that are downloaded as part of installs. The cache is Spack caches resources that are downloaded as part of installs. The cache is
a valid spack mirror: it uses the same directory structure and naming scheme a valid spack mirror: it uses the same directory structure and naming scheme
as other Spack mirrors (so it can be copied anywhere and referenced with a URL as other Spack mirrors (so it can be copied anywhere and referenced with a URL
like other mirrors). The mirror is maintained locally (within the Spack like other mirrors). The mirror is maintained locally (within the Spack
installation directory) at :file:`var/spack/cache/`. It is always enabled (and installation directory) at :file:`var/spack/cache/`. It is always enabled (and
is always searched first when attempting to retrieve files for an installation) is always searched first when attempting to retrieve files for an installation)
but can be cleared with :ref:`purge <spack-purge>`; the cache directory can also but can be cleared with :ref:`purge <spack-purge>`; the cache directory can also
be deleted manually without issue. be deleted manually without issue.
Caching includes retrieved tarball archives and source control repositories, but Caching includes retrieved tarball archives and source control repositories, but
only resources with an associated digest or commit ID (e.g. a revision number only resources with an associated digest or commit ID (e.g. a revision number
for SVN) will be cached. for SVN) will be cached.

808
lib/spack/docs/packaging_guide.rst
View File

File diff suppressed because it is too large Load Diff