zhangyiss/spack
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #558 from adamjstewart/features/pgi

Browse Source 
Add Licensed Software Support
This commit is contained in:
Todd Gamblin 2016-05-11 17:17:47 -07:00
commit 14fe0b8ad2
7 changed files with 446 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

121
lib/spack/docs/packaging_guide.rst
View File

@ -703,6 +703,127 @@ Fetching a revision
Subversion branches are handled as part of the directory structure, so
you can check out a branch or tag by changing the ``url``.
.. _license:
Licensed software
------------------------------------------
In order to install licensed software, Spack needs to know a few more
details about a package. The following class attributes should be defined.
``license_required``
~~~~~~~~~~~~~~~~~~~~~
Boolean. If set to ``True``, this software requires a license. If set to
``False``, all of the following attributes will be ignored. Defaults to
``False``.
``license_comment``
~~~~~~~~~~~~~~~~~~~~~
String. Contains the symbol used by the license manager to denote a comment.
Defaults to ``#``.
``license_files``
~~~~~~~~~~~~~~~~~~~~~
List of strings. These are files that the software searches for when
looking for a license. All file paths must be relative to the installation
directory. More complex packages like Intel may require multiple
licenses for individual components. Defaults to the empty list.
``license_vars``
~~~~~~~~~~~~~~~~~~~~~
List of strings. Environment variables that can be set to tell the software
where to look for a license if it is not in the usual location. Defaults
to the empty list.
``license_url``
~~~~~~~~~~~~~~~~~~~~~
String. A URL pointing to license setup instructions for the software.
Defaults to the empty string.
For example, let's take a look at the package for the PGI compilers.
.. code-block:: python
# Licensing
license_required = True
license_comment = '#'
license_files = ['license.dat']
license_vars = ['PGROUPD_LICENSE_FILE', 'LM_LICENSE_FILE']
license_url = 'http://www.pgroup.com/doc/pgiinstall.pdf'
As you can see, PGI requires a license. Its license manager, FlexNet, uses
the ``#`` symbol to denote a comment. It expects the license file to be
named ``license.dat`` and to be located directly in the installation prefix.
If you would like the installation file to be located elsewhere, simply set
``PGROUPD_LICENSE_FILE`` or ``LM_LICENSE_FILE`` after installation. For
further instructions on installation and licensing, see the URL provided.
Let's walk through a sample PGI installation to see exactly what Spack is
and isn't capable of. Since PGI does not provide a download URL, it must
be downloaded manually. It can either be added to a mirror or located in
the current directory when ``spack install pgi`` is run. See :ref:`mirrors`
for instructions on setting up a mirror.
After running ``spack install pgi``, the first thing that will happen is
Spack will create a global license file located at
``$SPACK_ROOT/etc/spack/licenses/pgi/license.dat``. It will then open up the
file using the editor set in ``$EDITOR``, or vi if unset. It will look like
this:
.. code-block::
# A license is required to use pgi.
#
# The recommended solution is to store your license key in this global
# license file. After installation, the following symlink(s) will be
# added to point to this file (relative to the installation prefix):
#
# license.dat
#
# Alternatively, use one of the following environment variable(s):
#
# PGROUPD_LICENSE_FILE
# LM_LICENSE_FILE
#
# If you choose to store your license in a non-standard location, you may
# set one of these variable(s) to the full pathname to the license file, or
# port@host if you store your license keys on a dedicated license server.
# You will likely want to set this variable in a module file so that it
# gets loaded every time someone tries to use pgi.
#
# For further information on how to acquire a license, please refer to:
#
# http://www.pgroup.com/doc/pgiinstall.pdf
#
# You may enter your license below.
You can add your license directly to this file, or tell FlexNet to use a
license stored on a separate license server. Here is an example that
points to a license server called licman1:
.. code-block::
SERVER licman1.mcs.anl.gov 00163eb7fba5 27200
USE_SERVER
If your package requires the license to install, you can reference the
location of this global license using ``self.global_license_file``.
After installation, symlinks for all of the files given in
``license_files`` will be created, pointing to this global license.
If you install a different version or variant of the package, Spack
will automatically detect and reuse the already existing global license.
If the software you are trying to package doesn't rely on license files,
Spack will print a warning message, letting the user know that they
need to set an environment variable or pointing them to installation
documentation.
.. _patching:
Patches

136
lib/spack/spack/hooks/licensing.py Normal file
View File

@ -0,0 +1,136 @@
import os
import spack
import llnl.util.tty as tty
from llnl.util.filesystem import join_path
def pre_install(pkg):
"""This hook handles global license setup for licensed software."""
if pkg.license_required:
set_up_license(pkg)
def set_up_license(pkg):
"""Prompt the user, letting them know that a license is required.
For packages that rely on license files, a global license file is
created and opened for editing.
For packages that rely on environment variables to point to a
license, a warning message is printed.
For all other packages, documentation on how to set up a license
is printed."""
# If the license can be stored in a file, create one
if pkg.license_files:
license_path = pkg.global_license_file
if not os.path.exists(license_path):
# Create a new license file
write_license_file(pkg, license_path)
# Open up file in user's favorite $EDITOR for editing
spack.editor(license_path)
tty.msg("Added global license file %s" % license_path)
else:
# Use already existing license file
tty.msg("Found already existing license %s" % license_path)
# If not a file, what about an environment variable?
elif pkg.license_vars:
tty.warn("A license is required to use %s. Please set %s to the "
"full pathname to the license file, or port@host if you"
" store your license keys on a dedicated license server" %
(pkg.name, ' or '.join(pkg.license_vars)))
# If not a file or variable, suggest a website for further info
elif pkg.license_url:
tty.warn("A license is required to use %s. See %s for details" %
(pkg.name, pkg.license_url))
# If all else fails, you're on your own
else:
tty.warn("A license is required to use %s" % pkg.name)
def write_license_file(pkg, license_path):
"""Writes empty license file.
Comments give suggestions on alternative methods of
installing a license."""
comment = pkg.license_comment
# Global license directory may not already exist
if not os.path.exists(os.path.dirname(license_path)):
os.makedirs(os.path.dirname(license_path))
license = open(license_path, 'w')
# License files
license.write("""\
{0} A license is required to use {1}.
{0}
{0} The recommended solution is to store your license key in this global
{0} license file. After installation, the following symlink(s) will be
{0} added to point to this file (relative to the installation prefix):
{0}
""".format(comment, pkg.name))
for filename in pkg.license_files:
license.write("{0}\t{1}\n".format(comment, filename))
license.write("{0}\n".format(comment))
# Environment variables
if pkg.license_vars:
license.write("""\
{0} Alternatively, use one of the following environment variable(s):
{0}
""".format(comment))
for var in pkg.license_vars:
license.write("{0}\t{1}\n".format(comment, var))
license.write("""\
{0}
{0} If you choose to store your license in a non-standard location, you may
{0} set one of these variable(s) to the full pathname to the license file, or
{0} port@host if you store your license keys on a dedicated license server.
{0} You will likely want to set this variable in a module file so that it
{0} gets loaded every time someone tries to use {1}.
{0}
""".format(comment, pkg.name))
# Documentation
if pkg.license_url:
license.write("""\
{0} For further information on how to acquire a license, please refer to:
{0}
{0}\t{1}
{0}
""".format(comment, pkg.license_url))
license.write("""\
{0} You may enter your license below.
""".format(comment))
license.close()
def post_install(pkg):
"""This hook symlinks local licenses to the global license for
licensed software."""
if pkg.license_required:
symlink_license(pkg)
def symlink_license(pkg):
"""Create local symlinks that point to the global license file."""
target = pkg.global_license_file
for filename in pkg.license_files:
link_name = join_path(pkg.prefix, filename)
if os.path.exists(target):
os.symlink(target, link_name)
tty.msg("Added local symlink %s to global license file" %
link_name)

33
lib/spack/spack/package.py
View File

@ -374,6 +374,22 @@ def __init__(self, spec):
if not hasattr(self, 'list_depth'):
self.list_depth = 1
# Set default licensing information
if not hasattr(self, 'license_required'):
self.license_required = False
if not hasattr(self, 'license_comment'):
self.license_comment = '#'
if not hasattr(self, 'license_files'):
self.license_files = []
if not hasattr(self, 'license_vars'):
self.license_vars = []
if not hasattr(self, 'license_url'):
self.license_url = None
# Set up some internal variables for timing.
self._fetch_time = 0.0
self._total_time = 0.0
@ -381,6 +397,16 @@ def __init__(self, spec):
if self.is_extension:
spack.repo.get(self.extendee_spec)._check_extendable()
@property
def global_license_file(self):
"""Returns the path where a global license file should be stored."""
if not self.license_files:
return
spack_root = ancestor(__file__, 4)
global_license_dir = join_path(spack_root, 'etc', 'spack', 'licenses')
return join_path(global_license_dir, self.name,
os.path.basename(self.license_files[0]))
@property
def version(self):
if not self.spec.versions.concrete:
@ -882,6 +908,7 @@ def do_install(self,
def build_process():
"""Forked for each build. Has its own process and python
module space set up by build_environment.fork()."""
start_time = time.time()
if not fake:
if not skip_patch:
@ -1320,11 +1347,9 @@ def fetch_remote_versions(self):
def rpath(self):
"""Get the rpath this package links with, as a list of paths."""
rpaths = [self.prefix.lib, self.prefix.lib64]
rpaths.extend(d.prefix.lib
for d in self.spec.traverse(root=False)
rpaths.extend(d.prefix.lib for d in self.spec.traverse(root=False)
if os.path.isdir(d.prefix.lib))
rpaths.extend(d.prefix.lib64
for d in self.spec.traverse(root=False)
rpaths.extend(d.prefix.lib64 for d in self.spec.traverse(root=False)
if os.path.isdir(d.prefix.lib64))
return rpaths

28
var/spack/repos/builtin/packages/allinea-forge/package.py Normal file
View File

@ -0,0 +1,28 @@
from spack import *
class AllineaForge(Package):
"""Allinea Forge is the complete toolsuite for software development - with
everything needed to debug, profile, optimize, edit and build C, C++ and
Fortran applications on Linux for high performance - from single threads
through to complex parallel HPC codes with MPI, OpenMP, threads or CUDA."""
homepage = "http://www.allinea.com/products/develop-allinea-forge"
version('6.0.4', 'df7f769975048477a36f208d0cd57d7e')
# Licensing
license_required = True
license_comment = '#'
license_files = ['licences/Licence']
license_vars = ['ALLINEA_LICENCE_FILE', 'ALLINEA_LICENSE_FILE']
license_url = 'http://www.allinea.com/user-guide/forge/Installation.html'
def url_for_version(self, version):
# TODO: add support for other architectures/distributions
url = "http://content.allinea.com/downloads/"
return url + "allinea-forge-%s-Redhat-6.0-x86_64.tar" % version
def install(self, spec, prefix):
textinstall = which('textinstall.sh')
textinstall('--accept-licence', prefix)

28
var/spack/repos/builtin/packages/allinea-reports/package.py Normal file
View File

@ -0,0 +1,28 @@
from spack import *
class AllineaReports(Package):
"""Allinea Performance Reports are the most effective way to characterize
and understand the performance of HPC application runs. One single-page
HTML report elegantly answers a range of vital questions for any HPC site
"""
homepage = "http://www.allinea.com/products/allinea-performance-reports"
version('6.0.4', '3f13b08a32682737bc05246fbb2fcc77')
# Licensing
license_required = True
license_comment = '#'
license_files = ['licences/Licence']
license_vars = ['ALLINEA_LICENCE_FILE', 'ALLINEA_LICENSE_FILE']
license_url = 'http://www.allinea.com/user-guide/reports/Installation.html'
def url_for_version(self, version):
# TODO: add support for other architectures/distributions
url = "http://content.allinea.com/downloads/"
return url + "allinea-reports-%s-Redhat-6.0-x86_64.tar" % version
def install(self, spec, prefix):
textinstall = which('textinstall.sh')
textinstall('--accept-licence', prefix)

32
var/spack/repos/builtin/packages/nag/package.py Normal file
View File

@ -0,0 +1,32 @@
from spack import *
import os
class Nag(Package):
"""The NAG Fortran Compiler."""
homepage = "http://www.nag.com/nagware/np.asp"
version('6.1', '1e29d9d435b7ccc2842a320150b28ba4')
version('6.0', '3fa1e7f7b51ef8a23e6c687cdcad9f96')
# Licensing
license_required = True
license_comment = '!'
license_files = ['lib/nag.key']
license_vars = ['NAG_KUSARI_FILE']
license_url = 'http://www.nag.com/doc/inun/np61/lin-mac/klicence.txt'
def url_for_version(self, version):
# TODO: url and checksum are architecture dependent
# TODO: We currently only support x86_64
return 'http://www.nag.com/downloads/impl/npl6a%sna_amd64.tgz' % \
str(version).replace('.', '')
def install(self, spec, prefix):
# Set installation directories
os.environ['INSTALL_TO_BINDIR'] = prefix.bin
os.environ['INSTALL_TO_LIBDIR'] = prefix.lib
os.environ['INSTALL_TO_MANDIR'] = prefix + '/share/man/man'
# Run install script
os.system('./INSTALLU.sh')

72
var/spack/repos/builtin/packages/pgi/package.py Normal file
View File

@ -0,0 +1,72 @@
from spack import *
import os
class Pgi(Package):
"""PGI optimizing multi-core x64 compilers for Linux, MacOS & Windows
with support for debugging and profiling of local MPI processes.
Note: The PGI compilers are licensed software. You will need to create
an account on the PGI homepage and download PGI yourself. Once the download
finishes, rename the file (which may contain information such as the
architecture) to the format: pgi-<version>.tar.gz. Spack will search your
current directory for a file of this format. Alternatively, add this
file to a mirror so that Spack can find it. For instructions on how to
set up a mirror, see http://software.llnl.gov/spack/mirrors.html"""
homepage = "http://www.pgroup.com/"
url = "file://%s/pgi-16.3.tar.gz" % os.getcwd()
version('16.3', '618cb7ddbc57d4e4ed1f21a0ab25f427')
variant('network', default=True,
description="Perform a network install")
variant('single', default=False,
description="Perform a single system install")
variant('nvidia', default=False,
description="Enable installation of optional NVIDIA components")
variant('amd', default=False,
description="Enable installation of optional AMD components")
variant('java', default=False,
description="Enable installation of Java Runtime Environment")
variant('mpi', default=False,
description="Enable installation of Open MPI")
# Licensing
license_required = True
license_comment = '#'
license_files = ['license.dat']
license_vars = ['PGROUPD_LICENSE_FILE', 'LM_LICENSE_FILE']
license_url = 'http://www.pgroup.com/doc/pgiinstall.pdf'
def install(self, spec, prefix):
# Enable the silent installation feature
os.environ['PGI_SILENT'] = "true"
os.environ['PGI_ACCEPT_EULA'] = "accept"
os.environ['PGI_INSTALL_DIR'] = prefix
if '+network' in spec and '~single' in spec:
os.environ['PGI_INSTALL_TYPE'] = "network"
os.environ['PGI_INSTALL_LOCAL_DIR'] = "%s/%s/share_objects" % \
(prefix, self.version)
elif '+single' in spec and '~network' in spec:
os.environ['PGI_INSTALL_TYPE'] = "single"
else:
msg = 'You must choose either a network install or a single '
msg += 'system install.\nYou cannot choose both.'
raise RuntimeError(msg)
if '+nvidia' in spec:
os.environ['PGI_INSTALL_NVIDIA'] = "true"
if '+amd' in spec:
os.environ['PGI_INSTALL_AMD'] = "true"
if '+java' in spec:
os.environ['PGI_INSTALL_JAVA'] = "true"
if '+mpi' in spec:
os.environ['PGI_INSTALL_MPI'] = "true"
# Run install script
os.system("./install")