remove platform=cray (#43796)
Remove support for `cray` as a separate platform. Any platform previously detected as `cray` is now detected as `linux`. Users who still need platform=cray have to stick to Spack 0.22
This commit is contained in:
Massimiliano Culpo
committed by
GitHub
GitHub
parent
67b5f6b838
commit
f242e0fd0c
@@ -1433,22 +1433,12 @@ the reserved keywords ``platform``, ``os`` and ``target``:
|
||||
$ spack install libelf os=ubuntu18.04
|
||||
$ spack install libelf target=broadwell
|
||||
|
||||
or together by using the reserved keyword ``arch``:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack install libelf arch=cray-CNL10-haswell
|
||||
|
||||
Normally users don't have to bother specifying the architecture if they
|
||||
are installing software for their current host, as in that case the
|
||||
values will be detected automatically. If you need fine-grained control
|
||||
over which packages use which targets (or over *all* packages' default
|
||||
target), see :ref:`package-preferences`.
|
||||
|
||||
.. admonition:: Cray machines
|
||||
|
||||
The situation is a little bit different for Cray machines and a detailed
|
||||
explanation on how the architecture can be set on them can be found at :ref:`cray-support`
|
||||
|
||||
.. _support-for-microarchitectures:
|
||||
|
||||
|
||||
@@ -1364,187 +1364,6 @@ This will write the private key to the file `dinosaur.priv`.
|
||||
or for help on an issue or the Spack slack.
|
||||
|
||||
|
||||
.. _cray-support:
|
||||
|
||||
-------------
|
||||
Spack on Cray
|
||||
-------------
|
||||
|
||||
Spack differs slightly when used on a Cray system. The architecture spec
|
||||
can differentiate between the front-end and back-end processor and operating system.
|
||||
For example, on Edison at NERSC, the back-end target processor
|
||||
is "Ivy Bridge", so you can specify to use the back-end this way:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack install zlib target=ivybridge
|
||||
|
||||
You can also use the operating system to build against the back-end:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack install zlib os=CNL10
|
||||
|
||||
Notice that the name includes both the operating system name and the major
|
||||
version number concatenated together.
|
||||
|
||||
Alternatively, if you want to build something for the front-end,
|
||||
you can specify the front-end target processor. The processor for a login node
|
||||
on Edison is "Sandy bridge" so we specify on the command line like so:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack install zlib target=sandybridge
|
||||
|
||||
And the front-end operating system is:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack install zlib os=SuSE11
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Cray compiler detection
|
||||
^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Spack can detect compilers using two methods. For the front-end, we treat
|
||||
everything the same. The difference lies in back-end compiler detection.
|
||||
Back-end compiler detection is made via the Tcl module avail command.
|
||||
Once it detects the compiler it writes the appropriate PrgEnv and compiler
|
||||
module name to compilers.yaml and sets the paths to each compiler with Cray\'s
|
||||
compiler wrapper names (i.e. cc, CC, ftn). During build time, Spack will load
|
||||
the correct PrgEnv and compiler module and will call appropriate wrapper.
|
||||
|
||||
The compilers.yaml config file will also differ. There is a
|
||||
modules section that is filled with the compiler's Programming Environment
|
||||
and module name. On other systems, this field is empty []:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
- compiler:
|
||||
modules:
|
||||
- PrgEnv-intel
|
||||
- intel/15.0.109
|
||||
|
||||
As mentioned earlier, the compiler paths will look different on a Cray system.
|
||||
Since most compilers are invoked using cc, CC and ftn, the paths for each
|
||||
compiler are replaced with their respective Cray compiler wrapper names:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
paths:
|
||||
cc: cc
|
||||
cxx: CC
|
||||
f77: ftn
|
||||
fc: ftn
|
||||
|
||||
As opposed to an explicit path to the compiler executable. This allows Spack
|
||||
to call the Cray compiler wrappers during build time.
|
||||
|
||||
For more on compiler configuration, check out :ref:`compiler-config`.
|
||||
|
||||
Spack sets the default Cray link type to dynamic, to better match other
|
||||
other platforms. Individual packages can enable static linking (which is the
|
||||
default outside of Spack on cray systems) using the ``-static`` flag.
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Setting defaults and using Cray modules
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
If you want to use default compilers for each PrgEnv and also be able
|
||||
to load cray external modules, you will need to set up a ``packages.yaml``.
|
||||
|
||||
Here's an example of an external configuration for cray modules:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
packages:
|
||||
mpich:
|
||||
externals:
|
||||
- spec: "mpich@7.3.1%gcc@5.2.0 arch=cray_xc-haswell-CNL10"
|
||||
modules:
|
||||
- cray-mpich
|
||||
- spec: "mpich@7.3.1%intel@16.0.0.109 arch=cray_xc-haswell-CNL10"
|
||||
modules:
|
||||
- cray-mpich
|
||||
all:
|
||||
providers:
|
||||
mpi: [mpich]
|
||||
|
||||
This tells Spack that for whatever package that depends on mpi, load the
|
||||
cray-mpich module into the environment. You can then be able to use whatever
|
||||
environment variables, libraries, etc, that are brought into the environment
|
||||
via module load.
|
||||
|
||||
.. note::
|
||||
|
||||
For Cray-provided packages, it is best to use ``modules:`` instead of ``prefix:``
|
||||
in ``packages.yaml``, because the Cray Programming Environment heavily relies on
|
||||
modules (e.g., loading the ``cray-mpich`` module adds MPI libraries to the
|
||||
compiler wrapper link line).
|
||||
|
||||
You can set the default compiler that Spack can use for each compiler type.
|
||||
If you want to use the Cray defaults, then set them under ``all:`` in packages.yaml.
|
||||
In the compiler field, set the compiler specs in your order of preference.
|
||||
Whenever you build with that compiler type, Spack will concretize to that version.
|
||||
|
||||
Here is an example of a full packages.yaml used at NERSC
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
packages:
|
||||
mpich:
|
||||
externals:
|
||||
- spec: "mpich@7.3.1%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge"
|
||||
modules:
|
||||
- cray-mpich
|
||||
- spec: "mpich@7.3.1%intel@16.0.0.109 arch=cray_xc-SuSE11-ivybridge"
|
||||
modules:
|
||||
- cray-mpich
|
||||
buildable: False
|
||||
netcdf:
|
||||
externals:
|
||||
- spec: "netcdf@4.3.3.1%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge"
|
||||
modules:
|
||||
- cray-netcdf
|
||||
- spec: "netcdf@4.3.3.1%intel@16.0.0.109 arch=cray_xc-CNL10-ivybridge"
|
||||
modules:
|
||||
- cray-netcdf
|
||||
buildable: False
|
||||
hdf5:
|
||||
externals:
|
||||
- spec: "hdf5@1.8.14%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge"
|
||||
modules:
|
||||
- cray-hdf5
|
||||
- spec: "hdf5@1.8.14%intel@16.0.0.109 arch=cray_xc-CNL10-ivybridge"
|
||||
modules:
|
||||
- cray-hdf5
|
||||
buildable: False
|
||||
all:
|
||||
compiler: [gcc@5.2.0, intel@16.0.0.109]
|
||||
providers:
|
||||
mpi: [mpich]
|
||||
|
||||
Here we tell spack that whenever we want to build with gcc use version 5.2.0 or
|
||||
if we want to build with intel compilers, use version 16.0.0.109. We add a spec
|
||||
for each compiler type for each cray modules. This ensures that for each
|
||||
compiler on our system we can use that external module.
|
||||
|
||||
For more on external packages check out the section :ref:`sec-external-packages`.
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Using Linux containers on Cray machines
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Spack uses environment variables particular to the Cray programming
|
||||
environment to determine which systems are Cray platforms. These
|
||||
environment variables may be propagated into containers that are not
|
||||
using the Cray programming environment.
|
||||
|
||||
To ensure that Spack does not autodetect the Cray programming
|
||||
environment, unset the environment variable ``MODULEPATH``. This
|
||||
will cause Spack to treat a linux container on a Cray system as a base
|
||||
linux distro.
|
||||
|
||||
.. _windows_support:
|
||||
|
||||
----------------
|
||||
|
||||
@@ -91,7 +91,7 @@
|
||||
)
|
||||
from spack.util.executable import Executable
|
||||
from spack.util.log_parse import make_log_context, parse_log_events
|
||||
from spack.util.module_cmd import load_module, module, path_from_modules
|
||||
from spack.util.module_cmd import load_module, path_from_modules
|
||||
|
||||
#
|
||||
# This can be set by the user to globally disable parallel builds.
|
||||
@@ -190,14 +190,6 @@ def __call__(self, *args, **kwargs):
|
||||
return super().__call__(*args, **kwargs)
|
||||
|
||||
|
||||
def _on_cray():
|
||||
host_platform = spack.platforms.host()
|
||||
host_os = host_platform.operating_system("default_os")
|
||||
on_cray = str(host_platform) == "cray"
|
||||
using_cnl = re.match(r"cnl\d+", str(host_os))
|
||||
return on_cray, using_cnl
|
||||
|
||||
|
||||
def clean_environment():
|
||||
# Stuff in here sanitizes the build environment to eliminate
|
||||
# anything the user has set that may interfere. We apply it immediately
|
||||
@@ -241,17 +233,6 @@ def clean_environment():
|
||||
if varname.endswith("_ROOT") and varname != "SPACK_ROOT":
|
||||
env.unset(varname)
|
||||
|
||||
# On Cray "cluster" systems, unset CRAY_LD_LIBRARY_PATH to avoid
|
||||
# interference with Spack dependencies.
|
||||
# CNL requires these variables to be set (or at least some of them,
|
||||
# depending on the CNL version).
|
||||
on_cray, using_cnl = _on_cray()
|
||||
if on_cray and not using_cnl:
|
||||
env.unset("CRAY_LD_LIBRARY_PATH")
|
||||
for varname in os.environ.keys():
|
||||
if "PKGCONF" in varname:
|
||||
env.unset(varname)
|
||||
|
||||
# Unset the following variables because they can affect installation of
|
||||
# Autotools and CMake packages.
|
||||
build_system_vars = [
|
||||
@@ -381,11 +362,7 @@ def set_compiler_environment_variables(pkg, env):
|
||||
_add_werror_handling(keep_werror, env)
|
||||
|
||||
# Set the target parameters that the compiler will add
|
||||
# Don't set on cray platform because the targeting module handles this
|
||||
if spec.satisfies("platform=cray"):
|
||||
isa_arg = ""
|
||||
else:
|
||||
isa_arg = spec.architecture.target.optimization_flags(compiler)
|
||||
isa_arg = spec.architecture.target.optimization_flags(compiler)
|
||||
env.set("SPACK_TARGET_ARGS", isa_arg)
|
||||
|
||||
# Trap spack-tracked compiler flags as appropriate.
|
||||
@@ -833,14 +810,6 @@ def setup_package(pkg, dirty, context: Context = Context.BUILD):
|
||||
for mod in pkg.compiler.modules:
|
||||
load_module(mod)
|
||||
|
||||
# kludge to handle cray mpich and libsci being automatically loaded by
|
||||
# PrgEnv modules on cray platform. Module unload does no damage when
|
||||
# unnecessary
|
||||
on_cray, _ = _on_cray()
|
||||
if on_cray and not dirty:
|
||||
for mod in ["cray-mpich", "cray-libsci"]:
|
||||
module("unload", mod)
|
||||
|
||||
if target and target.module_name:
|
||||
load_module(target.module_name)
|
||||
|
||||
|
||||
@@ -110,9 +110,8 @@ def cuda_flags(arch_list):
|
||||
# From the NVIDIA install guide we know of conflicts for particular
|
||||
# platforms (linux, darwin), architectures (x86, powerpc) and compilers
|
||||
# (gcc, clang). We don't restrict %gcc and %clang conflicts to
|
||||
# platform=linux, since they should also apply to platform=cray, and may
|
||||
# apply to platform=darwin. We currently do not provide conflicts for
|
||||
# platform=darwin with %apple-clang.
|
||||
# platform=linux, since they may apply to platform=darwin. We currently
|
||||
# do not provide conflicts for platform=darwin with %apple-clang.
|
||||
|
||||
# Linux x86_64 compiler conflicts from here:
|
||||
# https://gist.github.com/ax3l/9489132
|
||||
|
||||
@@ -937,32 +937,15 @@ def mpi_setup_dependent_build_environment(self, env, dependent_spec, compilers_o
|
||||
"I_MPI_ROOT": self.normalize_path("mpi"),
|
||||
}
|
||||
|
||||
# CAUTION - SIMILAR code in:
|
||||
# var/spack/repos/builtin/packages/mpich/package.py
|
||||
# var/spack/repos/builtin/packages/openmpi/package.py
|
||||
# var/spack/repos/builtin/packages/mvapich2/package.py
|
||||
#
|
||||
# On Cray, the regular compiler wrappers *are* the MPI wrappers.
|
||||
if "platform=cray" in self.spec:
|
||||
# TODO: Confirm
|
||||
wrapper_vars.update(
|
||||
{
|
||||
"MPICC": compilers_of_client["CC"],
|
||||
"MPICXX": compilers_of_client["CXX"],
|
||||
"MPIF77": compilers_of_client["F77"],
|
||||
"MPIF90": compilers_of_client["F90"],
|
||||
}
|
||||
)
|
||||
else:
|
||||
compiler_wrapper_commands = self.mpi_compiler_wrappers
|
||||
wrapper_vars.update(
|
||||
{
|
||||
"MPICC": compiler_wrapper_commands["MPICC"],
|
||||
"MPICXX": compiler_wrapper_commands["MPICXX"],
|
||||
"MPIF77": compiler_wrapper_commands["MPIF77"],
|
||||
"MPIF90": compiler_wrapper_commands["MPIF90"],
|
||||
}
|
||||
)
|
||||
compiler_wrapper_commands = self.mpi_compiler_wrappers
|
||||
wrapper_vars.update(
|
||||
{
|
||||
"MPICC": compiler_wrapper_commands["MPICC"],
|
||||
"MPICXX": compiler_wrapper_commands["MPICXX"],
|
||||
"MPIF77": compiler_wrapper_commands["MPIF77"],
|
||||
"MPIF90": compiler_wrapper_commands["MPIF90"],
|
||||
}
|
||||
)
|
||||
|
||||
# Ensure that the directory containing the compiler wrappers is in the
|
||||
# PATH. Spack packages add `prefix.bin` to their dependents' paths,
|
||||
|
||||
@@ -24,7 +24,6 @@ class MSBuildPackage(spack.package_base.PackageBase):
|
||||
build_system("msbuild")
|
||||
conflicts("platform=linux", when="build_system=msbuild")
|
||||
conflicts("platform=darwin", when="build_system=msbuild")
|
||||
conflicts("platform=cray", when="build_system=msbuild")
|
||||
|
||||
|
||||
@spack.builder.builder("msbuild")
|
||||
|
||||
@@ -24,7 +24,6 @@ class NMakePackage(spack.package_base.PackageBase):
|
||||
build_system("nmake")
|
||||
conflicts("platform=linux", when="build_system=nmake")
|
||||
conflicts("platform=darwin", when="build_system=nmake")
|
||||
conflicts("platform=cray", when="build_system=nmake")
|
||||
|
||||
|
||||
@spack.builder.builder("nmake")
|
||||
|
||||
@@ -36,9 +36,8 @@ class IntelOneApiPackage(Package):
|
||||
"target=ppc64:",
|
||||
"target=ppc64le:",
|
||||
"target=aarch64:",
|
||||
"platform=darwin:",
|
||||
"platform=cray:",
|
||||
"platform=windows:",
|
||||
"platform=darwin",
|
||||
"platform=windows",
|
||||
]:
|
||||
conflicts(c, msg="This package in only available for x86_64 and Linux")
|
||||
|
||||
|
||||
@@ -50,7 +50,7 @@
|
||||
@B{++}, @r{--}, @r{~~}, @B{==} propagate variants to package dependencies
|
||||
|
||||
architecture variants:
|
||||
@m{platform=platform} linux, darwin, cray, etc.
|
||||
@m{platform=platform} linux, darwin, freebsd, windows
|
||||
@m{os=operating_system} specific <operating_system>
|
||||
@m{target=target} specific <target> processor
|
||||
@m{arch=platform-os-target} shortcut for all three above
|
||||
|
||||
@@ -695,10 +695,6 @@ def compiler_environment(self):
|
||||
try:
|
||||
# load modules and set env variables
|
||||
for module in self.modules:
|
||||
# On cray, mic-knl module cannot be loaded without cce module
|
||||
# See: https://github.com/spack/spack/issues/3153
|
||||
if os.environ.get("CRAY_CPU_TARGET") == "mic-knl":
|
||||
spack.util.module_cmd.load_module("cce")
|
||||
spack.util.module_cmd.load_module(module)
|
||||
|
||||
# apply other compiler environment changes
|
||||
|
||||
@@ -140,7 +140,7 @@ def post_install(spec, explicit=None):
|
||||
return
|
||||
|
||||
# Only enable on platforms using ELF.
|
||||
if not spec.satisfies("platform=linux") and not spec.satisfies("platform=cray"):
|
||||
if not spec.satisfies("platform=linux"):
|
||||
return
|
||||
|
||||
# Disable this hook when bootstrapping, to avoid recursion.
|
||||
|
||||
@@ -117,7 +117,7 @@ def post_install(spec, explicit=None):
|
||||
return
|
||||
|
||||
# Only enable on platforms using ELF.
|
||||
if not spec.satisfies("platform=linux") and not spec.satisfies("platform=cray"):
|
||||
if not spec.satisfies("platform=linux"):
|
||||
return
|
||||
|
||||
visit_directory_tree(spec.prefix, ElfFilesWithRPathVisitor())
|
||||
|
||||
@@ -3,22 +3,12 @@
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
from ._operating_system import OperatingSystem
|
||||
from .cray_backend import CrayBackend
|
||||
from .cray_frontend import CrayFrontend
|
||||
from .freebsd import FreeBSDOs
|
||||
from .linux_distro import LinuxDistro
|
||||
from .mac_os import MacOs
|
||||
from .windows_os import WindowsOs
|
||||
|
||||
__all__ = [
|
||||
"OperatingSystem",
|
||||
"LinuxDistro",
|
||||
"MacOs",
|
||||
"CrayFrontend",
|
||||
"CrayBackend",
|
||||
"WindowsOs",
|
||||
"FreeBSDOs",
|
||||
]
|
||||
__all__ = ["OperatingSystem", "LinuxDistro", "MacOs", "WindowsOs", "FreeBSDOs"]
|
||||
|
||||
#: List of all the Operating Systems known to Spack
|
||||
operating_systems = [LinuxDistro, MacOs, CrayFrontend, CrayBackend, WindowsOs, FreeBSDOs]
|
||||
operating_systems = [LinuxDistro, MacOs, WindowsOs, FreeBSDOs]
|
||||
|
||||
@@ -1,172 +0,0 @@
|
||||
# Copyright 2013-2024 Lawrence Livermore National Security, LLC and other
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
import os
|
||||
import re
|
||||
|
||||
import llnl.util.tty as tty
|
||||
|
||||
import spack.error
|
||||
import spack.version
|
||||
from spack.util.module_cmd import module
|
||||
|
||||
from .linux_distro import LinuxDistro
|
||||
|
||||
#: Possible locations of the Cray CLE release file,
|
||||
#: which we look at to get the CNL OS version.
|
||||
_cle_release_file = "/etc/opt/cray/release/cle-release"
|
||||
_clerelease_file = "/etc/opt/cray/release/clerelease"
|
||||
|
||||
|
||||
def read_cle_release_file():
|
||||
"""Read the CLE release file and return a dict with its attributes.
|
||||
|
||||
This file is present on newer versions of Cray.
|
||||
|
||||
The release file looks something like this::
|
||||
|
||||
RELEASE=6.0.UP07
|
||||
BUILD=6.0.7424
|
||||
...
|
||||
|
||||
The dictionary we produce looks like this::
|
||||
|
||||
{
|
||||
"RELEASE": "6.0.UP07",
|
||||
"BUILD": "6.0.7424",
|
||||
...
|
||||
}
|
||||
|
||||
Returns:
|
||||
dict: dictionary of release attributes
|
||||
"""
|
||||
with open(_cle_release_file) as release_file:
|
||||
result = {}
|
||||
for line in release_file:
|
||||
# use partition instead of split() to ensure we only split on
|
||||
# the first '=' in the line.
|
||||
key, _, value = line.partition("=")
|
||||
result[key] = value.strip()
|
||||
return result
|
||||
|
||||
|
||||
def read_clerelease_file():
|
||||
"""Read the CLE release file and return the Cray OS version.
|
||||
|
||||
This file is present on older versions of Cray.
|
||||
|
||||
The release file looks something like this::
|
||||
|
||||
5.2.UP04
|
||||
|
||||
Returns:
|
||||
str: the Cray OS version
|
||||
"""
|
||||
with open(_clerelease_file) as release_file:
|
||||
for line in release_file:
|
||||
return line.strip()
|
||||
|
||||
|
||||
class CrayBackend(LinuxDistro):
|
||||
"""Compute Node Linux (CNL) is the operating system used for the Cray XC
|
||||
series super computers. It is a very stripped down version of GNU/Linux.
|
||||
Any compilers found through this operating system will be used with
|
||||
modules. If updated, user must make sure that version and name are
|
||||
updated to indicate that OS has been upgraded (or downgraded)
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
name = "cnl"
|
||||
version = self._detect_crayos_version()
|
||||
if version:
|
||||
# If we found a CrayOS version, we do not want the information
|
||||
# from LinuxDistro. In order to skip the logic from
|
||||
# distro.linux_distribution, while still calling __init__
|
||||
# methods further up the MRO, we skip LinuxDistro in the MRO and
|
||||
# call the OperatingSystem superclass __init__ method
|
||||
super(LinuxDistro, self).__init__(name, version)
|
||||
else:
|
||||
super().__init__()
|
||||
self.modulecmd = module
|
||||
|
||||
def __str__(self):
|
||||
return self.name + str(self.version)
|
||||
|
||||
@classmethod
|
||||
def _detect_crayos_version(cls):
|
||||
if os.path.isfile(_cle_release_file):
|
||||
release_attrs = read_cle_release_file()
|
||||
if "RELEASE" not in release_attrs:
|
||||
# This Cray system uses a base OS not CLE/CNL
|
||||
return None
|
||||
v = spack.version.Version(release_attrs["RELEASE"])
|
||||
return v[0]
|
||||
elif os.path.isfile(_clerelease_file):
|
||||
v = read_clerelease_file()
|
||||
return spack.version.Version(v)[0]
|
||||
else:
|
||||
# Not all Cray systems run CNL on the backend.
|
||||
# Systems running in what Cray calls "cluster" mode run other
|
||||
# linux OSs under the Cray PE.
|
||||
# So if we don't detect any Cray OS version on the system,
|
||||
# we return None. We can't ever be sure we will get a Cray OS
|
||||
# version.
|
||||
# Returning None allows the calling code to test for the value
|
||||
# being "True-ish" rather than requiring a try/except block.
|
||||
return None
|
||||
|
||||
def arguments_to_detect_version_fn(self, paths):
|
||||
import spack.compilers
|
||||
|
||||
command_arguments = []
|
||||
for compiler_name in spack.compilers.supported_compilers():
|
||||
cmp_cls = spack.compilers.class_for_compiler_name(compiler_name)
|
||||
|
||||
# If the compiler doesn't have a corresponding
|
||||
# Programming Environment, skip to the next
|
||||
if cmp_cls.PrgEnv is None:
|
||||
continue
|
||||
|
||||
if cmp_cls.PrgEnv_compiler is None:
|
||||
tty.die("Must supply PrgEnv_compiler with PrgEnv")
|
||||
|
||||
compiler_id = spack.compilers.CompilerID(self, compiler_name, None)
|
||||
detect_version_args = spack.compilers.DetectVersionArgs(
|
||||
id=compiler_id, variation=(None, None), language="cc", path="cc"
|
||||
)
|
||||
command_arguments.append(detect_version_args)
|
||||
return command_arguments
|
||||
|
||||
def detect_version(self, detect_version_args):
|
||||
import spack.compilers
|
||||
|
||||
modulecmd = self.modulecmd
|
||||
compiler_name = detect_version_args.id.compiler_name
|
||||
compiler_cls = spack.compilers.class_for_compiler_name(compiler_name)
|
||||
output = modulecmd("avail", compiler_cls.PrgEnv_compiler)
|
||||
version_regex = r"({0})/([\d\.]+[\d]-?[\w]*)".format(compiler_cls.PrgEnv_compiler)
|
||||
matches = re.findall(version_regex, output)
|
||||
version = tuple(version for _, version in matches if "classic" not in version)
|
||||
compiler_id = detect_version_args.id
|
||||
value = detect_version_args._replace(id=compiler_id._replace(version=version))
|
||||
return value, None
|
||||
|
||||
def make_compilers(self, compiler_id, paths):
|
||||
import spack.spec
|
||||
|
||||
name = compiler_id.compiler_name
|
||||
cmp_cls = spack.compilers.class_for_compiler_name(name)
|
||||
compilers = []
|
||||
for v in compiler_id.version:
|
||||
comp = cmp_cls(
|
||||
spack.spec.CompilerSpec(name + "@=" + v),
|
||||
self,
|
||||
"any",
|
||||
["cc", "CC", "ftn"],
|
||||
[cmp_cls.PrgEnv, name + "/" + v],
|
||||
)
|
||||
|
||||
compilers.append(comp)
|
||||
return compilers
|
||||
@@ -1,105 +0,0 @@
|
||||
# Copyright 2013-2024 Lawrence Livermore National Security, LLC and other
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
import contextlib
|
||||
import os
|
||||
import re
|
||||
|
||||
import llnl.util.filesystem as fs
|
||||
import llnl.util.lang
|
||||
import llnl.util.tty as tty
|
||||
|
||||
from spack.util.environment import get_path
|
||||
from spack.util.module_cmd import module
|
||||
|
||||
from .linux_distro import LinuxDistro
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def unload_programming_environment():
|
||||
"""Context manager that unloads Cray Programming Environments."""
|
||||
env_bu = None
|
||||
|
||||
# We rely on the fact that the PrgEnv-* modules set the PE_ENV
|
||||
# environment variable.
|
||||
if "PE_ENV" in os.environ:
|
||||
# Copy environment variables to restore them after the compiler
|
||||
# detection. We expect that the only thing PrgEnv-* modules do is
|
||||
# the environment variables modifications.
|
||||
env_bu = os.environ.copy()
|
||||
|
||||
# Get the name of the module from the environment variable.
|
||||
prg_env = "PrgEnv-" + os.environ["PE_ENV"].lower()
|
||||
|
||||
# Unload the PrgEnv-* module. By doing this we intentionally
|
||||
# provoke errors when the Cray's compiler wrappers are executed
|
||||
# (Error: A PrgEnv-* modulefile must be loaded.) so they will not
|
||||
# be detected as valid compilers by the overridden method. We also
|
||||
# expect that the modules that add the actual compilers' binaries
|
||||
# into the PATH environment variable (i.e. the following modules:
|
||||
# 'intel', 'cce', 'gcc', etc.) will also be unloaded since they are
|
||||
# specified as prerequisites in the PrgEnv-* modulefiles.
|
||||
module("unload", prg_env)
|
||||
|
||||
yield
|
||||
|
||||
# Restore the environment.
|
||||
if env_bu is not None:
|
||||
os.environ.clear()
|
||||
os.environ.update(env_bu)
|
||||
|
||||
|
||||
class CrayFrontend(LinuxDistro):
|
||||
"""Represents OS that runs on login and service nodes of the Cray platform.
|
||||
It acts as a regular Linux without Cray-specific modules and compiler
|
||||
wrappers."""
|
||||
|
||||
@property
|
||||
def compiler_search_paths(self):
|
||||
"""Calls the default function but unloads Cray's programming
|
||||
environments first.
|
||||
|
||||
This prevents from detecting Cray compiler wrappers and avoids
|
||||
possible false detections.
|
||||
"""
|
||||
import spack.compilers
|
||||
|
||||
with unload_programming_environment():
|
||||
search_paths = get_path("PATH")
|
||||
|
||||
extract_path_re = re.compile(r"prepend-path[\s]*PATH[\s]*([/\w\.:-]*)")
|
||||
|
||||
for compiler_cls in spack.compilers.all_compiler_types():
|
||||
# Check if the compiler class is supported on Cray
|
||||
prg_env = getattr(compiler_cls, "PrgEnv", None)
|
||||
compiler_module = getattr(compiler_cls, "PrgEnv_compiler", None)
|
||||
if not (prg_env and compiler_module):
|
||||
continue
|
||||
|
||||
# It is supported, check which versions are available
|
||||
output = module("avail", compiler_cls.PrgEnv_compiler)
|
||||
version_regex = r"({0})/([\d\.]+[\d]-?[\w]*)".format(compiler_cls.PrgEnv_compiler)
|
||||
matches = re.findall(version_regex, output)
|
||||
versions = tuple(version for _, version in matches if "classic" not in version)
|
||||
|
||||
# Now inspect the modules and add to paths
|
||||
msg = "[CRAY FE] Detected FE compiler [name={0}, versions={1}]"
|
||||
tty.debug(msg.format(compiler_module, versions))
|
||||
for v in versions:
|
||||
try:
|
||||
current_module = compiler_module + "/" + v
|
||||
out = module("show", current_module)
|
||||
match = extract_path_re.search(out)
|
||||
search_paths += match.group(1).split(":")
|
||||
except Exception as e:
|
||||
msg = (
|
||||
"[CRAY FE] An unexpected error occurred while "
|
||||
"detecting FE compiler [compiler={0}, "
|
||||
" version={1}, error={2}]"
|
||||
)
|
||||
tty.debug(msg.format(compiler_cls.name, v, str(e)))
|
||||
|
||||
search_paths = list(llnl.util.lang.dedupe(search_paths))
|
||||
return fs.search_paths_for_executables(*search_paths)
|
||||
@@ -6,7 +6,6 @@
|
||||
|
||||
from ._functions import _host, by_name, platforms, prevent_cray_detection, reset
|
||||
from ._platform import Platform
|
||||
from .cray import Cray
|
||||
from .darwin import Darwin
|
||||
from .freebsd import FreeBSD
|
||||
from .linux import Linux
|
||||
@@ -15,7 +14,6 @@
|
||||
|
||||
__all__ = [
|
||||
"Platform",
|
||||
"Cray",
|
||||
"Darwin",
|
||||
"Linux",
|
||||
"FreeBSD",
|
||||
|
||||
@@ -8,7 +8,6 @@
|
||||
|
||||
import spack.util.environment
|
||||
|
||||
from .cray import Cray
|
||||
from .darwin import Darwin
|
||||
from .freebsd import FreeBSD
|
||||
from .linux import Linux
|
||||
@@ -16,7 +15,7 @@
|
||||
from .windows import Windows
|
||||
|
||||
#: List of all the platform classes known to Spack
|
||||
platforms = [Cray, Darwin, Linux, Windows, FreeBSD, Test]
|
||||
platforms = [Darwin, Linux, Windows, FreeBSD, Test]
|
||||
|
||||
|
||||
@llnl.util.lang.memoized
|
||||
|
||||
@@ -2,254 +2,10 @@
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
import os
|
||||
import os.path
|
||||
import platform
|
||||
import re
|
||||
|
||||
import archspec.cpu
|
||||
|
||||
import llnl.util.tty as tty
|
||||
from llnl.util.symlink import readlink
|
||||
|
||||
import spack.target
|
||||
import spack.version
|
||||
from spack.operating_systems.cray_backend import CrayBackend
|
||||
from spack.operating_systems.cray_frontend import CrayFrontend
|
||||
from spack.paths import build_env_path
|
||||
from spack.util.executable import Executable
|
||||
from spack.util.module_cmd import module
|
||||
|
||||
from ._platform import NoPlatformError, Platform
|
||||
|
||||
_craype_name_to_target_name = {
|
||||
"x86-cascadelake": "cascadelake",
|
||||
"x86-naples": "zen",
|
||||
"x86-rome": "zen2",
|
||||
"x86-milan": "zen3",
|
||||
"x86-skylake": "skylake_avx512",
|
||||
"mic-knl": "mic_knl",
|
||||
"interlagos": "bulldozer",
|
||||
"abudhabi": "piledriver",
|
||||
}
|
||||
|
||||
_ex_craype_dir = "/opt/cray/pe/cpe"
|
||||
_xc_craype_dir = "/opt/cray/pe/cdt"
|
||||
|
||||
|
||||
def slingshot_network():
|
||||
return os.path.exists("/opt/cray/pe") and (
|
||||
os.path.exists("/lib64/libcxi.so") or os.path.exists("/usr/lib64/libcxi.so")
|
||||
)
|
||||
|
||||
|
||||
def _target_name_from_craype_target_name(name):
|
||||
return _craype_name_to_target_name.get(name, name)
|
||||
|
||||
|
||||
class Cray(Platform):
|
||||
priority = 10
|
||||
|
||||
def __init__(self):
|
||||
"""Create a Cray system platform.
|
||||
|
||||
Target names should use craype target names but not include the
|
||||
'craype-' prefix. Uses first viable target from:
|
||||
self
|
||||
envars [SPACK_FRONT_END, SPACK_BACK_END]
|
||||
configuration file "targets.yaml" with keys 'front_end', 'back_end'
|
||||
scanning /etc/bash/bashrc.local for back_end only
|
||||
"""
|
||||
super().__init__("cray")
|
||||
|
||||
# Make all craype targets available.
|
||||
for target in self._avail_targets():
|
||||
name = _target_name_from_craype_target_name(target)
|
||||
self.add_target(name, spack.target.Target(name, "craype-%s" % target))
|
||||
|
||||
self.back_end = os.environ.get("SPACK_BACK_END", self._default_target_from_env())
|
||||
self.default = self.back_end
|
||||
if self.back_end not in self.targets:
|
||||
# We didn't find a target module for the backend
|
||||
raise NoPlatformError()
|
||||
|
||||
# Setup frontend targets
|
||||
for name in archspec.cpu.TARGETS:
|
||||
if name not in self.targets:
|
||||
self.add_target(name, spack.target.Target(name))
|
||||
self.front_end = os.environ.get("SPACK_FRONT_END", archspec.cpu.host().name)
|
||||
if self.front_end not in self.targets:
|
||||
self.add_target(self.front_end, spack.target.Target(self.front_end))
|
||||
|
||||
front_distro = CrayFrontend()
|
||||
back_distro = CrayBackend()
|
||||
|
||||
self.default_os = str(back_distro)
|
||||
self.back_os = self.default_os
|
||||
self.front_os = str(front_distro)
|
||||
|
||||
self.add_operating_system(self.back_os, back_distro)
|
||||
if self.front_os != self.back_os:
|
||||
self.add_operating_system(self.front_os, front_distro)
|
||||
|
||||
def setup_platform_environment(self, pkg, env):
|
||||
"""Change the linker to default dynamic to be more
|
||||
similar to linux/standard linker behavior
|
||||
"""
|
||||
# Unload these modules to prevent any silent linking or unnecessary
|
||||
# I/O profiling in the case of darshan.
|
||||
modules_to_unload = ["cray-mpich", "darshan", "cray-libsci", "altd"]
|
||||
for mod in modules_to_unload:
|
||||
module("unload", mod)
|
||||
|
||||
env.set("CRAYPE_LINK_TYPE", "dynamic")
|
||||
cray_wrapper_names = os.path.join(build_env_path, "cray")
|
||||
|
||||
if os.path.isdir(cray_wrapper_names):
|
||||
env.prepend_path("PATH", cray_wrapper_names)
|
||||
env.prepend_path("SPACK_ENV_PATH", cray_wrapper_names)
|
||||
|
||||
# Makes spack installed pkg-config work on Crays
|
||||
env.append_path("PKG_CONFIG_PATH", "/usr/lib64/pkgconfig")
|
||||
env.append_path("PKG_CONFIG_PATH", "/usr/local/lib64/pkgconfig")
|
||||
|
||||
# CRAY_LD_LIBRARY_PATH is used at build time by the cray compiler
|
||||
# wrappers to augment LD_LIBRARY_PATH. This is to avoid long load
|
||||
# times at runtime. This behavior is not always respected on cray
|
||||
# "cluster" systems, so we reproduce it here.
|
||||
if os.environ.get("CRAY_LD_LIBRARY_PATH"):
|
||||
env.prepend_path("LD_LIBRARY_PATH", os.environ["CRAY_LD_LIBRARY_PATH"])
|
||||
|
||||
@classmethod
|
||||
def craype_type_and_version(cls):
|
||||
if os.path.isdir(_ex_craype_dir):
|
||||
craype_dir = _ex_craype_dir
|
||||
craype_type = "EX"
|
||||
elif os.path.isdir(_xc_craype_dir):
|
||||
craype_dir = _xc_craype_dir
|
||||
craype_type = "XC"
|
||||
else:
|
||||
return (None, None)
|
||||
|
||||
# Take the default version from known symlink path
|
||||
default_path = os.path.join(craype_dir, "default")
|
||||
if os.path.islink(default_path):
|
||||
version = spack.version.Version(readlink(default_path))
|
||||
return (craype_type, version)
|
||||
|
||||
# If no default version, sort available versions and return latest
|
||||
versions_available = [spack.version.Version(v) for v in os.listdir(craype_dir)]
|
||||
versions_available.sort(reverse=True)
|
||||
if not versions_available:
|
||||
return (craype_type, None)
|
||||
return (craype_type, versions_available[0])
|
||||
|
||||
@classmethod
|
||||
def detect(cls):
|
||||
"""
|
||||
Detect whether this system requires CrayPE module support.
|
||||
|
||||
Systems with newer CrayPE (21.10 for EX systems, future work for CS and
|
||||
XC systems) have compilers and MPI wrappers that can be used directly
|
||||
by path. These systems are considered ``linux`` platforms.
|
||||
|
||||
For systems running an older CrayPE, we detect the Cray platform based
|
||||
on the availability through `module` of the Cray programming
|
||||
environment. If this environment is available, we can use it to find
|
||||
compilers, target modules, etc. If the Cray programming environment is
|
||||
not available via modules, then we will treat it as a standard linux
|
||||
system, as the Cray compiler wrappers and other components of the Cray
|
||||
programming environment are irrelevant without module support.
|
||||
"""
|
||||
if "opt/cray" not in os.environ.get("MODULEPATH", ""):
|
||||
return False
|
||||
|
||||
craype_type, craype_version = cls.craype_type_and_version()
|
||||
if craype_type == "XC":
|
||||
return True
|
||||
if craype_type == "EX" and craype_version < spack.version.Version("21.10"):
|
||||
return True
|
||||
return False
|
||||
|
||||
def _default_target_from_env(self):
|
||||
"""Set and return the default CrayPE target loaded in a clean login
|
||||
session.
|
||||
|
||||
A bash subshell is launched with a wiped environment and the list of
|
||||
loaded modules is parsed for the first acceptable CrayPE target.
|
||||
"""
|
||||
# env -i /bin/bash -lc echo $CRAY_CPU_TARGET 2> /dev/null
|
||||
if getattr(self, "default", None) is None:
|
||||
bash = Executable("/bin/bash")
|
||||
output = bash(
|
||||
"--norc",
|
||||
"--noprofile",
|
||||
"-lc",
|
||||
"echo $CRAY_CPU_TARGET",
|
||||
env={"TERM": os.environ.get("TERM", "")},
|
||||
output=str,
|
||||
error=os.devnull,
|
||||
)
|
||||
|
||||
default_from_module = "".join(output.split()) # rm all whitespace
|
||||
if default_from_module:
|
||||
tty.debug("Found default module:%s" % default_from_module)
|
||||
return default_from_module
|
||||
else:
|
||||
front_end = archspec.cpu.host()
|
||||
# Look for the frontend architecture or closest ancestor
|
||||
# available in cray target modules
|
||||
avail = [_target_name_from_craype_target_name(x) for x in self._avail_targets()]
|
||||
for front_end_possibility in [front_end] + front_end.ancestors:
|
||||
if front_end_possibility.name in avail:
|
||||
tty.debug("using front-end architecture or available ancestor")
|
||||
return front_end_possibility.name
|
||||
else:
|
||||
tty.debug("using platform.machine as default")
|
||||
return platform.machine()
|
||||
|
||||
def _avail_targets(self):
|
||||
"""Return a list of available CrayPE CPU targets."""
|
||||
|
||||
def modules_in_output(output):
|
||||
"""Returns a list of valid modules parsed from modulecmd output"""
|
||||
return [i for i in re.split(r"\s\s+|\n", output)]
|
||||
|
||||
def target_names_from_modules(modules):
|
||||
# Craype- module prefixes that are not valid CPU targets.
|
||||
targets = []
|
||||
for mod in modules:
|
||||
if "craype-" in mod:
|
||||
name = mod[7:]
|
||||
name = name.split()[0]
|
||||
_n = name.replace("-", "_") # test for mic-knl/mic_knl
|
||||
is_target_name = name in archspec.cpu.TARGETS or _n in archspec.cpu.TARGETS
|
||||
is_cray_target_name = name in _craype_name_to_target_name
|
||||
if is_target_name or is_cray_target_name:
|
||||
targets.append(name)
|
||||
|
||||
return targets
|
||||
|
||||
def modules_from_listdir():
|
||||
craype_default_path = "/opt/cray/pe/craype/default/modulefiles"
|
||||
if os.path.isdir(craype_default_path):
|
||||
return os.listdir(craype_default_path)
|
||||
return []
|
||||
|
||||
if getattr(self, "_craype_targets", None) is None:
|
||||
strategies = [
|
||||
lambda: modules_in_output(module("avail", "-t", "craype-")),
|
||||
modules_from_listdir,
|
||||
]
|
||||
for available_craype_modules in strategies:
|
||||
craype_modules = available_craype_modules()
|
||||
craype_targets = target_names_from_modules(craype_modules)
|
||||
if craype_targets:
|
||||
self._craype_targets = craype_targets
|
||||
break
|
||||
else:
|
||||
# If nothing is found add platform.machine()
|
||||
# to avoid Spack erroring out
|
||||
self._craype_targets = [platform.machine()]
|
||||
|
||||
return self._craype_targets
|
||||
|
||||
@@ -3799,12 +3799,6 @@ class Solver:
|
||||
def __init__(self):
|
||||
self.driver = PyclingoDriver()
|
||||
self.selector = ReusableSpecsSelector(configuration=spack.config.CONFIG)
|
||||
if spack.platforms.host().name == "cray":
|
||||
msg = (
|
||||
"The Cray platform, i.e. 'platform=cray', will be removed in Spack v0.23. "
|
||||
"All Cray machines will be then detected as 'platform=linux'."
|
||||
)
|
||||
warnings.warn(msg)
|
||||
|
||||
@staticmethod
|
||||
def _check_input_and_extract_concrete_specs(specs):
|
||||
|
||||
@@ -2816,9 +2816,7 @@ def _old_concretize(self, tests=False, deprecation_warning=True):
|
||||
|
||||
# Check if we can produce an optimized binary (will throw if
|
||||
# there are declared inconsistencies)
|
||||
# No need on platform=cray because of the targeting modules
|
||||
if not self.satisfies("platform=cray"):
|
||||
self.architecture.target.optimization_flags(self.compiler)
|
||||
self.architecture.target.optimization_flags(self.compiler)
|
||||
|
||||
def _patches_assigned(self):
|
||||
"""Whether patches have been assigned to this spec by the concretizer."""
|
||||
|
||||
@@ -2,16 +2,12 @@
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
import os
|
||||
import platform
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
import archspec.cpu
|
||||
|
||||
import llnl.util.filesystem as fs
|
||||
|
||||
import spack.compilers
|
||||
import spack.concretize
|
||||
import spack.operating_systems
|
||||
@@ -25,9 +21,8 @@ def current_host_platform():
|
||||
"""Return the platform of the current host as detected by the
|
||||
'platform' stdlib package.
|
||||
"""
|
||||
if os.path.exists("/opt/cray/pe"):
|
||||
current_platform = spack.platforms.Cray()
|
||||
elif "Linux" in platform.system():
|
||||
current_platform = None
|
||||
if "Linux" in platform.system():
|
||||
current_platform = spack.platforms.Linux()
|
||||
elif "Darwin" in platform.system():
|
||||
current_platform = spack.platforms.Darwin()
|
||||
@@ -222,28 +217,3 @@ def test_concretize_target_ranges(root_target_range, dep_target_range, result, m
|
||||
with spack.concretize.disable_compiler_existence_check():
|
||||
spec.concretize()
|
||||
assert spec.target == spec["b"].target == result
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"versions,default,expected",
|
||||
[
|
||||
(["21.11", "21.9"], "21.11", False),
|
||||
(["21.11", "21.9"], "21.9", True),
|
||||
(["21.11", "21.9"], None, False),
|
||||
],
|
||||
)
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="Cray does not use windows")
|
||||
def test_cray_platform_detection(versions, default, expected, tmpdir, monkeypatch, working_env):
|
||||
ex_path = str(tmpdir.join("fake_craype_dir"))
|
||||
fs.mkdirp(ex_path)
|
||||
|
||||
with fs.working_dir(ex_path):
|
||||
for version in versions:
|
||||
fs.touch(version)
|
||||
if default:
|
||||
os.symlink(default, "default")
|
||||
|
||||
monkeypatch.setattr(spack.platforms.cray, "_ex_craype_dir", ex_path)
|
||||
os.environ["MODULEPATH"] = "/opt/cray/pe"
|
||||
|
||||
assert spack.platforms.cray.Cray.detect() == expected
|
||||
|
||||
@@ -556,24 +556,6 @@ def test_build_jobs_defaults():
|
||||
)
|
||||
|
||||
|
||||
def test_dirty_disable_module_unload(config, mock_packages, working_env, mock_module_cmd):
|
||||
"""Test that on CRAY platform 'module unload' is not called if the 'dirty'
|
||||
option is on.
|
||||
"""
|
||||
s = spack.spec.Spec("a").concretized()
|
||||
|
||||
# If called with "dirty" we don't unload modules, so no calls to the
|
||||
# `module` function on Cray
|
||||
spack.build_environment.setup_package(s.package, dirty=True)
|
||||
assert not mock_module_cmd.calls
|
||||
|
||||
# If called without "dirty" we unload modules on Cray
|
||||
spack.build_environment.setup_package(s.package, dirty=False)
|
||||
assert mock_module_cmd.calls
|
||||
assert any(("unload", "cray-libsci") == item[0] for item in mock_module_cmd.calls)
|
||||
assert any(("unload", "cray-mpich") == item[0] for item in mock_module_cmd.calls)
|
||||
|
||||
|
||||
class TestModuleMonkeyPatcher:
|
||||
def test_getting_attributes(self, default_mock_concretization):
|
||||
s = default_mock_concretization("libelf")
|
||||
|
||||
@@ -125,18 +125,8 @@ def print_spack_cc(*args):
|
||||
print(os.environ.get("CC", ""))
|
||||
|
||||
|
||||
# `module unload cray-libsci` in test environment causes failure
|
||||
# It does not fail for actual installs
|
||||
# build_environment.py imports module directly, so we monkeypatch it there
|
||||
# rather than in module_cmd
|
||||
def mock_module_noop(*args):
|
||||
pass
|
||||
|
||||
|
||||
def test_dev_build_drop_in(tmpdir, mock_packages, monkeypatch, install_mockery, working_env):
|
||||
monkeypatch.setattr(os, "execvp", print_spack_cc)
|
||||
monkeypatch.setattr(spack.build_environment, "module", mock_module_noop)
|
||||
|
||||
with tmpdir.as_cwd():
|
||||
output = dev_build("-b", "edit", "--drop-in", "sh", "dev-build-test-install@0.0.0")
|
||||
assert "lib/spack/env" in output
|
||||
|
||||
@@ -3,12 +3,8 @@
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
"""Test detection of compiler version"""
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
import llnl.util.filesystem as fs
|
||||
|
||||
import spack.compilers.aocc
|
||||
import spack.compilers.arm
|
||||
import spack.compilers.cce
|
||||
@@ -23,7 +19,6 @@
|
||||
import spack.compilers.xl
|
||||
import spack.compilers.xl_r
|
||||
import spack.util.module_cmd
|
||||
from spack.operating_systems.cray_frontend import CrayFrontend
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
@@ -413,48 +408,6 @@ def test_xl_version_detection(version_str, expected_version):
|
||||
assert version == expected_version
|
||||
|
||||
|
||||
@pytest.mark.not_on_windows("Not supported on Windows (yet)")
|
||||
@pytest.mark.parametrize(
|
||||
"compiler,version",
|
||||
[
|
||||
("gcc", "8.1.0"),
|
||||
("gcc", "1.0.0-foo"),
|
||||
("pgi", "19.1"),
|
||||
("pgi", "19.1a"),
|
||||
("intel", "9.0.0"),
|
||||
("intel", "0.0.0-foobar"),
|
||||
# ('oneapi', '2021.1'),
|
||||
# ('oneapi', '2021.1-foobar')
|
||||
],
|
||||
)
|
||||
def test_cray_frontend_compiler_detection(compiler, version, tmpdir, monkeypatch, working_env):
|
||||
"""Test that the Cray frontend properly finds compilers form modules"""
|
||||
# setup the fake compiler directory
|
||||
compiler_dir = tmpdir.join(compiler)
|
||||
compiler_exe = compiler_dir.join("cc").ensure()
|
||||
fs.set_executable(str(compiler_exe))
|
||||
|
||||
# mock modules
|
||||
def _module(cmd, *args):
|
||||
module_name = "%s/%s" % (compiler, version)
|
||||
module_contents = "prepend-path PATH %s" % compiler_dir
|
||||
if cmd == "avail":
|
||||
return module_name if compiler in args[0] else ""
|
||||
if cmd == "show":
|
||||
return module_contents if module_name in args else ""
|
||||
|
||||
monkeypatch.setattr(spack.operating_systems.cray_frontend, "module", _module)
|
||||
|
||||
# remove PATH variable
|
||||
os.environ.pop("PATH", None)
|
||||
|
||||
# get a CrayFrontend object
|
||||
cray_fe_os = CrayFrontend()
|
||||
|
||||
paths = cray_fe_os.compiler_search_paths
|
||||
assert paths == [str(compiler_dir)]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"version_str,expected_version",
|
||||
[
|
||||
|
||||
@@ -1,77 +0,0 @@
|
||||
# Copyright 2013-2024 Lawrence Livermore National Security, LLC and other
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
import spack.operating_systems.cray_backend as cray_backend
|
||||
|
||||
|
||||
def test_read_cle_release_file(tmpdir, monkeypatch):
|
||||
"""test reading the Cray cle-release file"""
|
||||
cle_release_path = tmpdir.join("cle-release")
|
||||
with cle_release_path.open("w") as f:
|
||||
f.write(
|
||||
"""\
|
||||
RELEASE=6.0.UP07
|
||||
BUILD=6.0.7424
|
||||
DATE=20190611
|
||||
ARCH=noarch
|
||||
NETWORK=ari
|
||||
PATCHSET=35-201906112304
|
||||
DUMMY=foo=bar
|
||||
"""
|
||||
)
|
||||
|
||||
monkeypatch.setattr(cray_backend, "_cle_release_file", str(cle_release_path))
|
||||
attrs = cray_backend.read_cle_release_file()
|
||||
|
||||
assert attrs["RELEASE"] == "6.0.UP07"
|
||||
assert attrs["BUILD"] == "6.0.7424"
|
||||
assert attrs["DATE"] == "20190611"
|
||||
assert attrs["ARCH"] == "noarch"
|
||||
assert attrs["NETWORK"] == "ari"
|
||||
assert attrs["PATCHSET"] == "35-201906112304"
|
||||
assert attrs["DUMMY"] == "foo=bar"
|
||||
|
||||
assert cray_backend.CrayBackend._detect_crayos_version() == 6
|
||||
|
||||
|
||||
def test_read_clerelease_file(tmpdir, monkeypatch):
|
||||
"""test reading the Cray clerelease file"""
|
||||
clerelease_path = tmpdir.join("clerelease")
|
||||
with clerelease_path.open("w") as f:
|
||||
f.write("5.2.UP04\n")
|
||||
|
||||
monkeypatch.setattr(cray_backend, "_clerelease_file", str(clerelease_path))
|
||||
v = cray_backend.read_clerelease_file()
|
||||
|
||||
assert v == "5.2.UP04"
|
||||
|
||||
assert cray_backend.CrayBackend._detect_crayos_version() == 5
|
||||
|
||||
|
||||
def test_cle_release_precedence(tmpdir, monkeypatch):
|
||||
"""test that cle-release file takes precedence over clerelease file."""
|
||||
cle_release_path = tmpdir.join("cle-release")
|
||||
clerelease_path = tmpdir.join("clerelease")
|
||||
|
||||
with cle_release_path.open("w") as f:
|
||||
f.write(
|
||||
"""\
|
||||
RELEASE=6.0.UP07
|
||||
BUILD=6.0.7424
|
||||
DATE=20190611
|
||||
ARCH=noarch
|
||||
NETWORK=ari
|
||||
PATCHSET=35-201906112304
|
||||
DUMMY=foo=bar
|
||||
"""
|
||||
)
|
||||
|
||||
with clerelease_path.open("w") as f:
|
||||
f.write("5.2.UP04\n")
|
||||
|
||||
monkeypatch.setattr(cray_backend, "_clerelease_file", str(clerelease_path))
|
||||
monkeypatch.setattr(cray_backend, "_cle_release_file", str(cle_release_path))
|
||||
|
||||
assert cray_backend.CrayBackend._detect_crayos_version() == 6
|
||||
Reference in New Issue
Block a user