- [x] add `concretize.lp`, `spack.yaml`, etc. to licensed files
- [x] update all licensed files to say 2013-2021 using
`spack license update-copyright-year`
- [x] appease mypy with some additions to package.py that needed
for oneapi.py
* Update and simplify julia package
The current Spack Julia package potentially installs a few julia
packages, with the installation being controlled by variants. There are
a couple of problems with this.
First, Julia handles packages very differently from systems such as R
and Python. Julia requires write access to the repository directories in
order for user installs of packages to work. If spack installs julia
packages then there will be a repository, DEPOT_PATH, in the
installation directory. If spack is used on an individual basis this
would work but would mean that package data is written to the spack
installation directory after installation. If spack is used to provide
packages for end users then user installs of julia packages will fail
due to lack of write access to the repository in the installation
directory. It seems best for spack to just install julia without any
julia packages, and drop the configuration for those packages.
Second, having spack install package as variants seems to be counter to
how spack works for other extendable systems, like R and Python. Julia
should be an extendable package in spack but it is not clear how to make
that work. As pointed out above, installing user packages requires write
access to the julia repositories, including the one in the install
directory. Essentially, a user package installation will try to update
metadata for *all* julia repositories. Furthermore, each of those
repositories, assuming one per package with spack, would need to have
the Project.toml files merged to present the package stack to julia.
Again, it seems best for spack to just install julia itself and not try
to install julia packages, at least at this time. A good discussion on
this can be found at
https://discourse.julialang.org/t/how-does-one-set-up-a-centralized-julia-installation/13922.
This PR does the following:
- adds versions 1.2.0 and 1.3.1
- removes variants that correspond to julia packages
- changes python to build dependency as it seems to only be needed for
LLVM
- the new versions can use Python-3
- removes dependencies for packages
- adds a conflict statement for Intel compiler, with comment
- add a setup_build_environment method to find GCC libraries
- make formatting consistent
- adds JULIA_CPU_TARGET option to correspond with target to help with
running julia image on hardware older than build host
- added intel build options, for when they can be used
- removed code for installing packages
- removed code for julia config that was needed for packages
Note that versions below 0.5.1 fail to build, with or without these
changes. It is not clear why that is.
* Update var/spack/repos/builtin/packages/julia/package.py
Yes, need to use correct grammar even in the midst of numbers and symbols. Good catch!
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* More cleanup of Julia package
This commit does more cleanup and sets more constraints.
- Removed release-0.4 and release-0.5. I am not sure if those are
actually useful but they are quite old and there are released versions
from the same timeframe.
- Remove the binutils variant.
- Made cmake a build dependency for versions >= 1.
- Added git as a dependency for @master.
- Limit curl dependency to released versions.
- Do not use external curl for master. When I checked, using the
external version failed but the internal curl worked.
- Versions <= 0.5.0 need an older version of openssl.
- Set conflicts directive for cxx variant.
- Added conflicts directive for needing +mkl with Intel compiler.
- Removed configuration settings as these prevented julia from working
properly in all cases that I looked at.
* Fix flake8 error
Remove 'import sys' that is no longer used.
* More dependency tweaks
This commit sets further version constraints on dependencies. It really
looks like julia requires its internal dependencies more over time.
- curl only up to 0.5.0
- openssl only up to 0.5.0
- override with system curl up to version 0.5.0
* Fix spec for curl certs
Only depending on curl through 0.5.0.
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
We'd like to use a consistent checksum scheme everywhere so that we can:
a) incorporate archive checksums into our specs and have a
consistent hashing algorithm across all specs.
b) index mirrors with a consistent type of checksum, and not one that
is dependent on how spack packages are written.
- [x] convert existing md5, sha224, sha512, sha1 checksums to sha256
Seamless translation from 'target=<generic>' to either
- target.family == <generic> (in methods)
- 'target=<generic>:' (in directives)
Also updated docs to show ranges in directives.
- remove the old LGPL license headers from all files in Spack
- add SPDX headers to all files
- core and most packages are (Apache-2.0 OR MIT)
- a very small number of remaining packages are LGPL-2.1-only
## Motivation
Python installations are both important and unfortunately inconsistent. Depending on the Python version, OS, and the strength of the Earth's magnetic field when it was installed, the name of the Python executable, directory containing its libraries, library names, and the directory containing its headers can vary drastically.
I originally got into this mess with #3274, where I discovered that Boost could not be built with Python 3 because the executable is called `python3` and we were telling it to use `python`. I got deeper into this mess when I started hacking on #3140, where I discovered just how difficult it is to find the location and name of the Python libraries and headers.
Currently, half of the packages that depend on Python and need to know this information jump through hoops to determine the correct information. The other half are hard-coded to use `python`, `spec['python'].prefix.lib`, and `spec['python'].prefix.include`. Obviously, none of these packages would work for Python 3, and there's no reason to duplicate the effort. The Python package itself should contain all of the information necessary to use it properly. This is in line with the recent work by @alalazo and @davydden with respect to `spec['blas'].libs` and friends.
## Prefix
For most packages in Spack, we assume that the installation directory is `spec['python'].prefix`. This generally works for anything installed with Spack, but gets complicated when we include external packages. Python is a commonly used external package (it needs to be installed just to run Spack). If it was installed with Homebrew, `which python` would return `/usr/local/bin/python`, and most users would erroneously assume that `/usr/local` is the installation directory. If you peruse through #2173, you'll immediately see why this is not the case. Homebrew actually installs Python in `/usr/local/Cellar/python/2.7.12_2` and symlinks the executable to `/usr/local/bin/python`. `PYTHONHOME` (and presumably most things that need to know where Python is installed) needs to be set to the actual installation directory, not `/usr/local`.
Normally I would say, "sounds like user error, make sure to use the real installation directory in your `packages.yaml`". But I think we can make a special case for Python. That's what we decided in #2173 anyway. If we change our minds, I would be more than happy to simplify things.
To solve this problem, I created a `spec['python'].home` attribute that works the same way as `spec['python'].prefix` but queries Python to figure out where it was actually installed. @tgamblin Is there any way to overwrite `spec['python'].prefix`? I think it's currently immutable.
## Command
In general, Python 2 comes with both `python` and `python2` commands, while Python 3 only comes with a `python3` command. But this is up to the OS developers. For example, `/usr/bin/python` on Gentoo is actually Python 3. Worse yet, if someone is using an externally installed Python, all 3 commands may exist in the same directory! Here's what I'm thinking:
If the spec is for Python 3, try searching for the `python3` command.
If the spec is for Python 2, try searching for the `python2` command.
If neither are found, try searching for the `python` command.
## Libraries
Spack installs Python libraries in `spec['python'].prefix.lib`. Except on openSUSE 13, where it installs to `spec['python'].prefix.lib64` (see #2295 and #2253). On my CentOS 6 machine, the Python libraries are installed in `/usr/lib64`. Both need to work.
The libraries themselves change name depending on OS and Python version. For Python 2.7 on macOS, I'm seeing:
```
lib/libpython2.7.dylib
```
For Python 3.6 on CentOS 6, I'm seeing:
```
lib/libpython3.so
lib/libpython3.6m.so.1.0
lib/libpython3.6m.so -> lib/libpython3.6m.so.1.0
```
Notice the `m` after the version number. Yeah, that's a thing.
## Headers
In Python 2.7, I'm seeing:
```
include/python2.7/pyconfig.h
```
In Python 3.6, I'm seeing:
```
include/python3.6m/pyconfig.h
```
It looks like all Python 3 installations have this `m`. Tested with Python 3.2 and 3.6 on macOS and CentOS 6
Spack has really nice support for libraries (`find_libraries` and `LibraryList`), but nothing for headers. Fixed.
* Rename packages
* Upcasing depends_on() in packages.
* Downcased extends('r')
* Fixed erroneously changed URL that had slipped through.
* Fixed typo
* Fixed link from documentation into package source code.
* Fixed another doc problem.
* Changed underscores to dashes in package names.
* Added test to enforce lowercase, no-underscore naming convention.
* Fix r-xgboost
* Downcase more instances of 'R' in package auto-creation.
* Fix test.
* Converted unit test packages to use dashes not underscores
* Downcase `r` in the docs.
* Update module_file_support.rst
Fix r->R for class R.
- This moves var/spack/packages to var/spack/repos/builtin/packages.
- Packages that did not exist in the source branch, or were changed in
develop, were moved into var/spack/repos/builtin/packages as part of
the integration.
Conflicts:
lib/spack/spack/test/unit_install.py
var/spack/repos/builtin/packages/clang/package.py
