Commit Graph

19 Commits

Author SHA1 Message Date
Todd Gamblin
5425a3dad4 flake8: fix flake8 issues with invalid escapes
- in many files, regular strings were used in places where raw strings
  should've been used.

- convert these to raw strings and get rid of new flake8 errors
2019-01-01 00:44:28 -08:00
Todd Gamblin
6f50cd52ed copyright: update license headers for 2013-2019 copyright. 2019-01-01 00:44:28 -08:00
Matt Belhorn
0f5f49f751 spark: set JAVA_HOME for hadoop CLASSPATH fetching (#9439)
Sets the `JAVA_HOME` in the `hadoop` subprocess called during
`spark.setup_environment`.
2018-10-18 12:45:11 -07:00
Todd Gamblin
eea786f4e8 relicense: replace LGPL headers with Apache-2.0/MIT SPDX headers
- remove the old LGPL license headers from all files in Spack
- add SPDX headers to all files
  - core and most packages are (Apache-2.0 OR MIT)
  - a very small number of remaining packages are LGPL-2.1-only
2018-10-17 14:42:06 -07:00
Adam J. Stewart
be42b8df19 Java updates: new versions, classpath, and JAVA_HOME (#8613)
This PR includes the following changes:

* Added JDK 10
* Changed the JDK version numbers according to the consensus reached
  in #2284
* Added spec['java'].home and spec['java'].libs, similar to #3367
  (JDK and IcedTea)
* Added a check to prevent people from installing JDK on macOS
* Set CLASSPATH for packages depending on Java (JDK and IcedTea)
* Add TODO for extending virtual packages (not currently possible)
* Add TODO for adding Java dependents to views
* Add TODO for packages which extend multiple packages (e.g. Java
  and Python)
2018-08-28 14:35:13 -04:00
Adam J. Stewart
73c978ddd9 install_tree, copy_tree can install into existing directory structures (#8289)
Replace use of `shutil.copytree` with `copy_tree` and `install_tree` functions in `llnl.util.filesystem`.

- `copy_tree` copies without setting permissions.  It should be used to copy files around in the build directory.
- `install_tree` copies files and sets permissions.  It should be used to copy files into the installation directory.
- `install` and `copy` are analogous single-file functions.
- add more extensive tests for these functions
- update packages to use these functions.
2018-08-15 09:30:09 -07:00
Massimiliano Culpo
50122a585a
Added the latest versions of a few packages (#7910)
* automake: added version 1.16.1

* lmod: added version 7.7.29

* git: added version 2.17.0

* tmux: added version 2.7

* scala: added version 2.12.5, fixed typo in version 2.10.6

* Added spark@2.3.0, hadoop@3.1.0, jdk@8u172-b11

* picard: added version 2.18.3

* tar: added version 1.30
2018-04-26 14:46:45 +02:00
Todd Gamblin
54f97d1dec
Update copyright on LLNL files for 2018. (#7592) 2018-03-24 12:13:52 -07:00
Todd Gamblin
05fa302655
Replace github.com/llnl/spack with github.com/spack/spack (#6142)
We moved to a new GitHub org! Now make the code and docs reflect that.
2017-11-04 17:08:04 -07:00
Michael Kuhn
84ae7872d3 Update copyright notices for 2017 (#5295) 2017-09-06 17:44:16 -10:00
snehring
db74f4c3a1 Add a new package for IcedTea (#4804)
* alsa-lib: new package

* cups: new package

* icedtea: new package

* icedtea: adding JAVA_HOME variable

* alsa: removing placeholder

* icedtea: typo fix

* alsa: fixing formattting

* icedtea: formatting fixes and cleanup

* icedtea: additional cleanup

* jdk: adding provides('java') to jdk, setting jdk to default java.

* icedtea: adding jdk as a dependency

* java: changing depends_on('jdk') and similar entries to depends_on('java')

* icedtea: removing unused imports, trying to placate flake8.

* fastqc: fixing flake8 error

* bazel: fixing typo in java dependency

* jdk: changing provides java to a range

* icedtea: cleaning up. Splitting up checksums.

* icedtea: jdk dependency is build only
2017-08-05 14:53:09 -05:00
Todd Gamblin
cac4362f64 Make LICENSE recognizable by GitHub. (#4598) 2017-06-24 22:22:55 -07:00
Adam J. Stewart
e627447417 Prefer vim to vi for default editor (#4230)
* vim > vi
* Allow which to accept multiple args
* Update __init__ to use which with multiple args
* Fix doc tests
2017-06-15 11:27:18 +02:00
Massimiliano Culpo
a3b12a60d1 scala + spark: added legacy versions (#82) (#4461) 2017-06-09 07:15:54 -05:00
Massimiliano Culpo
e4a3295c37 spark: remove whitespaces from run_env variable (#4210) 2017-05-20 23:40:59 -05:00
Adam J. Stewart
ce3ab503de Python command, libraries, and headers (#3367)
## Motivation

Python installations are both important and unfortunately inconsistent. Depending on the Python version, OS, and the strength of the Earth's magnetic field when it was installed, the name of the Python executable, directory containing its libraries, library names, and the directory containing its headers can vary drastically. 

I originally got into this mess with #3274, where I discovered that Boost could not be built with Python 3 because the executable is called `python3` and we were telling it to use `python`. I got deeper into this mess when I started hacking on #3140, where I discovered just how difficult it is to find the location and name of the Python libraries and headers.

Currently, half of the packages that depend on Python and need to know this information jump through hoops to determine the correct information. The other half are hard-coded to use `python`, `spec['python'].prefix.lib`, and `spec['python'].prefix.include`. Obviously, none of these packages would work for Python 3, and there's no reason to duplicate the effort. The Python package itself should contain all of the information necessary to use it properly. This is in line with the recent work by @alalazo and @davydden with respect to `spec['blas'].libs` and friends.

## Prefix

For most packages in Spack, we assume that the installation directory is `spec['python'].prefix`. This generally works for anything installed with Spack, but gets complicated when we include external packages. Python is a commonly used external package (it needs to be installed just to run Spack). If it was installed with Homebrew, `which python` would return `/usr/local/bin/python`, and most users would erroneously assume that `/usr/local` is the installation directory. If you peruse through #2173, you'll immediately see why this is not the case. Homebrew actually installs Python in `/usr/local/Cellar/python/2.7.12_2` and symlinks the executable to `/usr/local/bin/python`. `PYTHONHOME` (and presumably most things that need to know where Python is installed) needs to be set to the actual installation directory, not `/usr/local`.

Normally I would say, "sounds like user error, make sure to use the real installation directory in your `packages.yaml`". But I think we can make a special case for Python. That's what we decided in #2173 anyway. If we change our minds, I would be more than happy to simplify things.

To solve this problem, I created a `spec['python'].home` attribute that works the same way as `spec['python'].prefix` but queries Python to figure out where it was actually installed. @tgamblin Is there any way to overwrite `spec['python'].prefix`? I think it's currently immutable.

## Command

In general, Python 2 comes with both `python` and `python2` commands, while Python 3 only comes with a `python3` command. But this is up to the OS developers. For example, `/usr/bin/python` on Gentoo is actually Python 3. Worse yet, if someone is using an externally installed Python, all 3 commands may exist in the same directory! Here's what I'm thinking:

If the spec is for Python 3, try searching for the `python3` command.
If the spec is for Python 2, try searching for the `python2` command.
If neither are found, try searching for the `python` command.

## Libraries

Spack installs Python libraries in `spec['python'].prefix.lib`. Except on openSUSE 13, where it installs to `spec['python'].prefix.lib64` (see #2295 and #2253). On my CentOS 6 machine, the Python libraries are installed in `/usr/lib64`. Both need to work.

The libraries themselves change name depending on OS and Python version. For Python 2.7 on macOS, I'm seeing:
```
lib/libpython2.7.dylib
```
For Python 3.6 on CentOS 6, I'm seeing:
```
lib/libpython3.so
lib/libpython3.6m.so.1.0
lib/libpython3.6m.so -> lib/libpython3.6m.so.1.0
```
Notice the `m` after the version number. Yeah, that's a thing.

## Headers

In Python 2.7, I'm seeing:
```
include/python2.7/pyconfig.h
```
In Python 3.6, I'm seeing:
```
include/python3.6m/pyconfig.h
```
It looks like all Python 3 installations have this `m`. Tested with Python 3.2 and 3.6 on macOS and CentOS 6

Spack has really nice support for libraries (`find_libraries` and `LibraryList`), but nothing for headers. Fixed.
2017-04-29 17:24:13 -07:00
sknigh
dfff1f5cf4 Changed spark URL, Added version 2.0.2, 2.1.0 (#3027) 2017-02-05 15:56:34 -08:00
Adam J. Stewart
29bac34c1d Ensure that every file in Spack has a license (#2659)
* Ensure that every package has a license

Also fixes URLs with http://http:// doubled.

This is a continuation of #2656.

* Add license to every file in Spack

* Make sure Todd is the author of all packages

* Fix flake8 tests

* Don't license external Sphinx docs

* Don't display licenses in tutorial example packages

Also fixes typos and converts command-line examples
from tcsh to bash, which is more common
2016-12-27 00:17:12 -08:00
Alfredo Adolfo Gimenez
1ea9582218 Added hadoop, spark, and variant spark+hadoop (#1833)
* Added hadoop, spark, and variant spark+hadoop

* Docstrings, dependency types, urls, copyright

* Flake8 fixes, link dependency for hadoop

* Build type for spark, env problem setting JAVA_HOME
2016-10-02 18:49:30 -07:00