Commit Graph

125 Commits

Author SHA1 Message Date
healther
9b5492ca19 add version 3.6.5 to python (#7643) 2018-03-30 12:20:16 +02:00
Massimiliano Culpo
dffd8f384c
python: added a variant to enable PGO + using --with-icc if compiled with Intel (#7496)
Added a variant to enable default optimizations for production builds.
Uses --with-icc if built with Intel compilers.
2018-03-25 09:10:40 +02:00
Todd Gamblin
54f97d1dec
Update copyright on LLNL files for 2018. (#7592) 2018-03-24 12:13:52 -07:00
Massimiliano Culpo
c14c0d8e74
python: dbm module + version 3.6.4 (#7520)
A build of python@3.X had the following in the logs:
```
The necessary bits to build these optional modules were not found:
_dbm                  _gdbm                 _tkinter
```
As Tkinter is already a variant, we adopt the same strategy for dbm.
2018-03-19 22:19:26 +01:00
Massimiliano Culpo
b7baeb542d Set a default python that is consistent with the libraries in PYTHONHOME (#7157)
fixes #7128

Before this PR packages that were indirectly dependent on python might
have failed due to inconsistency between the python found in the
environment and the standard libraries set in PYTHONHOME
2018-02-11 23:08:45 -08:00
healther
bddc1556e1 symlink python3 to python (#7103)
* symlink python3 to python

* otherwise not all autotools are smart enough

* Update package.py
2018-01-31 16:11:25 -08:00
scheibelp
278901c02c
Handle test dependencies that are python extensions (#6904)
See #6794

This fixes cases where test-only dependencies were omitted from
consideration when modifying the environment at build time. This
includes an update to the python package definition to add
testing-related python extensions to its specialized environment
setup.
2018-01-12 19:02:54 -08:00
健美猫
0d91f51a27 Python: add 3.6.3 . (#6454) 2017-11-25 11:34:24 -07:00
Todd Gamblin
05fa302655
Replace github.com/llnl/spack with github.com/spack/spack (#6142)
We moved to a new GitHub org! Now make the code and docs reflect that.
2017-11-04 17:08:04 -07:00
Oliver Breitwieser
643a28aac5 python: extensions support for easy-install.pth 2017-11-02 18:45:40 -07:00
Todd Gamblin
5449884b2e Fix bare 'except:' to placate errors in new flake8 version.
- fixes E722 errors from latest version of flake8
- requires us to not use 'bare except:' and catch BaseException instead
2017-10-24 10:05:36 +02:00
Todd Gamblin
7dd79094b0 remove wildcards from make spack core and packages
- This removes all wildcard imports EXCEPT `from spack import *` in packages
2017-10-24 10:05:36 +02:00
Mike Pozulp
2ad3a1bff7 Update pic/shared variants for hdf5, python, silo (#5423) 2017-09-21 18:14:31 -07:00
Gregory Lee
34d2762b1f added python version 2.7.14 and 3.6.2 packages (#5389) 2017-09-18 22:05:26 -05:00
Michael Kuhn
84ae7872d3 Update copyright notices for 2017 (#5295) 2017-09-06 17:44:16 -10:00
Todd Gamblin
cac4362f64 Make LICENSE recognizable by GitHub. (#4598) 2017-06-24 22:22:55 -07:00
Milton Woods
7ea0e143c5 python: RPATH on Cray (#4209)
* python: ensure that distutils sets valid compiler options for RPATH on cray

* python: add cray-rpath patches for python@2.3:3.0.1 and python@3.1:

* python: in patch(..., when="@a:b platform=c"), limit b must be specified

* python: assume that python@4 will break the cray patches
2017-05-28 09:17:16 -07:00
Adam J. Stewart
fa90a65d09 Fix spec['python'].home (#4228) 2017-05-18 12:12:08 -05:00
Adam J. Stewart
ce3ab503de Python command, libraries, and headers (#3367)
## Motivation

Python installations are both important and unfortunately inconsistent. Depending on the Python version, OS, and the strength of the Earth's magnetic field when it was installed, the name of the Python executable, directory containing its libraries, library names, and the directory containing its headers can vary drastically. 

I originally got into this mess with #3274, where I discovered that Boost could not be built with Python 3 because the executable is called `python3` and we were telling it to use `python`. I got deeper into this mess when I started hacking on #3140, where I discovered just how difficult it is to find the location and name of the Python libraries and headers.

Currently, half of the packages that depend on Python and need to know this information jump through hoops to determine the correct information. The other half are hard-coded to use `python`, `spec['python'].prefix.lib`, and `spec['python'].prefix.include`. Obviously, none of these packages would work for Python 3, and there's no reason to duplicate the effort. The Python package itself should contain all of the information necessary to use it properly. This is in line with the recent work by @alalazo and @davydden with respect to `spec['blas'].libs` and friends.

## Prefix

For most packages in Spack, we assume that the installation directory is `spec['python'].prefix`. This generally works for anything installed with Spack, but gets complicated when we include external packages. Python is a commonly used external package (it needs to be installed just to run Spack). If it was installed with Homebrew, `which python` would return `/usr/local/bin/python`, and most users would erroneously assume that `/usr/local` is the installation directory. If you peruse through #2173, you'll immediately see why this is not the case. Homebrew actually installs Python in `/usr/local/Cellar/python/2.7.12_2` and symlinks the executable to `/usr/local/bin/python`. `PYTHONHOME` (and presumably most things that need to know where Python is installed) needs to be set to the actual installation directory, not `/usr/local`.

Normally I would say, "sounds like user error, make sure to use the real installation directory in your `packages.yaml`". But I think we can make a special case for Python. That's what we decided in #2173 anyway. If we change our minds, I would be more than happy to simplify things.

To solve this problem, I created a `spec['python'].home` attribute that works the same way as `spec['python'].prefix` but queries Python to figure out where it was actually installed. @tgamblin Is there any way to overwrite `spec['python'].prefix`? I think it's currently immutable.

## Command

In general, Python 2 comes with both `python` and `python2` commands, while Python 3 only comes with a `python3` command. But this is up to the OS developers. For example, `/usr/bin/python` on Gentoo is actually Python 3. Worse yet, if someone is using an externally installed Python, all 3 commands may exist in the same directory! Here's what I'm thinking:

If the spec is for Python 3, try searching for the `python3` command.
If the spec is for Python 2, try searching for the `python2` command.
If neither are found, try searching for the `python` command.

## Libraries

Spack installs Python libraries in `spec['python'].prefix.lib`. Except on openSUSE 13, where it installs to `spec['python'].prefix.lib64` (see #2295 and #2253). On my CentOS 6 machine, the Python libraries are installed in `/usr/lib64`. Both need to work.

The libraries themselves change name depending on OS and Python version. For Python 2.7 on macOS, I'm seeing:
```
lib/libpython2.7.dylib
```
For Python 3.6 on CentOS 6, I'm seeing:
```
lib/libpython3.so
lib/libpython3.6m.so.1.0
lib/libpython3.6m.so -> lib/libpython3.6m.so.1.0
```
Notice the `m` after the version number. Yeah, that's a thing.

## Headers

In Python 2.7, I'm seeing:
```
include/python2.7/pyconfig.h
```
In Python 3.6, I'm seeing:
```
include/python3.6m/pyconfig.h
```
It looks like all Python 3 installations have this `m`. Tested with Python 3.2 and 3.6 on macOS and CentOS 6

Spack has really nice support for libraries (`find_libraries` and `LibraryList`), but nothing for headers. Fixed.
2017-04-29 17:24:13 -07:00
Massimiliano Culpo
e78e87fd4b python: added version 3.6.1 (#4035) 2017-04-28 09:55:02 -05:00
Todd Gamblin
221f179716 Add better tests for web.py; fix some bugs found with spidering.
- _spider in web.py was actually failing to spider deeper than a certain
  point.
- Fixed multiprocessing pools to not use daemons and to allow recursive
  spawning.
- Added detailed tests for spidering and for finding archive versions.
- left some xfail URL finding exercises for the reader.
- Fix noqa annotations for some @when decorators
2017-04-01 15:10:45 -07:00
Todd Gamblin
dead67ae56 Restore newlines to easy-install.pth files. (#3583)
Previous syntax fix in 8a873bb was not quite right.
2017-03-28 09:32:50 -07:00
Gregory Lee
8a873bbd8d Fix easy-install.pth syntax (#3569) 2017-03-27 10:56:55 -07:00
Adam J. Stewart
dca4d2b15e Consistent docs and usage of env mod methods (#3351) 2017-03-14 22:26:44 -07:00
Adam J. Stewart
bde0a67415 Add new package for PaGMO/PyGMO (#3210)
* Add new package for PaGMO/PyGMO

* Rename py-pygmo to pagmo

* Fix concretization bug

* Correct Python detection, use correct flags

* Install PyGMO to the pagmo prefix, not the Python prefix
2017-03-02 18:35:19 -06:00
Gregory Lee
6bc7be9217 updated config file name for Python 3.6 (#2881) 2017-01-23 09:34:50 -08:00
Adam J. Stewart
c0aaa8fcea Add PythonPackage base class
- Add a PythonPackage class with build system support.
  - Support build phases in PythonPackage
  - Add a custom sanity check for PythonPackages
  - Get rid of nolink dependencies in python packages

- Update spack create to use new PythonPackage class

- Port most of Python packages to new PythonPackage class

- Conducted a massive install and activate of Python packages.
  - Fixed bugs introduced by install and activate.

- Update API docs on PythonPackage
2017-01-16 20:00:31 -08:00
Elizabeth Fischer
402dfe30f9 Get Rid of nobuild, nolink, and alldeps (#2765)
* Removing the nobuild, nolink, and alldeps dependency types in favor of being explicit.
* This will help with maintenance going forward, as adding more dependency types won't affect existing declared dependencies in weird ways.
* default deptype is still `('build', 'link')`
2017-01-07 19:59:02 -08:00
Gregory Lee
7b337de93e various python package version and dependency updates (#2751)
* various python package version and dependency updates

* various python package version and dependency updates
2017-01-06 08:50:06 -08:00
Adam J. Stewart
b240fdb2b9 Add latest version of Python 2 (#2668) 2016-12-25 18:53:16 -08:00
Sergey Kosukhin
d8c4418952 Updated python: refactoring and warning messages. 2016-12-13 08:36:56 +01:00
Sergey Kosukhin
3ba88a7509 Updated python: pass LDSHARED to dependants' setup scripts. 2016-12-13 08:36:56 +01:00
Sergey Kosukhin
ec855df071 Updated python: account for lib64 when filtering compilers. 2016-12-13 08:36:56 +01:00
Sergey Kosukhin
58cb2cc2af Updated python: reformat code. 2016-12-13 08:36:56 +01:00
Denis Davydov
a5fb33b0e4 python: symlink lib64/python2.7/lib-dynload/ to lib/python2.7/lib-dynload/ (#2295) 2016-11-16 07:29:08 -07:00
Sergey Kosukhin
62eba4774d Updated python package reflecting the changes in 9347f86 2016-11-01 16:15:34 +01:00
Barry Smith
a714377123 python: PYTHONHOME may not be the same as the prefix of the python executable (#2173)
On MacOS, brew installs /usr/local/bin/python but the Python prefix is not /usr/local/bin
Use the python command sys.exec to get the correct directory, instead of the ad hoc self.prefix
previously used

This was a bear to debug; been driving me nuts since I started using spack.
Since spack passes PYTHONHOME down to package builds in the environment
it was passing PYTHONHOME of /usr/local/bin to the PETSc build that uses Python so
the PETSc Python ./configure errored immediately with

ImportError: No module named site

since python could find no python modules. Todd Gamblin pointed out that my first try to fix
this was wrong since it assumed the spack python was the same python used to run spack. Elizabeth Fischer
suggested how to get it to work also with python3

Funded-by: IDEAS
Project: IDEAS/xSDK
Time:  7 hours
Thanks-to: Todd Gamblin, Elizabeth Fischer
2016-10-30 17:13:16 -07:00
Glenn Johnson
c1284a9046 Stop python from linking with system ncurses (#1672)
Python will look to link with libncursesw in preference to libncurses. Since
ncurses in spack is built without suffixes there is no libncursesw and
python will link to the system libncursesw for _curses.so and
_curses_panel.so, as well as libpanelw for _curses_panel.so.

This PR introduces a patch that simple removes the check for ncursesw in
setup.py and therefore sets `curses_library` to `ncurses`.
2016-09-21 04:02:28 -07:00
Patrick Gartung
e6dc448f18 Compiling python with gnu c++ on macos fails with toolboxglue enabled. 2016-09-20 23:34:15 -07:00
Adam J. Stewart
8a0824df09 Update matplotlibrc filtering for GUI backends 2016-08-11 10:28:35 -05:00
Todd Gamblin
84e331c586 Remove unneeded noqa comments. 2016-08-10 16:33:39 -07:00
Todd Gamblin
240f1fd223 Spack packages now PEP8 compliant. 2016-08-10 16:33:39 -07:00
Joseph Ciurej
ac2e0962ce Improved the build flag construction code in the python install method. 2016-07-22 11:12:02 -07:00
Joseph Ciurej
fb2d2303d6 Fixed a bug that was causing Python installs to be affected by user config. 2016-07-22 11:11:08 -07:00
Adam J. Stewart
583232ea52 Build Python with Tkinter support 2016-07-18 07:51:08 -05:00
becker33
f761130343 Merge pull request #1187 from paulhopkins/bugfix/python-updates
Bugfix/python updates
2016-07-15 14:04:46 -07:00
Ben Boeckel
0ca1a481a3 python: setup the dependent environment properly
We want the run dependencies of all build and run dependencies of the
current package.
2016-07-14 16:21:46 -04:00
Paul Hopkins
49e8e4b590 Add comments explaining UCS4 and prevent variant being used on Python 3.3 and above 2016-07-08 13:47:25 +01:00
Paul Hopkins
a2be05f24b Fix whitespace 2016-07-07 14:00:37 +01:00
Paul Hopkins
719b6bfffd Modify range for ucs4 2.* configure arguments 2016-07-07 13:18:40 +01:00
Paul Hopkins
8c1274bbbb Fix Python configure arguments for ucs4 variant on Python 3.0 and above 2016-07-07 13:12:43 +01:00
Paul Hopkins
fb6d850637 Add more python versions 2016-07-07 13:09:08 +01:00
Paul Hopkins
61fe89bda0 Add ucs4 unicode variant to Python package 2016-07-06 18:29:54 +01:00
Gregory L. Lee
e0fbf14f99 consolidated f2py regex 2016-06-21 09:23:58 -07:00
Gregory L. Lee
4041f45b6c fixed conflicts in python package 2016-06-21 08:47:11 -07:00
Adam J. Stewart
536fa73d5f Python flake8 overhaul 2016-06-17 13:33:09 -05:00
Adam J. Stewart
80d444624b Fix activation of sites.py for sncosmo 2016-06-17 11:08:16 -05:00
Todd Gamblin
e7ced54369 Correct LLNL LGPL license template for clarity. 2016-05-11 21:22:25 -07:00
Glenn Johnson
dfc0efa8ed Put f2py of py-numpy in python ignore list. (#827)
There are many python packages that depend on py-numpy. Each one of
those will have a copy of f2py that will need to be put in the ignore
list for activation. Thise PR adds f2py to the ignore list in the python
package.py file so that it does not have to be done for each package
that depends on py-numpy. This follows the model of what is done for
py-setuptools.

This PR also removes the f2py ignore expression for python packages
that were using it as it is no longer needed in the individual packages.
2016-04-26 10:32:14 -07:00
alalazo
758a9c9096 python extensions : create PYTHONPATH in module files 2016-03-24 10:57:46 +01:00
Todd Gamblin
c4134ee71e Fix #608: broken numpy build.
- Failed to catch all instances of modify_module when it was renamed
  to setup_dependent_package.

- Refactored remaining modify_module calls.

- Also modified Python's setup_dependent_package slightly: only
  creates empty site-packages directory for Python extensions now, not
  for all dependents.
2016-03-23 00:36:32 -07:00
Todd Gamblin
b1516f64eb Rename some environment methods to be less repetitive, add set_path. 2016-03-21 16:01:40 -07:00
Todd Gamblin
439d47b4e4 Refactor environment setup.
- Gave setup_environment and setup_dependent_environment more similar
  signatures. They now allows editing the Spack env and the runtime
  env for *this* package and dependents, respectively.

- modify_module renamed to setup_dependent_python_module for symmetry
  with setup_dependent_environment and to avoid confusion with
  environment modules.

- removed need for patching Package objects at runtime.

- adjust packages to reflect these changes.
2016-03-21 16:01:35 -07:00
alalazo
ec8cc2b528 PYTHONPATH : python patches methods for its extensions 2016-03-18 14:40:53 +01:00
alalazo
e558809043 python : fixed typo 2016-03-17 19:32:31 +01:00
alalazo
f0f0663d1b package : split environment_modifications into setup_environment and setup_dependent_environment.
package : renamed `module_modifications` to `modify_module` for consistency
2016-03-17 15:11:39 +01:00
alalazo
597727f8be tclmodules : added hooks to process EnvironmentModifications objects 2016-03-16 15:19:13 +01:00
alalazo
b45ec3f04e environment : simplified modification of the environment 2016-03-16 10:55:28 +01:00
alalazo
572cb93bf8 package : renamed setup_dependent_environment to module_modifications 2016-03-15 14:05:30 +01:00
alalazo
c85888eb57 package : added environment_modifications 2016-03-15 13:36:41 +01:00
Todd Gamblin
1fe196f95c whitespace and formatting 2016-03-10 01:29:55 -08:00
Gregory L. Lee
19d10291bf modify compiler commands in python config files, fix for #431 2016-02-18 15:45:29 -08:00
Todd Gamblin
8d6342c53d Merge branch 'mplegendre-multi_pkgsrc_roots' into develop
- This moves var/spack/packages to var/spack/repos/builtin/packages.

- Packages that did not exist in the source branch, or were changed in
  develop, were moved into var/spack/repos/builtin/packages as part of
  the integration.

Conflicts:
	lib/spack/spack/test/unit_install.py
	var/spack/repos/builtin/packages/clang/package.py
2016-01-19 01:16:08 -08:00
Todd Gamblin
c65fd3a289 Merge branch 'develop' into mplegendre-multi_pkgsrc_roots
Conflicts:
	lib/spack/spack/cmd/create.py
	lib/spack/spack/cmd/extensions.py
	lib/spack/spack/cmd/fetch.py
	lib/spack/spack/cmd/uninstall.py
	lib/spack/spack/config.py
	lib/spack/spack/database.py
	lib/spack/spack/directory_layout.py
	lib/spack/spack/packages.py
	lib/spack/spack/spec.py
2015-12-25 16:35:55 -08:00
Todd Gamblin
89d5127900 New, cleaner package repository structure.
Package repositories now look like this:

    top-level-dir/
        repo.yaml
        packages/
            libelf/
                package.py
            mpich/
                package.py
            ...

This leaves room at the top level for additional metadata, source,
per-repo configs, indexes, etc., and it makes it easy to see that
something is a spack repo (just look for repo.yaml and packages).
2015-11-26 14:19:27 -08:00