Commit Graph

17155 Commits

Author SHA1 Message Date
Greg Becker
19e226259c Features/spack test refactor cmds (#18518)
* no clean -t option, use 'spack test remove'

* refactor commands to make better use of TestSuite objects
2020-09-23 12:44:12 -07:00
Tamara Dahlgren
102b91203a Rename and make escaped text file utility readily available to packages (#18339) 2020-09-23 12:44:09 -07:00
Tamara Dahlgren
04ca718051 Updated hdf smoke test (#18337) 2020-09-23 12:44:06 -07:00
Tamara Dahlgren
61abc75bc6 Add remaining bugfixes and consistency changes from #18210 (#18334) 2020-09-23 12:44:03 -07:00
Greg Becker
86eececc5c Features/spack test refactor (#18277)
* refactor test code into a TestSuite object and install_test module

* update mpi tests

* refactor tests suites to use content hash for name and record reproducibility info

* update unit tests and fix bugs

* Fix tests using data dir for new format
Use new `self.test_stage` object to access current data dir

Co-authored by: Tamara Dahlgren <dahlgren1@llnl.gov>
2020-09-23 12:44:00 -07:00
Tamara Dahlgren
7551b66cd9 Smoke tests: handle package syntax errors (#17919) 2020-09-23 12:43:58 -07:00
Tamara Dahlgren
c9a00562c4 Preliminary HDF smoke tests (#18076) 2020-09-23 12:43:55 -07:00
Gregory Becker
dafda1ab1a update tests to use status=0 over status=None 2020-09-23 12:43:53 -07:00
Greg Becker
d6b9871169 Features/compiler tests (#17353)
* fix setup of run environment for tests

* remove unnecessary 'None' option from run_tests status arg

* allow package files for virtuals

* run tests for all virtuals provided by each package

* add tests for mpi

* add compiler tests for virtual packages

* run compiler tests automatically like virtuals

* use working_dir instead of os.chdir

* Move knowledge of virtual-ness from spec to repo

* refactor test/cmd/clean

* update cmd/pkg tests for correctness
2020-09-23 12:43:50 -07:00
Tamara Dahlgren
b6d1704729 Smoke tests: Preliminary berkeley-db tests (#17899) 2020-09-23 12:43:46 -07:00
Tamara Dahlgren
f49fa74bf7 Smoke test: Add test of a sequence of hdf5 commands (#17686) 2020-09-23 12:43:44 -07:00
Tamara Dahlgren
0b8bc43fb0 smoke test: preliminary sqlite tests 2020-09-23 12:43:40 -07:00
Tamara Dahlgren
70eb1960fb smoke test: Ensure expected test results and options are lists 2020-09-23 12:43:37 -07:00
Tamara Dahlgren
a7c109c3aa Smoke tests: cmake version checks (#17359)
* Smoke tests: cmake version checks

* Simplified cmake install checks: dict-to-list

Co-authored-by: Greg Becker <becker33@llnl.gov>
2020-09-23 12:43:34 -07:00
Tamara Dahlgren
da62f89f4a Smoke tests: hdf5 version checks and check_install (#17360) 2020-09-23 12:43:31 -07:00
Tamara Dahlgren
d9f0170024 Smoke tests: switched warn to debug plus bugfix (#17576) 2020-09-23 12:43:28 -07:00
Tamara Dahlgren
171ebd8189 Features/spack test emacs (#17363)
* Smoke tests: emacs version checks
2020-09-23 12:43:24 -07:00
Tamara Dahlgren
6d986b4478 Smoke tests: Preliminary Umpire install tests (#17178)
* Preliminary install tests for the Umpire package
2020-09-23 12:43:20 -07:00
Tamara Dahlgren
03569dee8d Add install tests for libsigsegv (#17064) 2020-09-23 12:43:18 -07:00
Tamara Dahlgren
e2ddd7846c bugfix: fix cache_extra_test_sources' file copy; add unit tests (#17057) 2020-09-23 12:43:16 -07:00
Gregory Becker
32693fa573 fixup bugs after rebase 2020-09-23 12:43:14 -07:00
Gregory Becker
e221ba6ba7 update macos test for new unit-test command 2020-09-23 12:43:12 -07:00
Gregory Becker
4146c3d135 flake 2020-09-23 12:43:10 -07:00
Gregory Becker
f5b165a76f flake 2020-09-23 12:43:08 -07:00
Tamara Dahlgren
3508362dde smoke tests: grab and run build examples (openmpi) (#16365)
* Snapshot smoke tests that grab and run examples

* Resolved openmpi example test issues for 2.0.0-4.0.3

* Use spec.satisfies; copy extra packages after install (vs. prior to install tests

* Added smoke tests for selected openmpi installed binaries

* Use which() to determine if install exe exists

* Switched onus for installer test source grab from installer to package

* Resolved (local) flake8 issues with package.py

* Use runner.name; use string format for *run_test* messages

* Renamed copy_src_to_install to cache_extra_test_source and added comments

* Metadata path cleanup: added metadata_dir property to and its use in package

* Support list of source paths to cache for install testing (with unit test)

* Added test subdir to install_test_root; changed skip_file to lambda
2020-09-23 12:43:05 -07:00
Tamara Dahlgren
fb145df4f4 bugfix: Resolve perl install test bug (#16501) 2020-09-23 12:43:02 -07:00
Tamara Dahlgren
fd46f67d63 smoke tests: Refined openmpi version checks (#16337)
* Refined openmpi version checks to pass for 2.1.0 through 4.0.3

* Allow skipping install tests with exe not in bin dir and revised openmpi version tests
2020-09-23 12:43:00 -07:00
Greg Becker
edf3e91a12 Add --fail-first and --fail-fast options to spack test run (#16277)
`spack test run --fail-first` exits after the first failed package.

`spack test run --fail-fast` stops each package test after the first
failure.
2020-09-23 12:42:58 -07:00
Gregory Becker
bca630a22a make output comparisons regex 2020-09-23 12:42:56 -07:00
Gregory Becker
5ff9ba320d remove debug log parser from ctest 2020-09-23 12:42:54 -07:00
Gregory Becker
50640d4924 update from Error to FAILED 2020-09-23 12:42:51 -07:00
Gregory Becker
f5cfcadfc5 change test headings from {name}-{hash} to {name}-{version}-{hash} 2020-09-23 12:42:49 -07:00
Gregory Becker
a5c534b86d update bash completion 2020-09-23 12:42:46 -07:00
Gregory Becker
99364b9c3f refactor 2020-09-23 12:42:43 -07:00
Gregory Becker
749ab2e79d Make Spack tests record their errors and continue
previously, tests would fail on the first error
now, we wrap them in a TestFailure object that records all failures
2020-09-23 12:42:40 -07:00
Greg Becker
1b3e1897ca Features/spack test subcommands (#16054)
* spack test: subcommands for asynchronous tests

* commands are `run`, `list`, `status`, `results`, `remove`.
2020-09-23 12:42:36 -07:00
Tamara Dahlgren
3976b2a083 tests: Preliminary libsigsegv smoke tests (updated) (#15981)
* tests: Preliminary libsigsegv smoke tests (updated)

* Cleaned up and added doc to libsigsegv smoke test
2020-09-23 12:42:33 -07:00
Tamara Dahlgren
54603cb91f tests: Update openmpi smoke tests to new run_test api (#15982)
* tests: Update openmpi smoke tests to new run_test api

* Removed version check try-except tracking per discussion

* Changed openmpi orted command status values to list
2020-09-23 12:42:29 -07:00
Tamara Dahlgren
aa630b8d71 install tests: added support for multiple test command status values (#15979)
* install tests: added support for multiple test command status values
2020-09-23 12:42:25 -07:00
Gregory Becker
77acf8ddc2 spack unit-test: fix pytest help command 2020-09-23 12:42:22 -07:00
Gregory Becker
dfb02e6d45 test runner: add options to check installation dir and print purpose 2020-09-23 12:42:18 -07:00
Gregory Becker
cf4a0cbc01 python: use self.command to get exe name in test 2020-09-23 12:42:16 -07:00
Gregory Becker
b0eb02a86f cmd/test.py: fix typo in spdx license header 2020-09-23 12:42:14 -07:00
Gregory Becker
73f76bc1b5 update bash completions 2020-09-23 12:42:12 -07:00
Gregory Becker
6f39d8011e spack test: factor out common args 2020-09-23 12:42:10 -07:00
Gregory Becker
97dc74c727 python: fix tests, remove intentional debug failures 2020-09-23 12:42:08 -07:00
Gregory Becker
d53eefa69f fix docs 2020-09-23 12:42:05 -07:00
Gregory Becker
bae57f2ae8 spack test: update existing docs for moved unit-test cmd 2020-09-23 12:41:23 -07:00
Gregory Becker
ba58ae9118 simplify error handling using language features 2020-09-23 12:36:23 -07:00
Gregory Becker
fdb8a59bae fix get_package_context check whether in a package file 2020-09-23 12:36:22 -07:00