Rename build logs and make names consistent (#11806)
Fixes #11781 * Rename build log to spack-build-log.txt * Rename environment variables file to spack-build-env.txt * The name of the log and env files is now the same during the build and after the build completes * Update packages which referred to the build log/env files * For packages installed before this commit using older names for the build and env files, search for the older names
This commit is contained in:
		 Tamara Dahlgren
					Tamara Dahlgren
				
			
				
					committed by
					
						 Peter Scheibel
						Peter Scheibel
					
				
			
			
				
	
			
			
			 Peter Scheibel
						Peter Scheibel
					
				
			
						parent
						
							d3be42fcca
						
					
				
				
					commit
					e3299e6923
				
			| @@ -347,7 +347,7 @@ we'll notice that this time the installation won't complete: | |||||||
|        11            options.extend([ |        11            options.extend([ | ||||||
|  |  | ||||||
|   See build log for details: |   See build log for details: | ||||||
|     /usr/local/var/spack/stage/arpack-ng-3.5.0-bloz7cqirpdxj33pg7uj32zs5likz2un/arpack-ng-3.5.0/spack-build.out |     /usr/local/var/spack/stage/arpack-ng-3.5.0-bloz7cqirpdxj33pg7uj32zs5likz2un/arpack-ng-3.5.0/spack-build-out.txt | ||||||
|  |  | ||||||
| Unlike ``openblas`` which provides a library named ``libopenblas.so``, | Unlike ``openblas`` which provides a library named ``libopenblas.so``, | ||||||
| ``netlib-lapack`` provides ``liblapack.so``, so it needs to implement | ``netlib-lapack`` provides ``liblapack.so``, so it needs to implement | ||||||
| @@ -459,7 +459,7 @@ Let's look at an example and try to install ``netcdf ^mpich``: | |||||||
|        56                config_args.append('--enable-pnetcdf') |        56                config_args.append('--enable-pnetcdf') | ||||||
|  |  | ||||||
|   See build log for details: |   See build log for details: | ||||||
|     /usr/local/var/spack/stage/netcdf-4.4.1.1-gk2xxhbqijnrdwicawawcll4t3c7dvoj/netcdf-4.4.1.1/spack-build.out |     /usr/local/var/spack/stage/netcdf-4.4.1.1-gk2xxhbqijnrdwicawawcll4t3c7dvoj/netcdf-4.4.1.1/spack-build-out.txt | ||||||
|  |  | ||||||
| We can see from the error that ``netcdf`` needs to know how to link the *high-level interface* | We can see from the error that ``netcdf`` needs to know how to link the *high-level interface* | ||||||
| of ``hdf5``, and thus passes the extra parameter ``hl`` after the request to retrieve it. | of ``hdf5``, and thus passes the extra parameter ``hl`` after the request to retrieve it. | ||||||
|   | |||||||
| @@ -123,7 +123,7 @@ to build this package: | |||||||
|     >> 3    make: *** No targets specified and no makefile found.  Stop. |     >> 3    make: *** No targets specified and no makefile found.  Stop. | ||||||
|  |  | ||||||
|   See build log for details: |   See build log for details: | ||||||
|     /home/ubuntu/packaging/spack/var/spack/stage/mpileaks-1.0-sv75n3u5ev6mljwcezisz3slooozbbxu/mpileaks-1.0/spack-build.out |     /home/ubuntu/packaging/spack/var/spack/stage/mpileaks-1.0-sv75n3u5ev6mljwcezisz3slooozbbxu/mpileaks-1.0/spack-build-out.txt | ||||||
|  |  | ||||||
| This obviously didn't work; we need to fill in the package-specific | This obviously didn't work; we need to fill in the package-specific | ||||||
| information.  Specifically, Spack didn't try to build any of mpileaks' | information.  Specifically, Spack didn't try to build any of mpileaks' | ||||||
| @@ -256,7 +256,7 @@ Now when we try to install this package a lot more happens: | |||||||
|     >> 3    make: *** No targets specified and no makefile found.  Stop. |     >> 3    make: *** No targets specified and no makefile found.  Stop. | ||||||
|  |  | ||||||
|   See build log for details: |   See build log for details: | ||||||
|     /home/ubuntu/packaging/spack/var/spack/stage/mpileaks-1.0-csoikctsalli4cdkkdk377gprkc472rb/mpileaks-1.0/spack-build.out |     /home/ubuntu/packaging/spack/var/spack/stage/mpileaks-1.0-csoikctsalli4cdkkdk377gprkc472rb/mpileaks-1.0/spack-build-out.txt | ||||||
|  |  | ||||||
| Note that this command may take a while to run and produce more output if | Note that this command may take a while to run and produce more output if | ||||||
| you don't have an MPI already installed or configured in Spack. | you don't have an MPI already installed or configured in Spack. | ||||||
| @@ -319,14 +319,15 @@ If we re-run we still get errors: | |||||||
|     >> 31    configure: error: unable to locate adept-utils installation |     >> 31    configure: error: unable to locate adept-utils installation | ||||||
|  |  | ||||||
|   See build log for details: |   See build log for details: | ||||||
|     /home/ubuntu/packaging/spack/var/spack/stage/mpileaks-1.0-csoikctsalli4cdkkdk377gprkc472rb/mpileaks-1.0/spack-build.out |     /home/ubuntu/packaging/spack/var/spack/stage/mpileaks-1.0-csoikctsalli4cdkkdk377gprkc472rb/mpileaks-1.0/spack-build-out.txt | ||||||
|  |  | ||||||
| Again, the problem may be obvious.  But let's pretend we're not | Again, the problem may be obvious.  But let's pretend we're not | ||||||
| all intelligent developers and use this opportunity spend some | all intelligent developers and use this opportunity spend some | ||||||
| time debugging.  We have a few options that can tell us about | time debugging.  We have a few options that can tell us about | ||||||
| what's going wrong: | what's going wrong: | ||||||
|  |  | ||||||
| As per the error message, Spack has given us a ``spack-build.out`` debug log: | As per the error message, Spack has given us a ``spack-build-out.txt`` debug | ||||||
|  | log: | ||||||
|  |  | ||||||
| .. code-block:: console | .. code-block:: console | ||||||
|  |  | ||||||
|   | |||||||
| @@ -993,7 +993,7 @@ def long_message(self): | |||||||
|  |  | ||||||
|         if self.build_log and os.path.exists(self.build_log): |         if self.build_log and os.path.exists(self.build_log): | ||||||
|             out.write('See build log for details:\n') |             out.write('See build log for details:\n') | ||||||
|             out.write('  %s' % self.build_log) |             out.write('  %s\n' % self.build_log) | ||||||
|  |  | ||||||
|         return out.getvalue() |         return out.getvalue() | ||||||
|  |  | ||||||
|   | |||||||
| @@ -193,8 +193,6 @@ def __init__(self, root, **kwargs): | |||||||
|         self.metadata_dir        = '.spack' |         self.metadata_dir        = '.spack' | ||||||
|         self.spec_file_name      = 'spec.yaml' |         self.spec_file_name      = 'spec.yaml' | ||||||
|         self.extension_file_name = 'extensions.yaml' |         self.extension_file_name = 'extensions.yaml' | ||||||
|         self.build_log_name      = 'build.txt'  # build log. |  | ||||||
|         self.build_env_name      = 'build.env'  # build environment |  | ||||||
|         self.packages_dir        = 'repos'  # archive of package.py files |         self.packages_dir        = 'repos'  # archive of package.py files | ||||||
|  |  | ||||||
|     @property |     @property | ||||||
| @@ -242,12 +240,6 @@ def disable_upstream_check(self): | |||||||
|     def metadata_path(self, spec): |     def metadata_path(self, spec): | ||||||
|         return os.path.join(spec.prefix, self.metadata_dir) |         return os.path.join(spec.prefix, self.metadata_dir) | ||||||
|  |  | ||||||
|     def build_log_path(self, spec): |  | ||||||
|         return os.path.join(self.metadata_path(spec), self.build_log_name) |  | ||||||
|  |  | ||||||
|     def build_env_path(self, spec): |  | ||||||
|         return os.path.join(self.metadata_path(spec), self.build_env_name) |  | ||||||
|  |  | ||||||
|     def build_packages_path(self, spec): |     def build_packages_path(self, spec): | ||||||
|         return os.path.join(self.metadata_path(spec), self.packages_dir) |         return os.path.join(self.metadata_path(spec), self.packages_dir) | ||||||
|  |  | ||||||
|   | |||||||
| @@ -63,6 +63,13 @@ | |||||||
| from spack.package_prefs import get_package_dir_permissions, get_package_group | from spack.package_prefs import get_package_dir_permissions, get_package_group | ||||||
|  |  | ||||||
|  |  | ||||||
|  | # Filename for the Spack build/install log. | ||||||
|  | _spack_build_logfile = 'spack-build-out.txt' | ||||||
|  |  | ||||||
|  | # Filename for the Spack build/install environment file. | ||||||
|  | _spack_build_envfile = 'spack-build-env.txt' | ||||||
|  |  | ||||||
|  |  | ||||||
| class InstallPhase(object): | class InstallPhase(object): | ||||||
|     """Manages a single phase of the installation. |     """Manages a single phase of the installation. | ||||||
|  |  | ||||||
| @@ -432,8 +439,9 @@ class PackageBase(with_metaclass(PackageMeta, PackageViewMixin, object)): | |||||||
|  |  | ||||||
|     #: List of glob expressions. Each expression must either be |     #: List of glob expressions. Each expression must either be | ||||||
|     #: absolute or relative to the package source path. |     #: absolute or relative to the package source path. | ||||||
|     #: Matching artifacts found at the end of the build process will |     #: Matching artifacts found at the end of the build process will be | ||||||
|     #: be copied in the same directory tree as build.env and build.txt. |     #: copied in the same directory tree as _spack_build_logfile and | ||||||
|  |     #: _spack_build_envfile. | ||||||
|     archive_files = [] |     archive_files = [] | ||||||
|  |  | ||||||
|     # |     # | ||||||
| @@ -782,11 +790,55 @@ def stage(self, stage): | |||||||
|  |  | ||||||
|     @property |     @property | ||||||
|     def env_path(self): |     def env_path(self): | ||||||
|         return os.path.join(self.stage.path, 'spack-build.env') |         """Return the build environment file path associated with staging.""" | ||||||
|  |         # Backward compatibility: Return the name of an existing log path; | ||||||
|  |         # otherwise, return the current install env path name. | ||||||
|  |         old_filename = os.path.join(self.stage.path, 'spack-build.env') | ||||||
|  |         if os.path.exists(old_filename): | ||||||
|  |             return old_filename | ||||||
|  |         else: | ||||||
|  |             return os.path.join(self.stage.path, _spack_build_envfile) | ||||||
|  |  | ||||||
|  |     @property | ||||||
|  |     def install_env_path(self): | ||||||
|  |         """ | ||||||
|  |         Return the build environment file path on successful installation. | ||||||
|  |         """ | ||||||
|  |         install_path = spack.store.layout.metadata_path(self.spec) | ||||||
|  |  | ||||||
|  |         # Backward compatibility: Return the name of an existing log path; | ||||||
|  |         # otherwise, return the current install env path name. | ||||||
|  |         old_filename = os.path.join(install_path, 'build.env') | ||||||
|  |         if os.path.exists(old_filename): | ||||||
|  |             return old_filename | ||||||
|  |         else: | ||||||
|  |             return os.path.join(install_path, _spack_build_envfile) | ||||||
|  |  | ||||||
|     @property |     @property | ||||||
|     def log_path(self): |     def log_path(self): | ||||||
|         return os.path.join(self.stage.path, 'spack-build.txt') |         """Return the build log file path associated with staging.""" | ||||||
|  |         # Backward compatibility: Return the name of an existing log path. | ||||||
|  |         for filename in ['spack-build.out', 'spack-build.txt']: | ||||||
|  |             old_log = os.path.join(self.stage.path, filename) | ||||||
|  |             if os.path.exists(old_log): | ||||||
|  |                 return old_log | ||||||
|  |  | ||||||
|  |         # Otherwise, return the current log path name. | ||||||
|  |         return os.path.join(self.stage.path, _spack_build_logfile) | ||||||
|  |  | ||||||
|  |     @property | ||||||
|  |     def install_log_path(self): | ||||||
|  |         """Return the build log file path on successful installation.""" | ||||||
|  |         install_path = spack.store.layout.metadata_path(self.spec) | ||||||
|  |  | ||||||
|  |         # Backward compatibility: Return the name of an existing install log. | ||||||
|  |         for filename in ['build.out', 'build.txt']: | ||||||
|  |             old_log = os.path.join(install_path, filename) | ||||||
|  |             if os.path.exists(old_log): | ||||||
|  |                 return old_log | ||||||
|  |  | ||||||
|  |         # Otherwise, return the current install log path name. | ||||||
|  |         return os.path.join(install_path, _spack_build_logfile) | ||||||
|  |  | ||||||
|     def _make_fetcher(self): |     def _make_fetcher(self): | ||||||
|         # Construct a composite fetcher that always contains at least |         # Construct a composite fetcher that always contains at least | ||||||
| @@ -1394,7 +1446,6 @@ def bootstrap_compiler(self, **kwargs): | |||||||
|             ) |             ) | ||||||
|  |  | ||||||
|     def do_install(self, **kwargs): |     def do_install(self, **kwargs): | ||||||
|  |  | ||||||
|         """Called by commands to install a package and its dependencies. |         """Called by commands to install a package and its dependencies. | ||||||
|  |  | ||||||
|         Package implementations should override install() to describe |         Package implementations should override install() to describe | ||||||
| @@ -1617,6 +1668,9 @@ def build_process(): | |||||||
|                 if mode != perms: |                 if mode != perms: | ||||||
|                     os.chmod(self.prefix, perms) |                     os.chmod(self.prefix, perms) | ||||||
|  |  | ||||||
|  |                 # Ensure the metadata path exists as well | ||||||
|  |                 mkdirp(spack.store.layout.metadata_path(self.spec), mode=perms) | ||||||
|  |  | ||||||
|             # Fork a child to do the actual installation |             # Fork a child to do the actual installation | ||||||
|             # we preserve verbosity settings across installs. |             # we preserve verbosity settings across installs. | ||||||
|             PackageBase._verbose = spack.build_environment.fork( |             PackageBase._verbose = spack.build_environment.fork( | ||||||
| @@ -1724,24 +1778,24 @@ def _do_install_pop_kwargs(self, kwargs): | |||||||
|                     .format(self.last_phase, self.name)) |                     .format(self.last_phase, self.name)) | ||||||
|  |  | ||||||
|     def log(self): |     def log(self): | ||||||
|         # Copy provenance into the install directory on success |         """Copy provenance into the install directory on success.""" | ||||||
|         log_install_path = spack.store.layout.build_log_path(self.spec) |  | ||||||
|         env_install_path = spack.store.layout.build_env_path(self.spec) |  | ||||||
|         packages_dir = spack.store.layout.build_packages_path(self.spec) |         packages_dir = spack.store.layout.build_packages_path(self.spec) | ||||||
|  |  | ||||||
|         # Remove first if we're overwriting another build |         # Remove first if we're overwriting another build | ||||||
|         # (can happen with spack setup) |         # (can happen with spack setup) | ||||||
|         try: |         try: | ||||||
|             # log_install_path and env_install_path are inside this |             # log and env install paths are inside this | ||||||
|             shutil.rmtree(packages_dir) |             shutil.rmtree(packages_dir) | ||||||
|         except Exception as e: |         except Exception as e: | ||||||
|             # FIXME : this potentially catches too many things... |             # FIXME : this potentially catches too many things... | ||||||
|             tty.debug(e) |             tty.debug(e) | ||||||
|  |  | ||||||
|         # Archive the whole stdout + stderr for the package |         # Archive the whole stdout + stderr for the package | ||||||
|         install(self.log_path, log_install_path) |         install(self.log_path, self.install_log_path) | ||||||
|  |  | ||||||
|         # Archive the environment used for the build |         # Archive the environment used for the build | ||||||
|         install(self.env_path, env_install_path) |         install(self.env_path, self.install_env_path) | ||||||
|  |  | ||||||
|         # Finally, archive files that are specific to each package |         # Finally, archive files that are specific to each package | ||||||
|         with working_dir(self.stage.path): |         with working_dir(self.stage.path): | ||||||
|             errors = StringIO() |             errors = StringIO() | ||||||
| @@ -1816,10 +1870,12 @@ def check_paths(path_list, filetype, predicate): | |||||||
|  |  | ||||||
|     @property |     @property | ||||||
|     def build_log_path(self): |     def build_log_path(self): | ||||||
|         if self.installed: |         """ | ||||||
|             return spack.store.layout.build_log_path(self.spec) |         Return the expected (or current) build log file path.  The path points | ||||||
|         else: |         to the staging build file until the software is successfully installed, | ||||||
|             return self.log_path |         when it points to the file in the installation directory. | ||||||
|  |         """ | ||||||
|  |         return self.install_log_path if self.installed else self.log_path | ||||||
|  |  | ||||||
|     @classmethod |     @classmethod | ||||||
|     def inject_flags(cls, name, flags): |     def inject_flags(cls, name, flags): | ||||||
|   | |||||||
| @@ -31,6 +31,19 @@ | |||||||
| find       = SpackCommand('find') | find       = SpackCommand('find') | ||||||
|  |  | ||||||
|  |  | ||||||
|  | def check_mpileaks_install(viewdir): | ||||||
|  |     """Check that the expected install directories exist.""" | ||||||
|  |     assert os.path.exists(str(viewdir.join('.spack', 'mpileaks'))) | ||||||
|  |     # Check that dependencies got in too | ||||||
|  |     assert os.path.exists(str(viewdir.join('.spack', 'libdwarf'))) | ||||||
|  |  | ||||||
|  |  | ||||||
|  | def check_viewdir_removal(viewdir): | ||||||
|  |     """Check that the uninstall/removal worked.""" | ||||||
|  |     assert (not os.path.exists(str(viewdir.join('.spack'))) or | ||||||
|  |             os.listdir(str(viewdir.join('.spack'))) == ['projections.yaml']) | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_add(): | def test_add(): | ||||||
|     e = ev.create('test') |     e = ev.create('test') | ||||||
|     e.add('mpileaks') |     e.add('mpileaks') | ||||||
| @@ -606,22 +619,18 @@ def test_uninstall_removes_from_env(mock_stage, mock_fetch, install_mockery): | |||||||
|  |  | ||||||
|  |  | ||||||
| def test_env_updates_view_install( | def test_env_updates_view_install( | ||||||
|     tmpdir, mock_stage, mock_fetch, install_mockery |         tmpdir, mock_stage, mock_fetch, install_mockery): | ||||||
| ): |  | ||||||
|     view_dir = tmpdir.mkdir('view') |     view_dir = tmpdir.mkdir('view') | ||||||
|     env('create', '--with-view=%s' % view_dir, 'test') |     env('create', '--with-view=%s' % view_dir, 'test') | ||||||
|     with ev.read('test'): |     with ev.read('test'): | ||||||
|         add('mpileaks') |         add('mpileaks') | ||||||
|         install('--fake') |         install('--fake') | ||||||
|  |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/mpileaks'))) |     check_mpileaks_install(view_dir) | ||||||
|     # Check that dependencies got in too |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/libdwarf'))) |  | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_env_without_view_install( | def test_env_without_view_install( | ||||||
|     tmpdir, mock_stage, mock_fetch, install_mockery |         tmpdir, mock_stage, mock_fetch, install_mockery): | ||||||
| ): |  | ||||||
|     # Test enabling a view after installing specs |     # Test enabling a view after installing specs | ||||||
|     env('create', '--without-view', 'test') |     env('create', '--without-view', 'test') | ||||||
|  |  | ||||||
| @@ -639,13 +648,11 @@ def test_env_without_view_install( | |||||||
|  |  | ||||||
|     # After enabling the view, the specs should be linked into the environment |     # After enabling the view, the specs should be linked into the environment | ||||||
|     # view dir |     # view dir | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/mpileaks'))) |     check_mpileaks_install(view_dir) | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/libdwarf'))) |  | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_env_config_view_default( | def test_env_config_view_default( | ||||||
|     tmpdir, mock_stage, mock_fetch, install_mockery |         tmpdir, mock_stage, mock_fetch, install_mockery): | ||||||
| ): |  | ||||||
|     # This config doesn't mention whether a view is enabled |     # This config doesn't mention whether a view is enabled | ||||||
|     test_config = """\ |     test_config = """\ | ||||||
| env: | env: | ||||||
| @@ -665,21 +672,17 @@ def test_env_config_view_default( | |||||||
|  |  | ||||||
|  |  | ||||||
| def test_env_updates_view_install_package( | def test_env_updates_view_install_package( | ||||||
|     tmpdir, mock_stage, mock_fetch, install_mockery |         tmpdir, mock_stage, mock_fetch, install_mockery): | ||||||
| ): |  | ||||||
|     view_dir = tmpdir.mkdir('view') |     view_dir = tmpdir.mkdir('view') | ||||||
|     env('create', '--with-view=%s' % view_dir, 'test') |     env('create', '--with-view=%s' % view_dir, 'test') | ||||||
|     with ev.read('test'): |     with ev.read('test'): | ||||||
|         install('--fake', 'mpileaks') |         install('--fake', 'mpileaks') | ||||||
|  |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/mpileaks'))) |     check_mpileaks_install(view_dir) | ||||||
|     # Check that dependencies got in too |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/libdwarf'))) |  | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_env_updates_view_add_concretize( | def test_env_updates_view_add_concretize( | ||||||
|     tmpdir, mock_stage, mock_fetch, install_mockery |         tmpdir, mock_stage, mock_fetch, install_mockery): | ||||||
| ): |  | ||||||
|     view_dir = tmpdir.mkdir('view') |     view_dir = tmpdir.mkdir('view') | ||||||
|     env('create', '--with-view=%s' % view_dir, 'test') |     env('create', '--with-view=%s' % view_dir, 'test') | ||||||
|     install('--fake', 'mpileaks') |     install('--fake', 'mpileaks') | ||||||
| @@ -687,33 +690,26 @@ def test_env_updates_view_add_concretize( | |||||||
|         add('mpileaks') |         add('mpileaks') | ||||||
|         concretize() |         concretize() | ||||||
|  |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/mpileaks'))) |     check_mpileaks_install(view_dir) | ||||||
|     # Check that dependencies got in too |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/libdwarf'))) |  | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_env_updates_view_uninstall( | def test_env_updates_view_uninstall( | ||||||
|     tmpdir, mock_stage, mock_fetch, install_mockery |         tmpdir, mock_stage, mock_fetch, install_mockery): | ||||||
| ): |  | ||||||
|     view_dir = tmpdir.mkdir('view') |     view_dir = tmpdir.mkdir('view') | ||||||
|     env('create', '--with-view=%s' % view_dir, 'test') |     env('create', '--with-view=%s' % view_dir, 'test') | ||||||
|     with ev.read('test'): |     with ev.read('test'): | ||||||
|         install('--fake', 'mpileaks') |         install('--fake', 'mpileaks') | ||||||
|  |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/mpileaks'))) |     check_mpileaks_install(view_dir) | ||||||
|     # Check that dependencies got in too |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/libdwarf'))) |  | ||||||
|  |  | ||||||
|     with ev.read('test'): |     with ev.read('test'): | ||||||
|         uninstall('-ay') |         uninstall('-ay') | ||||||
|  |  | ||||||
|     assert (not os.path.exists(str(view_dir.join('.spack'))) or |     check_viewdir_removal(view_dir) | ||||||
|             os.listdir(str(view_dir.join('.spack'))) == ['projections.yaml']) |  | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_env_updates_view_uninstall_referenced_elsewhere( | def test_env_updates_view_uninstall_referenced_elsewhere( | ||||||
|     tmpdir, mock_stage, mock_fetch, install_mockery |         tmpdir, mock_stage, mock_fetch, install_mockery): | ||||||
| ): |  | ||||||
|     view_dir = tmpdir.mkdir('view') |     view_dir = tmpdir.mkdir('view') | ||||||
|     env('create', '--with-view=%s' % view_dir, 'test') |     env('create', '--with-view=%s' % view_dir, 'test') | ||||||
|     install('--fake', 'mpileaks') |     install('--fake', 'mpileaks') | ||||||
| @@ -721,20 +717,16 @@ def test_env_updates_view_uninstall_referenced_elsewhere( | |||||||
|         add('mpileaks') |         add('mpileaks') | ||||||
|         concretize() |         concretize() | ||||||
|  |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/mpileaks'))) |     check_mpileaks_install(view_dir) | ||||||
|     # Check that dependencies got in too |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/libdwarf'))) |  | ||||||
|  |  | ||||||
|     with ev.read('test'): |     with ev.read('test'): | ||||||
|         uninstall('-ay') |         uninstall('-ay') | ||||||
|  |  | ||||||
|     assert (not os.path.exists(str(view_dir.join('.spack'))) or |     check_viewdir_removal(view_dir) | ||||||
|             os.listdir(str(view_dir.join('.spack'))) == ['projections.yaml']) |  | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_env_updates_view_remove_concretize( | def test_env_updates_view_remove_concretize( | ||||||
|     tmpdir, mock_stage, mock_fetch, install_mockery |         tmpdir, mock_stage, mock_fetch, install_mockery): | ||||||
| ): |  | ||||||
|     view_dir = tmpdir.mkdir('view') |     view_dir = tmpdir.mkdir('view') | ||||||
|     env('create', '--with-view=%s' % view_dir, 'test') |     env('create', '--with-view=%s' % view_dir, 'test') | ||||||
|     install('--fake', 'mpileaks') |     install('--fake', 'mpileaks') | ||||||
| @@ -742,40 +734,32 @@ def test_env_updates_view_remove_concretize( | |||||||
|         add('mpileaks') |         add('mpileaks') | ||||||
|         concretize() |         concretize() | ||||||
|  |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/mpileaks'))) |     check_mpileaks_install(view_dir) | ||||||
|     # Check that dependencies got in too |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/libdwarf'))) |  | ||||||
|  |  | ||||||
|     with ev.read('test'): |     with ev.read('test'): | ||||||
|         remove('mpileaks') |         remove('mpileaks') | ||||||
|         concretize() |         concretize() | ||||||
|  |  | ||||||
|     assert (not os.path.exists(str(view_dir.join('.spack'))) or |     check_viewdir_removal(view_dir) | ||||||
|             os.listdir(str(view_dir.join('.spack'))) == ['projections.yaml']) |  | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_env_updates_view_force_remove( | def test_env_updates_view_force_remove( | ||||||
|     tmpdir, mock_stage, mock_fetch, install_mockery |         tmpdir, mock_stage, mock_fetch, install_mockery): | ||||||
| ): |  | ||||||
|     view_dir = tmpdir.mkdir('view') |     view_dir = tmpdir.mkdir('view') | ||||||
|     env('create', '--with-view=%s' % view_dir, 'test') |     env('create', '--with-view=%s' % view_dir, 'test') | ||||||
|     with ev.read('test'): |     with ev.read('test'): | ||||||
|         install('--fake', 'mpileaks') |         install('--fake', 'mpileaks') | ||||||
|  |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/mpileaks'))) |     check_mpileaks_install(view_dir) | ||||||
|     # Check that dependencies got in too |  | ||||||
|     assert os.path.exists(str(view_dir.join('.spack/libdwarf'))) |  | ||||||
|  |  | ||||||
|     with ev.read('test'): |     with ev.read('test'): | ||||||
|         remove('-f', 'mpileaks') |         remove('-f', 'mpileaks') | ||||||
|  |  | ||||||
|     assert (not os.path.exists(str(view_dir.join('.spack'))) or |     check_viewdir_removal(view_dir) | ||||||
|             os.listdir(str(view_dir.join('.spack'))) == ['projections.yaml']) |  | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_env_activate_view_fails( | def test_env_activate_view_fails( | ||||||
|     tmpdir, mock_stage, mock_fetch, install_mockery |         tmpdir, mock_stage, mock_fetch, install_mockery): | ||||||
| ): |  | ||||||
|     """Sanity check on env activate to make sure it requires shell support""" |     """Sanity check on env activate to make sure it requires shell support""" | ||||||
|     out = env('activate', 'test') |     out = env('activate', 'test') | ||||||
|     assert "To initialize spack's shell commands:" in out |     assert "To initialize spack's shell commands:" in out | ||||||
|   | |||||||
| @@ -125,7 +125,7 @@ def test_package_output(tmpdir, capsys, install_mockery, mock_fetch): | |||||||
|     pkg = spec.package |     pkg = spec.package | ||||||
|     pkg.do_install(verbose=True) |     pkg.do_install(verbose=True) | ||||||
|  |  | ||||||
|     log_file = os.path.join(spec.prefix, '.spack', 'build.txt') |     log_file = pkg.build_log_path | ||||||
|     with open(log_file) as f: |     with open(log_file) as f: | ||||||
|         out = f.read() |         out = f.read() | ||||||
|  |  | ||||||
|   | |||||||
| @@ -29,9 +29,7 @@ def layout_and_dir(tmpdir): | |||||||
|     spack.store.layout = old_layout |     spack.store.layout = old_layout | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_yaml_directory_layout_parameters( | def test_yaml_directory_layout_parameters(tmpdir, config): | ||||||
|         tmpdir, config |  | ||||||
| ): |  | ||||||
|     """This tests the various parameters that can be used to configure |     """This tests the various parameters that can be used to configure | ||||||
|     the install location """ |     the install location """ | ||||||
|     spec = Spec('python') |     spec = Spec('python') | ||||||
| @@ -84,9 +82,7 @@ def test_yaml_directory_layout_parameters( | |||||||
|                             path_scheme=scheme_package7) |                             path_scheme=scheme_package7) | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_read_and_write_spec( | def test_read_and_write_spec(layout_and_dir, config, mock_packages): | ||||||
|         layout_and_dir, config, mock_packages |  | ||||||
| ): |  | ||||||
|     """This goes through each package in spack and creates a directory for |     """This goes through each package in spack and creates a directory for | ||||||
|     it.  It then ensures that the spec for the directory's |     it.  It then ensures that the spec for the directory's | ||||||
|     installed package can be read back in consistently, and |     installed package can be read back in consistently, and | ||||||
| @@ -162,9 +158,7 @@ def test_read_and_write_spec( | |||||||
|         assert not os.path.exists(install_dir) |         assert not os.path.exists(install_dir) | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_handle_unknown_package( | def test_handle_unknown_package(layout_and_dir, config, mock_packages): | ||||||
|         layout_and_dir, config, mock_packages |  | ||||||
| ): |  | ||||||
|     """This test ensures that spack can at least do *some* |     """This test ensures that spack can at least do *some* | ||||||
|     operations with packages that are installed but that it |     operations with packages that are installed but that it | ||||||
|     does not know about.  This is actually not such an uncommon |     does not know about.  This is actually not such an uncommon | ||||||
| @@ -234,3 +228,14 @@ def test_find(layout_and_dir, config, mock_packages): | |||||||
|     for name, spec in found_specs.items(): |     for name, spec in found_specs.items(): | ||||||
|         assert name in found_specs |         assert name in found_specs | ||||||
|         assert found_specs[name].eq_dag(spec) |         assert found_specs[name].eq_dag(spec) | ||||||
|  |  | ||||||
|  |  | ||||||
|  | def test_yaml_directory_layout_build_path(tmpdir, config): | ||||||
|  |     """This tests build path method.""" | ||||||
|  |     spec = Spec('python') | ||||||
|  |     spec.concretize() | ||||||
|  |  | ||||||
|  |     layout = YamlDirectoryLayout(str(tmpdir)) | ||||||
|  |     rel_path = os.path.join(layout.metadata_dir, layout.packages_dir) | ||||||
|  |     assert layout.build_packages_path(spec) == os.path.join(spec.prefix, | ||||||
|  |                                                             rel_path) | ||||||
|   | |||||||
| @@ -5,11 +5,15 @@ | |||||||
|  |  | ||||||
| import os | import os | ||||||
| import pytest | import pytest | ||||||
|  | import shutil | ||||||
|  |  | ||||||
|  | from llnl.util.filesystem import mkdirp, touch, working_dir | ||||||
|  |  | ||||||
| import spack.patch | import spack.patch | ||||||
| import spack.repo | import spack.repo | ||||||
| import spack.store | import spack.store | ||||||
| from spack.spec import Spec | from spack.spec import Spec | ||||||
|  | from spack.package import _spack_build_envfile, _spack_build_logfile | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_install_and_uninstall(install_mockery, mock_fetch, monkeypatch): | def test_install_and_uninstall(install_mockery, mock_fetch, monkeypatch): | ||||||
| @@ -269,3 +273,97 @@ def test_failing_build(install_mockery, mock_fetch): | |||||||
|  |  | ||||||
| class MockInstallError(spack.error.SpackError): | class MockInstallError(spack.error.SpackError): | ||||||
|     pass |     pass | ||||||
|  |  | ||||||
|  |  | ||||||
|  | def test_pkg_build_paths(install_mockery): | ||||||
|  |     # Get a basic concrete spec for the trivial install package. | ||||||
|  |     spec = Spec('trivial-install-test-package').concretized() | ||||||
|  |  | ||||||
|  |     log_path = spec.package.log_path | ||||||
|  |     assert log_path.endswith(_spack_build_logfile) | ||||||
|  |  | ||||||
|  |     env_path = spec.package.env_path | ||||||
|  |     assert env_path.endswith(_spack_build_envfile) | ||||||
|  |  | ||||||
|  |     # Backward compatibility checks | ||||||
|  |     log_dir = os.path.dirname(log_path) | ||||||
|  |     mkdirp(log_dir) | ||||||
|  |     with working_dir(log_dir): | ||||||
|  |         # Start with the older of the previous log filenames | ||||||
|  |         older_log = 'spack-build.out' | ||||||
|  |         touch(older_log) | ||||||
|  |         assert spec.package.log_path.endswith(older_log) | ||||||
|  |  | ||||||
|  |         # Now check the newer log filename | ||||||
|  |         last_log = 'spack-build.txt' | ||||||
|  |         os.rename(older_log, last_log) | ||||||
|  |         assert spec.package.log_path.endswith(last_log) | ||||||
|  |  | ||||||
|  |         # Check the old environment file | ||||||
|  |         last_env = 'spack-build.env' | ||||||
|  |         os.rename(last_log, last_env) | ||||||
|  |         assert spec.package.env_path.endswith(last_env) | ||||||
|  |  | ||||||
|  |     # Cleanup | ||||||
|  |     shutil.rmtree(log_dir) | ||||||
|  |  | ||||||
|  |  | ||||||
|  | def test_pkg_install_paths(install_mockery): | ||||||
|  |     # Get a basic concrete spec for the trivial install package. | ||||||
|  |     spec = Spec('trivial-install-test-package').concretized() | ||||||
|  |  | ||||||
|  |     log_path = os.path.join(spec.prefix, '.spack', _spack_build_logfile) | ||||||
|  |     assert spec.package.install_log_path == log_path | ||||||
|  |  | ||||||
|  |     env_path = os.path.join(spec.prefix, '.spack', _spack_build_envfile) | ||||||
|  |     assert spec.package.install_env_path == env_path | ||||||
|  |  | ||||||
|  |     # Backward compatibility checks | ||||||
|  |     log_dir = os.path.dirname(log_path) | ||||||
|  |     mkdirp(log_dir) | ||||||
|  |     with working_dir(log_dir): | ||||||
|  |         # Start with the older of the previous install log filenames | ||||||
|  |         older_log = 'build.out' | ||||||
|  |         touch(older_log) | ||||||
|  |         assert spec.package.install_log_path.endswith(older_log) | ||||||
|  |  | ||||||
|  |         # Now check the newer install log filename | ||||||
|  |         last_log = 'build.txt' | ||||||
|  |         os.rename(older_log, last_log) | ||||||
|  |         assert spec.package.install_log_path.endswith(last_log) | ||||||
|  |  | ||||||
|  |         # Check the old install environment file | ||||||
|  |         last_env = 'build.env' | ||||||
|  |         os.rename(last_log, last_env) | ||||||
|  |         assert spec.package.install_env_path.endswith(last_env) | ||||||
|  |  | ||||||
|  |     # Cleanup | ||||||
|  |     shutil.rmtree(log_dir) | ||||||
|  |  | ||||||
|  |  | ||||||
|  | def test_pkg_install_log(install_mockery): | ||||||
|  |     # Get a basic concrete spec for the trivial install package. | ||||||
|  |     spec = Spec('trivial-install-test-package').concretized() | ||||||
|  |  | ||||||
|  |     # Attempt installing log without the build log file | ||||||
|  |     with pytest.raises(IOError, match="No such file or directory"): | ||||||
|  |         spec.package.log() | ||||||
|  |  | ||||||
|  |     # Set up mock build files and try again | ||||||
|  |     log_path = spec.package.log_path | ||||||
|  |     log_dir = os.path.dirname(log_path) | ||||||
|  |     mkdirp(log_dir) | ||||||
|  |     with working_dir(log_dir): | ||||||
|  |         touch(log_path) | ||||||
|  |         touch(spec.package.env_path) | ||||||
|  |  | ||||||
|  |     install_path = os.path.dirname(spec.package.install_log_path) | ||||||
|  |     mkdirp(install_path) | ||||||
|  |  | ||||||
|  |     spec.package.log() | ||||||
|  |  | ||||||
|  |     assert os.path.exists(spec.package.install_log_path) | ||||||
|  |     assert os.path.exists(spec.package.install_env_path) | ||||||
|  |  | ||||||
|  |     # Cleanup | ||||||
|  |     shutil.rmtree(log_dir) | ||||||
|   | |||||||
| @@ -425,7 +425,7 @@ def install_links(self): | |||||||
|         # Make build log visible - it contains OpenFOAM-specific information |         # Make build log visible - it contains OpenFOAM-specific information | ||||||
|         with working_dir(self.projectdir): |         with working_dir(self.projectdir): | ||||||
|             os.symlink( |             os.symlink( | ||||||
|                 join_path('.spack', 'build.out'), |                 join_path(os.path.relpath(self.install_log_path)), | ||||||
|                 join_path('log.' + str(self.foam_arch))) |                 join_path('log.' + str(self.foam_arch))) | ||||||
|  |  | ||||||
| # ----------------------------------------------------------------------------- | # ----------------------------------------------------------------------------- | ||||||
|   | |||||||
| @@ -29,11 +29,3 @@ def install(self, spec, prefix): | |||||||
|            '-targetdir={0}'.format(prefix), |            '-targetdir={0}'.format(prefix), | ||||||
|            '-execdir={0}'.format(prefix.bin), |            '-execdir={0}'.format(prefix.bin), | ||||||
|            '-selinux=y') |            '-selinux=y') | ||||||
|  |  | ||||||
|         # after the install phase completes, spack tries to install build.out |  | ||||||
|         # into <prefix>/.spack, but the .spack dir will not exist, causing the |  | ||||||
|         # build to fail. package.py:1690 seems to show that the dir is created |  | ||||||
|         # right before writing build.out -- possible bug? |  | ||||||
|  |  | ||||||
|         # creating the .spack dir right after installing prevents explosions |  | ||||||
|         mkdirp(join_path(prefix, '.spack')) |  | ||||||
|   | |||||||
| @@ -344,7 +344,7 @@ def install_links(self): | |||||||
|         # Make build log visible - it contains OpenFOAM-specific information |         # Make build log visible - it contains OpenFOAM-specific information | ||||||
|         with working_dir(self.projectdir): |         with working_dir(self.projectdir): | ||||||
|             os.symlink( |             os.symlink( | ||||||
|                 join_path('.spack', 'build.out'), |                 join_path(os.path.relpath(self.install_log_path)), | ||||||
|                 join_path('log.' + str(self.foam_arch))) |                 join_path('log.' + str(self.foam_arch))) | ||||||
|  |  | ||||||
|         if not self.config['link']: |         if not self.config['link']: | ||||||
|   | |||||||
| @@ -724,7 +724,7 @@ def install_links(self): | |||||||
|         # Make build log visible - it contains OpenFOAM-specific information |         # Make build log visible - it contains OpenFOAM-specific information | ||||||
|         with working_dir(self.projectdir): |         with working_dir(self.projectdir): | ||||||
|             os.symlink( |             os.symlink( | ||||||
|                 join_path('.spack', 'build.out'), |                 join_path(os.path.relpath(self.install_log_path)), | ||||||
|                 join_path('log.' + str(self.foam_arch))) |                 join_path('log.' + str(self.foam_arch))) | ||||||
|  |  | ||||||
|         if not self.config['link']: |         if not self.config['link']: | ||||||
|   | |||||||
| @@ -42,7 +42,7 @@ class Picard(Package): | |||||||
|     def install(self, spec, prefix): |     def install(self, spec, prefix): | ||||||
|         mkdirp(prefix.bin) |         mkdirp(prefix.bin) | ||||||
|         # The list of files to install varies with release... |         # The list of files to install varies with release... | ||||||
|         # ... but skip the spack-{build.env}.out files. |         # ... but skip the spack-build-{env}out.txt files. | ||||||
|         files = [x for x in glob.glob("*") if not re.match("^spack-", x)] |         files = [x for x in glob.glob("*") if not re.match("^spack-", x)] | ||||||
|         for f in files: |         for f in files: | ||||||
|             install(f, prefix.bin) |             install(f, prefix.bin) | ||||||
|   | |||||||
		Reference in New Issue
	
	Block a user