Overhaul Spack's URL parsing (#2972)

* Remove fake URLs from Spack

* Ignore long lines for URLs that start with ftp:

* Preliminary changes to version regexes

* New redesign of version regexes

* Allow letters in version-only

* Fix detection of versions that end in Final

* Rearrange a few regexes and add examples

* Add tests for common download repositories

* Add test cases for common tarball naming schemes

* Finalize version regexes

* spack url test -> spack url summary

* Clean up comments

* Rearrange suffix checks

* Use query strings for name detection

* Remove no longer necessary url_for_version functions

* Strip off extraneous information after package name

* Add one more test

* Dot in square brackets does not need to be escaped

* Move renaming outside of parse_name_offset

* Fix versions for a couple more packages

* Fix flake8 and doc tests

* Correctly parse Python, Lua, and Bio++ package names

* Use effective URLs for mfem

* Add checksummed version to mitos

* Remove url_for_version from STAR-CCM+ package

* Revert changes to version numbers with underscores and dashes

* Fix name detection for tbb

* Correctly parse Ruby gems

* Reverted mfem back to shortened URLs.

* Updated instructions for better security

* Remove preferred=True from newest version

* Add tests for new `spack url list` flags

* Add tests for strip_name_suffixes

* Add unit tests for version separators

* Fix bugs related to parseable name but in parseable version

* Remove dead code, update docstring

* Ignore 'binary' at end of version string

* Remove platform from version

* Flip libedit version numbers

* Re-support weird NCO alpha/beta versions

* Rebase and remove one new fake URL

* Add / to beginning of regex to avoid picking up similarly named packages

* Ignore weird tar versions

* Fix bug in url parse --spider when no versions found

* Less strict version matching for spack versions

* Don't rename Python packages

* Be a little more selective, version must begin with a digit

* Re-add fake URLs

* Fix up several other packages

* Ignore more file endings

* Add parsing support for Miniconda

* Update tab completion

* XFAILS are now PASSES for 2 web tests

var/spack/repos/builtin/packages/mfem/package.py

@@ -31,15 +31,32 @@ class Mfem(Package):
     homepage = 'http://www.mfem.org'
     url      = 'https://github.com/mfem/mfem'
 
+    # mfem is downloaded from a URL shortener at request of upstream
+    # author Tzanio Kolev <tzanio@llnl.gov>.  See here:
+    #     https://github.com/mfem/mfem/issues/53
+    #
+    # The following procedure should be used to verify security when a
+    # new verison is added:
+    #
+    # 1. Verify that no checksums on old versions have changed.
+    #
+    # 2. Verify that the shortened URL for the new version is listed at:
+    #    http://mfem.org/download/
+    #
+    # 3. Use http://getlinkinfo.com or similar to verify that the
+    #    underling download link for the latest version comes has the
+    #    prefix: http://mfem.github.io/releases
+    #
+    # If this quick verification procedure fails, additional discussion
+    # will be required to verify the new version.
+
     version('3.2',
             '2938c3deed4ec4f7fd5b5f5cfe656845282e86e2dcd477d292390058b7b94340',
-            url='http://goo.gl/Y9T75B', preferred=True, extension='.tar.gz')
+            url='http://goo.gl/Y9T75B', extension='.tar.gz')
 
     version('3.1',
             '841ea5cf58de6fae4de0f553b0e01ebaab9cd9c67fa821e8a715666ecf18fc57',
             url='http://goo.gl/xrScXn', extension='.tar.gz')
-#    version('3.1', git='https://github.com/mfem/mfem.git',
-#            commit='dbae60fe32e071989b52efaaf59d7d0eb2a3b574')
 
     variant('metis', default=False, description='Activate support for metis')
     variant('hypre', default=False, description='Activate support for hypre')