Add better tests for web.py; fix some bugs found with spidering.

- _spider in web.py was actually failing to spider deeper than a certain
  point.
- Fixed multiprocessing pools to not use daemons and to allow recursive
  spawning.
- Added detailed tests for spidering and for finding archive versions.
- left some xfail URL finding exercises for the reader.
- Fix noqa annotations for some @when decorators

var/spack/repos/builtin/packages/patchelf/package.py

@@ -33,7 +33,7 @@ class Patchelf(AutotoolsPackage):
     url = "http://nixos.org/releases/patchelf/patchelf-0.8/patchelf-0.8.tar.gz"
 
     list_url = "http://nixos.org/releases/patchelf/"
-    list_depth = 2
+    list_depth = 1
 
     version('0.9', '3c265508526760f233620f35d79c79fc')
     version('0.8', '407b229e6a681ffb0e2cdd5915cb2d01')