Merge pull request #94 from yuvipanda/tljh-bin

Symlink tljh-config to /usr/local/bin

docs/index.rst

@@ -80,6 +80,7 @@ Topic guides provide in-depth explanations of specific topics.
    topic/requirements
    topic/security
    topic/customizing-installer
+   topic/installer-actions
    topic/tljh-config
    topic/authenticator-configuration
 

docs/topic/customizing-installer.rst

@@ -1,3 +1,5 @@
+.. _topic/customizing-installer:
+
 =========================
 Customizing the Installer
 =========================

docs/topic/installer-actions.rst

@@ -0,0 +1,92 @@
+.. _topic/installer-actions:
+
+===========================
+What does the installer do?
+===========================
+
+This document details what exactly the installer does to the machine it is 
+run on.
+
+``apt`` Packages installed
+==========================
+
+The packages ``python3`` and ``python3-venv`` are installed from the apt repositories.
+Since we need an recent & supported version of ``nodejs``, we install it from 
+`nodesource <https://github.com/nodesource/distributions>`_.
+
+Hub environment
+===============
+
+JupyterHub is run from a python3 virtual environment located in ``/opt/tljh/hub``. It
+uses the system's installed python and is owned by root. It also contains a ``npm``
+install of `configurable-http-proxy <https://github.com/jupyterhub/configurable-http-proxy>`_
+and a binary install of `traefik <http://traefik.io/>`_. This virtual environment is
+completely managed by TLJH.
+
+User environment
+================
+
+By default, a ``miniconda`` environment is installed in ``/opt/tljh/user``. This contains
+the notebook interface used to launch all users, and the various packages available to all
+users. The environment is owned by the ``root`` user. JupyterHub admins may use 
+to ``sudo -E conda install`` or ``sudo -E pip install`` packages into this environment.
+
+This conda environment is added to ``$PATH`` for all users started with JupyterHub. If you
+are using ``ssh`` instead, you can activate this environment by running the following:
+
+.. code-block:: bash
+
+   source /opt/tljh/user/bin/activate
+
+This should let you run various ``conda`` and ``pip`` commands. If  you run into errors like
+``Command 'conda' not found``, try prefixing your command with:
+
+.. code-block:: bash
+
+   sudo PATH=${PATH} <command> 
+
+By default, ``sudo`` does not respect any custom environments you have activated. The ``PATH=${PATH}``
+'fixes' that.
+
+``tljh-config`` symlink
+========================
+
+We create a symlink from ``/usr/local/bin/tljh-config`` to ``/opt/tljh/hub/bin/tljh-cohnfig``, so users
+can run ``sudo -E tljh-config <somethihng>`` from their terminal. While the user environment is added
+to users' ``$PATH`` when they launch through JupyterHub, the hub environment is not. This makes it
+hard to access the ``tljh-config`` command used to change most config parameters. Hence we symlink the
+``tljh-config`` command to ``/usr/local/bin``, so it is directly accessible with ``sudo -E tljh-config <command>``.
+
+Systemd Units
+=============
+
+TLJH places 3 systemd units on your computer. They all start on system startup.
+
+#. ``jupyterhub.service`` - starts the JupyterHub service.
+#. ``configurable-http-proxy.service`` - starts the nodejs based proxy that is used by JupyterHub.
+#. ``traefik.service`` - starts traefik proxy that manages HTTPS
+
+In addition, each running Jupyter user gets their own systemd unit of the name ``jupyter-<username>``.
+
+User groups
+===========
+
+TLJH creates two user groups when installed:
+
+#. ``jupyterhub-users`` contains all users managed by this JupyterHub
+#. ``jupyterhub-admins`` contains all users with admin rights managed by this JupyterHub.
+
+When a new JupyterHub user logs in, a unix user is created for them. The unix user is always added
+to the ``jupyterhub-users`` group. If the user is an admin, they are added to the ``jupyterhub-admins``
+group whenever they start / stop their notebook server.
+
+If you uninstall TLJH, you should probably remove all user accounts associated with both these
+user groups, and then remove the groups themselves. You might have to archive or delete the home
+directories of these users under ``/home/``.
+
+Passwordless ``sudo`` for JupyterHub admins
+============================================
+
+``/etc/sudoers.d/jupyterhub-admins`` is created to provide passwordless sudo for all JupyterHub
+admins. We also set it up to inherit ``$PATH`` with ``sudo -E``, to more easily call ``conda``,
+``pip``, etc.

docs/tutorials/custom.rst

@@ -37,6 +37,11 @@ Step 1: Installing The Littlest JupyterHub
         | sudo python3 - \
        --admin <admin-user-name>
 
+   .. note:
+
+      See :ref:`topic/installer-actions` if you want to understand exactly what the installer is doing.
+      :ref:`topic/customizing-installer` documents other options that can be passed to the installer.
+
 #. Press ``Enter`` to start the installation process. This will take 5-10 minutes,
    and will say 'Done!' when the installation process is complete.
 

docs/tutorials/digitalocean.rst

@@ -67,6 +67,11 @@ Let's create the server on which we can run JupyterHub.
         | sudo python3 - \
           --admin <admin-user-name>
 
+   .. note::
+
+      See :ref:`topic/installer-actions` if you want to understand exactly what the installer is doing.
+      :ref:`topic/customizing-installer` documents other options that can be passed to the installer.
+
 #. Under the **Finalize and create** section, enter a ``hostname`` that descriptively
    identifies this server for you.
 

docs/tutorials/google.rst

@@ -151,6 +151,11 @@ Let's create the server on which we can run JupyterHub.
    .. image:: ../images/providers/google/startup-script.png
       :alt: Install JupyterHub with the Startup script textbox
 
+   .. note::
+
+      See :ref:`topic/installer-actions` if you want to understand exactly what the installer is doing.
+      :ref:`topic/customizing-installer` documents other options that can be passed to the installer.
+
 #. Click the **Create** button at the bottom to start your server!
 
    .. image:: ../images/providers/google/create-vm-button.png

docs/tutorials/jetstream.rst

@@ -89,6 +89,11 @@ Let's create the server on which we can run JupyterHub.
         | sudo python3 - \
           --admin <admin-user-name>
 
+   .. note::
+
+      See :ref:`topic/installer-actions` if you want to understand exactly what the installer is doing.
+      :ref:`topic/customizing-installer` documents other options that can be passed to the installer.
+
 #. Under **Execution Strategy Type**, select **Run script on first boot**.
 
 #. Under **Deployment Type**, select **Wait for script to complete**.

integration-tests/Dockerfile

@@ -3,7 +3,7 @@ FROM ubuntu:18.04
 
 RUN apt-get update --yes
 
-RUN apt-get install --yes systemd curl git
+RUN apt-get install --yes systemd curl git sudo
 
 # Kill all the things we don't need
 RUN find /etc/systemd/system \

integration-tests/test_hub.py

@@ -10,6 +10,10 @@ import grp
 import sys
 
 
+# Use sudo to invoke it, since this is how users invoke it.
+# This catches issues with PATH
+TLJH_CONFIG_PATH = ['sudo', '-E', 'tljh-config']
+
 def test_hub_up():
     r = requests.get('http://127.0.0.1')
     r.raise_for_status()
@@ -45,10 +49,9 @@ async def test_user_admin_add():
     hub_url = 'http://localhost'
     username = secrets.token_hex(8)
 
-    tljh_config_path = [sys.executable, '-m', 'tljh.config']
 
-    assert 0 == await (await asyncio.create_subprocess_exec(*tljh_config_path, 'add-item', 'users.admin', username)).wait()
-    assert 0 == await (await asyncio.create_subprocess_exec(*tljh_config_path, 'reload')).wait()
+    assert 0 == await (await asyncio.create_subprocess_exec(*TLJH_CONFIG_PATH, 'add-item', 'users.admin', username)).wait()
+    assert 0 == await (await asyncio.create_subprocess_exec(*TLJH_CONFIG_PATH, 'reload')).wait()
 
     # FIXME: wait for reload to finish & hub to come up
     # Should be part of tljh-config reload
@@ -76,10 +79,8 @@ async def test_user_admin_remove():
     hub_url = 'http://localhost'
     username = secrets.token_hex(8)
 
-    tljh_config_path = [sys.executable, '-m', 'tljh.config']
-
-    assert 0 == await (await asyncio.create_subprocess_exec(*tljh_config_path, 'add-item', 'users.admin', username)).wait()
-    assert 0 == await (await asyncio.create_subprocess_exec(*tljh_config_path, 'reload')).wait()
+    assert 0 == await (await asyncio.create_subprocess_exec(*TLJH_CONFIG_PATH, 'add-item', 'users.admin', username)).wait()
+    assert 0 == await (await asyncio.create_subprocess_exec(*TLJH_CONFIG_PATH, 'reload')).wait()
 
     # FIXME: wait for reload to finish & hub to come up
     # Should be part of tljh-config reload
@@ -95,8 +96,8 @@ async def test_user_admin_remove():
             assert f'jupyter-{username}' in grp.getgrnam('jupyterhub-admins').gr_mem
 
 
-            assert 0 == await (await asyncio.create_subprocess_exec(*tljh_config_path, 'remove-item', 'users.admin', username)).wait()
-            assert 0 == await (await asyncio.create_subprocess_exec(*tljh_config_path, 'reload')).wait()
+            assert 0 == await (await asyncio.create_subprocess_exec(*TLJH_CONFIG_PATH, 'remove-item', 'users.admin', username)).wait()
+            assert 0 == await (await asyncio.create_subprocess_exec(*TLJH_CONFIG_PATH, 'reload')).wait()
             await asyncio.sleep(1)
 
             await u.stop_server()

integration-tests/test_install.py

@@ -204,3 +204,9 @@ def test_pip_upgrade(group, allowed):
             [python, "-m", "pip", "install", "--upgrade", "testpath"],
             preexec_fn=partial(setgroup, group),
         )
+
+def test_symlinks():
+    """
+    Test we symlink tljh-config to /usr/local/bin
+    """
+    assert os.path.exists('/usr/local/bin/tljh-config')

tljh/installer.py

@@ -275,6 +275,30 @@ def ensure_jupyterhub_running(times=4):
     raise Exception("Installation failed: JupyterHub did not start in {}s".format(times))
 
 
+def ensure_symlinks(prefix):
+    """
+    Ensure we symlink appropriate things into /usr/local/bin
+
+    We add the user conda environment to PATH for notebook terminals,
+    but not the hub venv. This means tljh-config is not actually accessible.
+
+    We symlink to /usr/local/bin to 'fix' this. /usr/local/bin is the appropriate
+    place, and works with sudo -E
+    """
+    tljh_config_src = os.path.join(prefix, 'bin', 'tljh-config')
+    tljh_config_dest = '/usr/local/bin/tljh-config'
+    if os.path.exists(tljh_config_dest):
+        if os.path.realpath(tljh_config_dest) != tljh_config_src:
+            #  tljh-config exists that isn't ours. We should *not* delete this file,
+            # instead we throw an error and abort. Deleting files owned by other people
+            # while running as root is dangerous, especially with symlinks involved.
+            raise FileExistsError(f'/usr/local/bin/tljh-config exists but is not a symlink to {tljh_config_src}')
+        else:
+            # We have a working symlink, so do nothing
+            return
+    os.symlink(tljh_config_src, tljh_config_dest)
+
+
 def main():
     argparser = argparse.ArgumentParser()
     argparser.add_argument(
@@ -302,6 +326,7 @@ def main():
     ensure_chp_package(HUB_ENV_PREFIX)
     ensure_jupyterhub_service(HUB_ENV_PREFIX)
     ensure_jupyterhub_running()
+    ensure_symlinks(HUB_ENV_PREFIX)
 
     logger.info("Done!")