zhangyiss/the-littlest-jupyterhub
1
0
Fork 0
mirror of https://github.com/jupyterhub/the-littlest-jupyterhub.git synced 2025-12-18 21:54:05 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #77 from jupyterhub/config

Browse Source 
Add tljh-config command
This commit is contained in:
Yuvi Panda
2018-07-28 13:12:46 -07:00
committed by GitHub
parent 6c43a492ac 7acd331e38
commit 1ba7a3a565
12 changed files with 684 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
.circleci/config.yml
View File

@@ -77,6 +77,13 @@ jobs:
python3 .circleci/integration-test.py copy . /srv/src
python3 .circleci/integration-test.py run 'python3 /srv/src/bootstrap/bootstrap.py'
- run:
name: switch to dummyauthenticator
command: |
python3 .circleci/integration-test.py run '/opt/tljh/hub/bin/tljh-config set auth.type dummyauthenticator.DummyAuthenticator'
python3 .circleci/integration-test.py run '/opt/tljh/hub/bin/tljh-config reload'
- run:
name: print systemd status + logs
command: |

57
docs/guides/admin.rst
View File

@@ -1,57 +0,0 @@
.. _admin_access:
=====================
Administrative Access
=====================
In The Littlest JupyterHub, we try to allow users to do as many administrative
tasks as possible within JupyterHub itself. Admin users can:
1. Have full root access with passwordless ``sudo``
2. Install system-wide packages with ``apt``
3. Install ``conda`` / ``pip`` packages for all JupyterHub users
4. Change the amount of RAM / CPU available to each user, and more!
By default, there are no admin users. You should add some after installation.
Adding admin users
==================
Admin users are specified in the `YAML <https://en.wikipedia.org/wiki/YAML>`_
config file at ``/opt/tljh/config.yaml``. This file is created upon installing
tljh.
1. Open the ``config.yaml`` file for editing.
.. code-block:: bash
sudo nano /opt/tljh/config.yaml
2. Add usernames that should have admin access.
.. code-block:: yaml
users:
admin:
- user1
- user2
Be careful around the syntax - indentation matters, and you should be using
spaces and not tabs.
When you are done, save the file and exit. In ``nano``, you can do this with
``Ctrl+X`` key.
3. When you are sure the format is ok, restart JupyterHub to let the config take
effect.
.. code-block:: bash
sudo systemctl restart jupyterhub
This should give you admin access from JupyterHub! You can verify this by:
1. Opening a Terminal in your JupyterHub and checking if ``sudo`` works
2. Opening your JupyterHub ``Control Panel`` and checking for the **Admin** tab
From now on, you can use the JupyterHub to do most configuration changes.

55
docs/howto/admin-users.rst Normal file
View File

@@ -0,0 +1,55 @@
.. _howto/admin-users:
========================
Add / Remove admin users
========================
Admin users in TLJH have the following powers:
#. Full root access to the server with passwordless ``sudo``.
This lets them do literally whatever they want in the server
#. Access servers / home directories of all other users
#. Install new packages for everyone with ``conda``, ``pip`` or ``apt``
#. Change configuration of TLJH
This is a lot of power, so make sure you know who you are giving it
to. Admin users should have decent passwords / secure logni mechanisms,
so attackers can not easily gain control of the system.
Make sure an admin user is present
==================================
You should make sure an admin user is present when you **install** TLJH
the very first time. The ``:ref:`--admin <topic/customizing-installer/admin>```
flag passed to the installer does this. If you had forgotten to do so, the
easiest way to fix this is to run the installer again.
Adding new admin users
======================
New admin users can be added by executing the following commands on an
admin terminal:
.. code-block:: bash
sudo -E tljh-config add-item users.admin <username>
sudo -E tljh-config reload
If the user is already using the JupyterHub, they might have to stop and
start their server from the control panel to gain new powers.
Removing admin users
====================
You can remove an existing admin user by executing the following commands in
an admin terminal:
.. code-block:: bash
sudo -E tljh-config remove-item users.admin <username>
sudo -E tljh-config reload
If the user is already using the JupyterHub, they will continue to have
some of their admin powers until their server is stopped. Another admin
can force their server to stop by clicking 'Stop Server' in the admin
panel.

43
docs/howto/auth/dummy.rst Normal file
View File

@@ -0,0 +1,43 @@
.. _howto/auth/dummy:
=====================================================
Authenticate *any* user with a single shared password
=====================================================
The **Dummy Authenticator** lets *any* user log in with the given password.
This authenticator is **extremely insecure**, so do not use it if you can
avoid it.
Enabling the authenticator
==========================
1. Always use DummyAuthenticator with a password. You can communicate this
password to all your users via an out of band mechanism (like writing on
a whiteboard). Once you have selected a password, configure TLJH to use
the password by executing the following from an admin console.
.. code-block:: bash
sudo -E tljh-config set auth.DummyAuthenticator.password <password>
Remember to replace ``<password>`` with the password you choose.
2. Enable the authenticator and reload config to apply configuration:
sudo -E tljh-config set auth.type dummyauthenticator.DummyAuthenticator
sudo -E tljh-config reload
Users who are currently logged in will continue to be logged in. When they
log out and try to log back in, they will be asked to provide a username and
password.
Changing the password
=====================
The password used by DummyAuthenticator can be changed with the following
commands:
.. code-block:: bash
tljh-config set auth.DummyAuthenticator.password <new-password>
tljh-config reload

11
docs/index.rst
View File

@@ -54,9 +54,18 @@ How-To guides answer the question 'How do I...?' for a lot of topics.
:titlesonly:
howto/user-environment
howto/admin-users
howto/notebook-interfaces
howto/resource-estimation
We have a special set of How-To Guides on using various forms of authentication
with your JupyterHub.
.. toctree::
:titlesonly:
howto/auth/dummy
Topic Guides
============
@@ -66,9 +75,9 @@ Topic guides provide in-depth explanations of specific topics.
:titlesonly:
topic/requirements
guides/admin
topic/security
topic/customizing-installer
topic/tljh-config
Troubleshooting

2
docs/topic/customizing-installer.rst
View File

@@ -13,6 +13,8 @@ is executed as:
This page documents the various options you can pass as commandline parameters to the installer.
.. _topic/customizing-installer/admin:
Adding admin users
===================

81
docs/topic/tljh-config.rst Normal file
View File

@@ -0,0 +1,81 @@
.. _topic/tljh-config:
=====================================
Configuring TLJH with ``tljh-config``
=====================================
``tljh-config`` is the commandline program used to make configuration
changes to TLJH.
Running ``tljh-config``
======================`
You can run ``tljh-config`` in two ways:
#. From inside a terminal in JupyterHub while logged in as an admin user.
This method is **recommended**.
#. By directly calling ``/opt/tljh/hub/bin/tljh-config`` as root when
logged in to the server via other means (such as SSH). This is an
advanced use case, and not covered much in this guide.
Set a configuration property
============================
TLJH's configuration is organized in a nested tree structure. You can
set a particular property with the following command:
.. code-block:: bash
sudo -E tljh-config set <property-path> <value>
where:
#. ``<property-path>`` is a dot-separated path to the property you want
to set.
#. ``<value>`` is the value you want to set the property to.
For example, to set the password for the DummyAuthenticator, you
need to set the ``auth.DummyAuthenticator.password`` property. You would
do so with the following:
.. code-block:: bash
sudo -E tljh-config set auth.DummyAuthenticator.password mypassword
This can only set string and numerical properties, not lists.
View current configuration
==========================
To see the current configuration, you can run the following command:
.. code-block:: bash
sudo -E tljh-config show
This will print the current configuration of your TLJH. This is very
useful when asking for support!
Reloading JupyterHub to apply configuration
===========================================
After modifying the configuration, you need to reload JupyterHub for
it to take effect. You can do so with:
.. code-block:: bash
sudo -E tljh-config reload
This should not affect any running users. The JupyterHub will be
restarted and loaded with the new configuration.
Advanced: ``config.yaml``
=========================
``tljh-config`` is a simple program that modifies the contents of the
``config.yaml`` file located at ``/opt/tljh/config.yaml``. ``tljh-config``
is the recommended method of editing / viewing configuration since editing
YAML by hand in a terminal text editor is a large source of errors.

3
integration-tests/requirements.txt
View File

@@ -1,2 +1,3 @@
pytest
requests
pytest-asyncio
git+https://github.com/yuvipanda/hubtraf.git

101
integration-tests/test_hub.py
View File

@@ -1,5 +1,106 @@
import requests
from hubtraf.user import User
from hubtraf.auth.dummy import login_dummy
import secrets
import pytest
from functools import partial
import asyncio
import pwd
import grp
import sys
def test_hub_up():
r = requests.get('http://127.0.0.1')
r.raise_for_status()
@pytest.mark.asyncio
async def test_user_code_execute():
"""
User logs in, starts a server & executes code
"""
# This *must* be localhost, not an IP
# aiohttp throws away cookies if we are connecting to an IP!
hub_url = 'http://localhost'
username = secrets.token_hex(8)
async with User(username, hub_url, partial(login_dummy, password='')) as u:
await u.login()
await u.ensure_server()
await u.start_kernel()
await u.assert_code_output("5 * 4", "20", 5, 5)
# Assert that the user exists
assert pwd.getpwnam(f'jupyter-{username}') is not None
@pytest.mark.asyncio
async def test_user_admin_add():
"""
User is made an admin, logs in and we check if they are in admin group
"""
# This *must* be localhost, not an IP
# aiohttp throws away cookies if we are connecting to an IP!
hub_url = 'http://localhost'
username = secrets.token_hex(8)
tljh_config_path = [sys.executable, '-m', 'tljh.config']
assert 0 == await (await asyncio.create_subprocess_exec(*tljh_config_path, 'add-item', 'users.admin', username)).wait()
assert 0 == await (await asyncio.create_subprocess_exec(*tljh_config_path, 'reload')).wait()
# FIXME: wait for reload to finish & hub to come up
# Should be part of tljh-config reload
await asyncio.sleep(1)
async with User(username, hub_url, partial(login_dummy, password='')) as u:
await u.login()
await u.ensure_server()
# Assert that the user exists
assert pwd.getpwnam(f'jupyter-{username}') is not None
# Assert that the user has admin rights
assert f'jupyter-{username}' in grp.getgrnam('jupyterhub-admins').gr_mem
@pytest.mark.asyncio
async def test_user_admin_remove():
"""
User is made an admin, logs in and we check if they are in admin group.
Then we remove them from admin group, and check they *aren't* in admin group :D
"""
# This *must* be localhost, not an IP
# aiohttp throws away cookies if we are connecting to an IP!
hub_url = 'http://localhost'
username = secrets.token_hex(8)
tljh_config_path = [sys.executable, '-m', 'tljh.config']
assert 0 == await (await asyncio.create_subprocess_exec(*tljh_config_path, 'add-item', 'users.admin', username)).wait()
assert 0 == await (await asyncio.create_subprocess_exec(*tljh_config_path, 'reload')).wait()
# FIXME: wait for reload to finish & hub to come up
# Should be part of tljh-config reload
await asyncio.sleep(1)
async with User(username, hub_url, partial(login_dummy, password='')) as u:
await u.login()
await u.ensure_server()
# Assert that the user exists
assert pwd.getpwnam(f'jupyter-{username}') is not None
# Assert that the user has admin rights
assert f'jupyter-{username}' in grp.getgrnam('jupyterhub-admins').gr_mem
assert 0 == await (await asyncio.create_subprocess_exec(*tljh_config_path, 'remove-item', 'users.admin', username)).wait()
assert 0 == await (await asyncio.create_subprocess_exec(*tljh_config_path, 'reload')).wait()
await asyncio.sleep(1)
await u.stop_server()
await u.ensure_server()
# Assert that the user does *not* have admin rights
assert f'jupyter-{username}' in grp.getgrnam('jupyterhub-admins').gr_mem

5
setup.py
View File

@@ -13,5 +13,10 @@ setup(
install_requires=[
'pyyaml==3.*',
'ruamel.yaml==0.15.*'
],
entry_points={
'console_scripts': [
'tljh-config = tljh.config:main'
]
}
)

135
tests/test_config.py Normal file
View File

@@ -0,0 +1,135 @@
"""
Test configuration commandline tools
"""
from tljh import config
from contextlib import redirect_stdout
import io
import pytest
import tempfile
def test_set_no_mutate():
conf = {}
new_conf = config.set_item_in_config(conf, 'a.b', 'c')
assert new_conf['a']['b'] == 'c'
assert conf == {}
def test_set_one_level():
conf = {}
new_conf = config.set_item_in_config(conf, 'a', 'b')
assert new_conf['a'] == 'b'
def test_set_multi_level():
conf = {}
new_conf = config.set_item_in_config(conf, 'a.b', 'c')
new_conf = config.set_item_in_config(new_conf, 'a.d', 'e')
new_conf = config.set_item_in_config(new_conf, 'f', 'g')
assert new_conf == {
'a': {'b': 'c', 'd': 'e'},
'f': 'g'
}
def test_set_overwrite():
"""
We can overwrite already existing config items.
This might be surprising destructive behavior to some :D
"""
conf = {
'a': 'b'
}
new_conf = config.set_item_in_config(conf, 'a', 'c')
assert new_conf == {'a': 'c'}
new_conf = config.set_item_in_config(new_conf, 'a.b', 'd')
assert new_conf == {'a': {'b': 'd'}}
new_conf = config.set_item_in_config(new_conf, 'a', 'hi')
assert new_conf == {'a': 'hi'}
def test_add_to_config_one_level():
conf = {}
new_conf = config.add_item_to_config(conf, 'a.b', 'c')
assert new_conf == {
'a': {'b': ['c']}
}
def test_add_to_config_zero_level():
conf = {}
new_conf = config.add_item_to_config(conf, 'a', 'b')
assert new_conf == {
'a': ['b']
}
def test_add_to_config_multiple():
conf = {}
new_conf = config.add_item_to_config(conf, 'a.b.c', 'd')
assert new_conf == {
'a': {'b': {'c': ['d']}}
}
new_conf = config.add_item_to_config(new_conf, 'a.b.c', 'e')
assert new_conf == {
'a': {'b': {'c': ['d', 'e']}}
}
def test_remove_from_config():
conf = {}
new_conf = config.add_item_to_config(conf, 'a.b.c', 'd')
new_conf = config.add_item_to_config(new_conf, 'a.b.c', 'e')
assert new_conf == {
'a': {'b': {'c': ['d', 'e']}}
}
new_conf = config.remove_item_from_config(new_conf, 'a.b.c', 'e')
assert new_conf == {
'a': {'b': {'c': ['d']}}
}
def test_remove_from_config_error():
with pytest.raises(ValueError):
config.remove_item_from_config({}, 'a.b.c', 'e')
with pytest.raises(ValueError):
config.remove_item_from_config({'a': 'b'}, 'a.b', 'e')
with pytest.raises(ValueError):
config.remove_item_from_config({'a': ['b']}, 'a', 'e')
def test_show_config():
"""
Test stdout output when showing config
"""
conf = """
# Just some test YAML
a:
b:
- h
- 1
""".strip()
with tempfile.NamedTemporaryFile() as tmp:
tmp.write(conf.encode())
tmp.flush()
out = io.StringIO()
with redirect_stdout(out):
config.show_config(tmp.name)
assert out.getvalue().strip() == conf

242
tljh/config.py Normal file
View File

@@ -0,0 +1,242 @@
"""
Commandline interface for setting config items in config.yaml.
Used as:
tljh-config set firstlevel.second_level something
tljh-config show
tljh-config show firstlevel
tljh-config show firstlevel.second_level
"""
import sys
import argparse
from ruamel.yaml import YAML
from copy import deepcopy
from tljh import systemd
yaml = YAML(typ='rt')
def set_item_in_config(config, property_path, value):
"""
Set key at property_path to value in config & return new config.
config is not mutated.
property_path is a series of dot separated values. Any part of the path
that does not exist is created.
"""
path_components = property_path.split('.')
# Mutate a copy of the config, not config itself
cur_part = config_copy = deepcopy(config)
for i, cur_path in enumerate(path_components):
cur_path = path_components[i]
if i == len(path_components) - 1:
# Final component
cur_part[cur_path] = value
else:
# If we are asked to create new non-leaf nodes, we will always make them dicts
# This means setting is *destructive* - will replace whatever is down there!
if cur_path not in cur_part or not isinstance(cur_part[cur_path], dict):
cur_part[cur_path] = {}
cur_part = cur_part[cur_path]
return config_copy
def add_item_to_config(config, property_path, value):
"""
Add an item to a list in config.
"""
path_components = property_path.split('.')
# Mutate a copy of the config, not config itself
cur_part = config_copy = deepcopy(config)
for i, cur_path in enumerate(path_components):
if i == len(path_components) - 1:
# Final component, it must be a list and we append to it
if cur_path not in cur_part or not isinstance(cur_part[cur_path], list):
cur_part[cur_path] = []
cur_part = cur_part[cur_path]
cur_part.append(value)
else:
# If we are asked to create new non-leaf nodes, we will always make them dicts
# This means setting is *destructive* - will replace whatever is down there!
if cur_path not in cur_part or not isinstance(cur_part[cur_path], dict):
cur_part[cur_path] = {}
cur_part = cur_part[cur_path]
return config_copy
def remove_item_from_config(config, property_path, value):
"""
Add an item to a list in config.
"""
path_components = property_path.split('.')
# Mutate a copy of the config, not config itself
cur_part = config_copy = deepcopy(config)
for i, cur_path in enumerate(path_components):
if i == len(path_components) - 1:
# Final component, it must be a list and we append to it
if cur_path not in cur_part or not isinstance(cur_part[cur_path], list):
raise ValueError(f'{property_path} is not a list')
cur_part = cur_part[cur_path]
cur_part.remove(value)
else:
if cur_path not in cur_part or not isinstance(cur_part[cur_path], dict):
raise ValueError(f'{property_path} does not exist in config!')
cur_part = cur_part[cur_path]
return config_copy
def show_config(config_path):
"""
Pretty print config from given config_path
"""
try:
with open(config_path) as f:
config = yaml.load(f)
except FileNotFoundError:
config = {}
yaml.dump(config, sys.stdout)
def set_config_value(config_path, key_path, value):
"""
Set key at key_path in config_path to value
"""
# FIXME: Have a file lock here
# FIXME: Validate schema here
try:
with open(config_path) as f:
config = yaml.load(f)
except FileNotFoundError:
config = {}
config = set_item_in_config(config, key_path, value)
with open(config_path, 'w') as f:
yaml.dump(config, f)
def add_config_value(config_path, key_path, value):
"""
Add value to list at key_path
"""
# FIXME: Have a file lock here
# FIXME: Validate schema here
try:
with open(config_path) as f:
config = yaml.load(f)
except FileNotFoundError:
config = {}
config = add_item_to_config(config, key_path, value)
with open(config_path, 'w') as f:
yaml.dump(config, f)
def reload_component(component):
"""
Reload a TLJH component.
component can be 'hub' or 'proxy'.
"""
if component == 'hub':
systemd.restart_service('jupyterhub')
# FIXME: Verify hub is back up?
print('Hub reload with new configuration complete')
elif component == 'proxy':
systemd.restart_service('configurable-http-proxy')
print('Proxy reload with new configuration complete')
def main():
argparser = argparse.ArgumentParser()
argparser.add_argument(
'--config-path',
default='/opt/tljh/config.yaml',
help='Path to TLJH config.yaml file'
)
subparsers = argparser.add_subparsers(dest='action')
show_parser = subparsers.add_parser(
'show',
help='Show current configuration'
)
set_parser = subparsers.add_parser(
'set',
help='Set a configuration property'
)
set_parser.add_argument(
'key_path',
help='Dot separated path to configuration key to set'
)
set_parser.add_argument(
'value',
help='Value ot set the configuration key to'
)
add_item_parser = subparsers.add_parser(
'add-item',
help='Add a value to a list for a configuration property'
)
add_item_parser.add_argument(
'key_path',
help='Dot separated path to configuration key to add value to'
)
add_item_parser.add_argument(
'value',
help='Value to add to the configuration key'
)
remove_item_parser = subparsers.add_parser(
'remove-item',
help='Remove a value from a list for a configuration property'
)
remove_item_parser.add_argument(
'key_path',
help='Dot separated path to configuration key to remove value from'
)
remove_item_parser.add_argument(
'value',
help='Value to remove from key_path'
)
reload_parser = subparsers.add_parser(
'reload',
help='Reload a component to apply configuration change'
)
reload_parser.add_argument(
'component',
choices=('hub', 'proxy'),
help='Which component to reload',
default='hub',
nargs='?'
)
args = argparser.parse_args()
if args.action == 'show':
show_config(args.config_path)
elif args.action == 'set':
set_config_value(args.config_path, args.key_path, args.value)
elif args.action == 'add-item':
add_config_value(args.config_path, args.key_path, args.value)
elif args.action == 'remove-item':
add_config_value(args.config_path, args.key_path, args.value)
elif args.action == 'reload':
reload_component(args.component)
if __name__ == '__main__':
main()