This article is a follow on from Realtime Chat with Vite, Vue3 and Python Tornado. But my son said this should have come first. In that article I suggested that I could provide a view of pylint, axblack, pytest and sphinx. I use Atom, but others are excellent too, such as VS Code, Eclipse and PyCharm. Atom has plug-ins to make it a usable python ide: language-python, python-black, linter-pylint, platformio-ide-terminal, to name but a few. What we'll do here is in the terminal, so any editor will do.
By engineering I mean tested, consistent, documented code. So let's start setting up some command line tools to make this less daunting.
In this article I'll be concentrating on python and cover javascript in another article when I address my appalling gui.
To place these tools into our environment we'll adapt our use of pip. We'll create two files to express the development and production configurations of python: dev.txt and requirements.txt. Dev will be used by our Makefile and Requirements by our Dockerfile.
requirements.txt
tornado
dev.txt
-r requirements.txt
invoke
pytest
pytest-coverage
pytest-benchmark
pytest-tornasync
pylint
axblack
pyyaml
sphinx
recommonmark
sphinx-autoapi
bumpversion
wheel
twine
nodeenv
NB. you should be putting version numbers beside these requirement packages. At the time of writing I was actually using tornado==6.1 — pip freeze will tell you what you've actually got. We'll lock them down later.
Makefile
setup:
if which python3 && [ ! -d venv ] ; then python3 -m venv venv ; fi
source venv/bin/activate \
&& python -m pip install -q -U pip \
&& pip install -r dev.txt \
&& if [ ! -d nenv ] ; then nodeenv nenv; fi
We've told pip to use our dev.txt which includes our requirements.txt - no duplication wherever possible. By running make setup, your environment will be updated to contain the new packages.
Dockerfile
FROM python:3-alpine as base
FROM base as builder
RUN mkdir /install
WORKDIR /install
COPY requirements.txt /requirements.txt
RUN pip install --prefix=/install -r /requirements.txt
FROM base
COPY --from=builder /install /usr/local
COPY . /app
WORKDIR /app
# Make port 80 available to the world outside this container
EXPOSE 80
CMD ["python", "-m", "chat.main", "--port=80"]
I alluded to docker in the previous article. It's simple to setup with two files: Dockerfile, docker-compose.yml. The first describes your container and the second enables you to run it locally.
Our Docker file uses multi-phase build to extend the base python image with our requirements.txt and then layered atop is our app. We expose a web port and specify our command to run the app.
docker-compose.yml
version: '3'
services:
web:
build: .
volumes:
- .:/app
image: chat:latest
container_name: chat
ports:
- "8080:80"
command: python -m chat.main --port=80 --debug=true
Here we declare our build and run-time environment. We map our project directory to our container's app directory for hot-reloading and map a local port to a container port so that our browsers can see the app. We also override the command with a debug option so we can see our changes should we make them.
To run the container: docker-compose build, followed by docker-compose up.
Invoke is a marvellous tool for those who cannot remember command line interfaces. You can write simple macros to do all your dev tasks. Just create a tasks.py in the root of your project:
tasks.py
""" our dev tasks """
from invoke import task
@task(help={'debug': 'run in hot-reload mode'})
def server(ctx, debug=False):
""" run our python server """
ctx.run(f'python -m chat.main --debug={debug}')
@task
def client(ctx):
""" run our vite server """
ctx.run('. nenv/bin/activate && cd client && npm run dev')
@task
def build(ctx):
""" build our vue client """
ctx.run('. nenv/bin/activate && cd client && npm run build')
@task
def docker(ctx):
""" build & run our docker server """
ctx.run('docker-compose build')
ctx.run('docker-compose up')
ctx.run('docker-compose down')
Activate your venv and you run invoke -l or the short-cut inv -l. You'll see the help text.
% . venv/bin/activate
% inv -l
Available tasks:
build build our vue client
client run our vite server
docker build & run our docker server
server run our python server
Notice that invoke is self documenting - try inv --help server. If we run our server now, it has a flag option for debug so we can run in debug mode by calling inv server -d.
Using all the tools we've installed we need to configure them. Each tool has its own config file, but most of them also look for setup.cfg.
[pylint.'MESSAGES CONTROL']
disable=
too-few-public-methods,
abstract-method
[isort]
line_length = 79
[tool:pytest]
addopts = -p no:warnings --capture=sys --cov chat --cov-report term-missing
filterwarnings = ignore::DeprecationWarning
testpaths = tests
This file holds the configurations for pylint, axblack and pytest. So let's use pylint and axblack.
black is a wonderful tool. Type away and it will clean up after you. Sadly, black itself goes for double quotes only. We live in a json world and snippets live in single quotes. So axblack gives you the opinionation of black, but uses double quotes for comments and single quotes for code.
We just add a task to tasks.py:
@task
def lint(ctx):
""" run axblack and pylint """
ctx.run('black chat')
ctx.run('pylint chat')
And to run it: inv lint. You should find unchanged files and 10.00/10! If not - fix it! Fixing it means either live with the badness and add to the setup.cfg or conform. Conforming is a contortion but following through I've found interesting changes to my code style, if nothing else.
Pytest is the swiss army knife of python testing tools. With the extension in our dev.txt we will be able to test, determine coverage and performance, and we have helpers for Tornado and Asyncio code.
First create a tests package and add a module: tests/conftests.py. This file will contain our fixtures - pytest's useful reusables for setting up tests.
conftest.py
""" our test fixtures """
import pytest
from tornado.httpclient import HTTPRequest
from tornado.websocket import websocket_connect
from chat.main import make_app
@pytest.fixture
def app():
""" return a tornado application """
return make_app()
@pytest.fixture
async def ws_client(http_server, http_server_port):
""" return a websocket client """
request = HTTPRequest(f'ws://localhost:{http_server_port[1]}/ws')
result = await websocket_connect(request)
return result
We want to test our websocket and to do that we'll need an app fixture as required by pytest-tornasync. We also return the result of tornado's websocket_connect.
tests/test_ws.py
""" test websocket client """
import asyncio
async def test_ws(ws_client):
""" test message send and receive """
message = 'hello, world'
client = await ws_client
await client.write_message(message)
response = await client.read_message()
print(response)
assert response == message
client.close()
await asyncio.sleep(0.01)
To run the test call: % pytest
Now that we can run and test our code, let's document it. Sphinx is already in the dev.txt with some extensions. To begin our sphinx project we'll call:
% mkdir docsrc
% cd docsrc
% sphinx quickstart
And answer all the questions (NB. defaults are good). First we have to add our extensions to docsrc/conf.py:
extensions = [
'sphinx.ext.autodoc',
'recommonmark',
'sphinx.ext.viewcode',
'autoapi.extension',
]
And at the bottom, configure our extensions:
html_theme_options = {'logo': 'favicon.ico'}
autoapi_dirs = ['../chat']
autoapi_type = 'python'
autodoc_typehints = 'description'
autoapi_add_toctree_entry = False
Let's bring it all together with our docker task.
@task(pre=[lint, build, test, docs_build])
def docker(ctx):
""" build & run our docker server """
ctx.run('docker-compose build')
ctx.run('docker-compose up')
ctx.run('docker-compose down')
Calling inv docker will build, test it all, and then run the docker container. You should see the chat site at http://localhost:8080.
For every choice made here there are a myriad of alternatives available. These are merely the expression of what has stuck with me through my practice. As I continue to code, my nosiness and laziness will continue to evolve a combination of tools to say: "this is cool!" or "I couldn't be bothered".
Next time, we'll add authentication and persistence with SQLAlchemy.
The source is on https://github.com/blueshed/chat/tree/article2