MUSE for Anything

Code style: black GitHub license Python: >= 3.7 Documentation Status

The current developer documentation can be found here: https://muse4anything.readthedocs.io/.

This package uses Poetry (documentation).

This package builds on the flask template https://github.com/buehlefs/flask-template

Docker

A docker image is built automatically. Docker images can be found under Packages.

VSCode

For vscode install the python extension and add the poetry venv path to the folders the python extension searches for venvs.

On linux:

{
    "python.venvFolders": [
        "~/.cache/pypoetry/virtualenvs"
    ]
}

Development

Run poetry install to install dependencies.

The flask dev server loads environment variables from .flaskenv and .env. To override any variable create a .env file. Environment variables in .env take precedence over .flaskenv. See the content of the .flaskenv file for the default environment variables.

Before the first start, be sure to create a database:

poetry run flask db upgrade

# create user admin:admin
poetry run flask create-admin-user

Run the development server with

poetry run flask run

To compile the frontend run the following commands in the muse-for-anything-ui folder.

npm install # update/install dependencies

# watching build
npm run watch
# normal build
npm run build

Libraries and special files/folders

This package uses the following libraries to build a rest app with a database on top of flask.

  • Flask (documentation)

  • Flask-Cors (documentation)
    Used to provide cors headers.
    Can be configured or removed in muse_for_anything/__init__.py.

  • flask-babel (documentation, babel documentation)
    Used to provide translations.
    Can be configured in muse_for_anything/babel.py and babel.cfg.
    Translation files and Folders: translations (and messages.pot currently in .gitignore)

  • Flask-SQLAlchemy (documentation, SQLAlchemy documentation)
    ORM Mapper for many SQL databases.
    Models: muse_for_anything/db/models
    Config: muse_for_anything/util/config/sqlalchemy_config.py and muse_for_anything/db/db.py

  • Flask-Migrate (documentation, Alembic documentation)
    Provides automatic migration support based on alembic.
    Migrations: migrations

  • flask-smorest (documentation, marshmallow documentation, apispec documentation, OpenAPI spec)
    Provides the API code and generates documentation in form of a OpenAPI specification.
    API: muse_for_anything/api
    API Models: muse_for_anything/api/v1_api/models
    Config: muse_for_anything/util/config/smorest_config.py and muse_for_anything/api/__init__.py

  • Flask-JWT-Extended (documentation)
    Provides authentication with JWT tokens.
    Config: muse_for_anything/util/config/smorest_config.py and muse_for_anything/api/jwt.py

  • Sphinx (documentation)
    The documentation generator.
    Config: pyproject.toml and docs/conf.py (toml config input is manually configured in conf.py)

  • sphinxcontrib-redoc (documantation) Renders the OpenAPI spec with redoc in sphinx html output. Config: docs/conf.py (API title is read from spec)

  • invoke (documentation)
    tool for scripting cli tasks in python Tasks: tasks.py

  • flask-static-digest (documentation)
    For serving the resources of a javascript SPA

Additional files and folders:

  • muse-for-anything-ui
    Aurelia frontend for the API.

  • default.nix and shell.nix
    For use with the nix ecosystem.

  • pyproject.toml
    Poetry package config and config for the black formatter.

  • .flaskenv
    Environment variables loaded by the flask command and the flask dev server.

  • .flake8
    Config for the flake8 linter

  • .editorconfig

  • tests
    Reserved for unit tests, this template has no unit tests.

  • instance (in .gitignore)

  • muse_for_anything/templates and muse_for_anything/static (currently empty)
    Templates and static files of the flask app

  • docs
    Folder containing a sphinx documentation

  • typings
    Python typing stubs for libraries that have no type information. Mostly generated with the pylance extension of vscode.

  • tasks.py
    Tasks that can be executed with invoke (see invoke tasks)

Library alternatives or recommendations:

Poetry Commands

# install dependencies from lock file in a virtualenv
poetry install

# open a shell in the virtualenv
poetry shell

# update dependencies
poetry update
poetry run invoke update-dependencies # to update other dependencies in the repository

# run a command in the virtualenv (replace cmd with the command to run without quotes)
poetry run cmd

Invoke Tasks

Invoke is a python tool for scripting cli commands. It allows to define complex commands in simple python functions in the tasks.py file.

:warning: Make sure to update the module name in tasks.py after renaming the flask_template module!

# list available commands
poetry run invoke --list

# update dependencies (requirements.txt in ./docs and licenses template)
poetry run invoke update-dependencies

# Compile the documentation
poetry run invoke doc

# Open the documentation in the default browser
poetry run invoke browse-doc

Babel

# initial
poetry run pybabel extract -F babel.cfg -o messages.pot .
# create language
poetry run pybabel init -i messages.pot -d translations -l en
# compile translations to be used
poetry run pybabel compile -d translations
# extract updated strings
poetry run pybabel update -i messages.pot -d translations

SQLAlchemy

# create dev db (this will NOT run migrations!)
poetry run flask create-db
# create an admin user
poetry run flask create-admin-user
# drop dev db
poetry run flask drop-db

This tool uses select IN eager loading which is incompatible with SQL Server and SQLite < 3.15.

Migrations

# create a new migration after changes in the db (Always manually review the created migration!)
poetry run flask db migrate -m "Initial migration."
# upgrade db to the newest migration
poetry run flask db upgrade
# help
poetry run flask db --help

Compiling the Documentation

# compile documentation
poetry run invoke doc

# Open the documentation in the default browser
poetry run invoke browse-doc

# Find reference targets defined in the documentation
poetry run invoke doc-index --filter=searchtext

# export/update requirements.txt from poetry dependencies (for readthedocs build)
poetry run invoke update-dependencies