Development
To set up a development environment for the API and app, make sure you have a relatively recent version of Docker and Make on your development machine (and that you’ve added your daily user to the docker group). For the agent, make sure you have pyenv with all the current versions of Python installed.
API
Clone the official Git repository at https://git.sr.ht/~arx10/procustodibus-api. All the development scripts assume that you will run the database and API via the dev docker-compose.yml file in a custom Docker network named custos (using the 10.242.4.0/24 address block).
Init
In your local procustodibus-api/ directory, run the make init command to start up the database and build the dev Docker image:
$ make init TODO
The dev Docker container builds the necessary version of Erlang and Elixir from scratch (it will take 10+ minutes to complete). All the other Make tasks run in this dev Docker container.
Once this command has completed, run the following command to generate a random application-level encryption key:
$ openssl rand -base64 32 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
Copy the random key, open up the .env file (generated by the make init command), and paste it in as the value for the DB_ALEK_1 environment variable:
# .env
DB_ALEK_1="value:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="Test
To run the full linting and test suite, run the make check command (which will take a minute or two to complete):
$ make check docker-compose run api mix format --check-formatted Creating api_api_run ... done docker-compose run api mix compile --all-warnings --warnings-as-errors Creating api_api_run ... done docker-compose run api mix credo --strict Creating api_api_run ... done Checking 365 source files (this might take a while) ... Please report incorrect results: https://github.com/rrrene/credo/issues Analysis took 17.1 seconds (2.5s to load, 14.6s running 67 checks on 365 files) 3756 mods/funs, found no issues. Use `mix credo explain` to explain issues, `mix credo --help` for options. docker-compose run api mix sobelow --config Creating api_api_run ... done Checking Sobelow version... A new version of Sobelow is available: mix archive.install hex sobelow ############################################## # # # Running Sobelow - v0.12.2 # # Created by Griffin Byatt - @griffinbyatt # # NCC Group - https://nccgroup.trust # # # ############################################## Config.HTTPS: HTTPS Not Enabled - High Confidence HTTPS configuration details could not be found in `prod.exs`. ----------------------------------------------- ... SCAN COMPLETE ... docker-compose run api mix test Creating api_api_run ... done Excluding tags: [:aws] .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Finished in 43.9 seconds (23.3s async, 20.6s sync) 58 doctests, 3570 tests, 0 failures, 2 excluded Randomized with seed 450375
Watch
To run a specific unit test file every time you make a change to the source code, run the make watch command, specifying the unit test file to run with the TEST option:
$ make watch TEST=test/api_web/controllers/page_controller_test.exs
docker-compose run shell mix test.watch test/api_web/controllers/page_controller_test.exs
Creating api_shell_run ... done
Running tests...
Compiling 16 files (.ex)
Generated api app
Excluding tags: [:aws]
1) test index GET / (ApiWeb.PageControllerTest)
test/api_web/controllers/page_controller_test.exs:5
Assertion with == failed
code: assert conn |> get(~p"/") |> json_response(200) == %{}
left: %{"foo" => "bar"}
right: %{}
stacktrace:
test/api_web/controllers/page_controller_test.exs:6: (test)
.
Finished in 0.09 seconds (0.00s async, 0.09s sync)
2 tests, 1 failure
Randomized with seed 498117
Server
The make dev command will run the API server, with an interactive IEx shell running in your current terminal:
$ make dev
docker-compose run --use-aliases api iex -S mix phx.server
Creating api_api_run ... done
00:07:41.582 application=swoosh mfa=Swoosh.Application.runtime_children/2 [info] Running Swoosh mailbox preview server with Cowboy using http on port 4001
00:07:41.999 application=phoenix mfa=Phoenix.Endpoint.Cowboy2Adapter.start_link/3 [info] Running ApiWeb.Endpoint with cowboy 2.10.0 at 0.0.0.0:4000 (http)
00:07:42.010 application=phoenix mfa=Phoenix.Endpoint.Supervisor.log_access_url/2 [info] Access ApiWeb.Endpoint at http://10.242.4.242:4000
00:07:42.061 application=ecto_sql mfa=Ecto.Adapters.SQL.log/4 [debug] QUERY OK db=2.7ms decode=2.6ms queue=1.1ms idle=57.2ms
INSERT INTO "system_recurring_tasks" ("name","node") VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING "id" ["alert_endpoint_stats", ""]
00:07:42.072 application=ecto_sql mfa=Ecto.Adapters.SQL.log/4 [debug] QUERY OK db=1.2ms queue=1.3ms idle=78.0ms
INSERT INTO "system_recurring_tasks" ("name","node") VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING "id" ["alert_log_authn", ""]
00:07:42.076 application=ecto_sql mfa=Ecto.Adapters.SQL.log/4 [debug] QUERY OK db=1.5ms queue=1.2ms idle=80.8ms
INSERT INTO "system_recurring_tasks" ("name","node") VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING "id" ["alert_log_authz", ""]
00:07:42.080 application=ecto_sql mfa=Ecto.Adapters.SQL.log/4 [debug] QUERY OK db=2.0ms queue=2.0ms idle=84.3ms
INSERT INTO "system_recurring_tasks" ("name","node") VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING "id" ["alert_log_changes", ""]
00:07:42.086 application=ecto_sql mfa=Ecto.Adapters.SQL.log/4 [debug] QUERY OK db=2.1ms queue=2.5ms idle=89.1ms
INSERT INTO "system_recurring_tasks" ("name","node") VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING "id" ["cache_geoip_uses", ""]
00:07:42.089 application=ecto_sql mfa=Ecto.Adapters.SQL.log/4 [debug] QUERY OK db=1.0ms queue=1.2ms idle=94.2ms
INSERT INTO "system_recurring_tasks" ("name","node") VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING "id" ["post_alerts_hook", ""]
00:07:42.091 application=ecto_sql mfa=Ecto.Adapters.SQL.log/4 [debug] QUERY OK db=1.1ms queue=1.1ms idle=96.7ms
INSERT INTO "system_recurring_tasks" ("name","node") VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING "id" ["post_log_authn_hook", ""]
00:07:42.094 application=ecto_sql mfa=Ecto.Adapters.SQL.log/4 [debug] QUERY OK db=1.5ms queue=1.1ms idle=99.3ms
INSERT INTO "system_recurring_tasks" ("name","node") VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING "id" ["post_log_changes_hook", ""]
00:07:42.098 application=ecto_sql mfa=Ecto.Adapters.SQL.log/4 [debug] QUERY OK db=1.7ms queue=1.8ms idle=102.3ms
INSERT INTO "system_recurring_tasks" ("name","node") VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING "id" ["post_endpoint_stats_hook", ""]
00:07:42.101 application=ecto_sql mfa=Ecto.Adapters.SQL.log/4 [debug] QUERY OK db=1.1ms queue=1.6ms idle=104.6ms
INSERT INTO "system_recurring_tasks" ("name","node") VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING "id" ["check_hung_ldap_polls", ""]
00:07:42.102 application=ecto_sql mfa=Ecto.Adapters.SQL.log/4 [debug] QUERY OK db=0.4ms idle=48.6ms
INSERT INTO "system_recurring_tasks" ("name","node") VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING "id" ["rotate_preshared_keys", ""]
00:07:42.103 application=ecto_sql mfa=Ecto.Adapters.SQL.log/4 [debug] QUERY OK db=0.3ms idle=29.6ms
INSERT INTO "system_recurring_tasks" ("name","node") VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING "id" ["update_sessions_last_used", ""]
Erlang/OTP 26 [erts-14.0.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [jit:ns]
Interactive Elixir (1.15.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Users.get_user!(1) |> Map.get(:name)
"System User"
iex(2)>
In a different terminal, run the following command to try sending a request to the API:
$ curl -w'\n' http://10.242.4.242:4000/health
[
{
"error": null,
"healthy": true,
"name": "DB",
"time": 4581
}
]
If you make changes to the API source code, it will for the most part reload automatically the next time you make a web request. For example, open up the lib/api_web/controllers/page_controller.ex file, and change its index/2 function to return some dummy JSON value:
# lib/api_web/controllers/page_controller.ex
defmodule ApiWeb.PageController do
use ApiWeb, :controller
def index(conn, _params) do
conn
|> cache_control("public, max-age=3600")
# change this line to return some dummy value like %{foo: "bar"}:
|> json(%{foo: "bar"})
end
# ...
endcURL the controller action you just changed to view the results of your change:
$ curl -w'\n' http://10.242.4.242:4000/
{"foo":"bar"}
Shell
To run arbitrary Mix tasks (such as mix format), start a shell for the dev Docker image with the make shell command:
$ make shell docker-compose run shell /bin/bash Creating api_shell_run ... done jose@dd383b95cd32:/srv/api$ mix format jose@dd383b95cd32:/srv/api$
Database
To run arbitrary SQL queries in the database, start a shell for the database with the make shell.db command:
$ make shell.db docker-compose up -d db Creating network "custos" with the default driver Creating api_db_1 ... done docker exec -u postgres -it api_db_1 psql psql (13.11) Type "help" for help. postgres=#
Run the \c custos command to switch to the dev database; then you can run SQL queries on the dev data:
postgres=# \c custos You are now connected to database "custos" as user "postgres". custos=# select * from users; id | pub_id | active | name | org_id | inserted_at | updated_at | type | locked ----+---------------------+--------+-------------+--------+------------------------+------------------------+-------+-------- 1 | 6620344162125037829 | t | System User | 1 | 2020-01-01 00:00:00+00 | 2020-01-01 00:00:00+00 | admin | f (1 row)
Directories
Here’s a quick tour of the directories in the API project:
-
assets: static files served by the API (such asfavicon.ico) -
bin: miscellaneous shell scripts -
_build: output of build scripts -
build: build scripts -
config: standard Elixir config directory -
ee: Enterprise Edition code and tests -
lib: standard Elixir code directory -
me: your private scratch directory -
ops: scripts run on deployed systems -
priv: standard Elixir resources directory -
test: standard Elixir tests directory
App
Clone the official Git repository at https://git.sr.ht/~arx10/procustodibus-app. All the development scripts assume that you will run the app via the dev docker-compose.yml file in a custom Docker network named custos_app (using the 10.242.5.0/28 address block).
Init
In your local procustodibus-app/ directory, run the make init command to build the dev Docker image:
$ make init TODO
Test
To run the full linting and test suite, run the make check command (which will take a minute or two to complete):
$ make check
docker-compose run app npm run lint
Creating app_app_run ... done
> custos@1.0.0 lint
> npm run lint-js && npm run lint-css && npm run format-check
> custos@1.0.0 lint-js
> eslint --ext .vue,.js,.jsx,.cjs,.mjs --report-unused-disable-directives --ignore-path .gitignore --cache --cache-location out/ .
> custos@1.0.0 lint-css
> stylelint "src/**/*.{css,less,sass,scss,sss,styl,vue}" --report-needless-disables --report-invalid-scope-disables --ignore-path .gitignore --cache --cache-location out/
> custos@1.0.0 format-check
> prettier --check src/ tests/
Checking formatting...
All matched files use Prettier code style!
docker-compose run app npm run test
Creating app_app_run ... done
> custos@1.0.0 test
> vitest run --coverage
RUN v0.34.1 /srv/app
Coverage enabled with v8
stderr | tests/unit/utils/text.spec.js > decodeBase64() > returns blank if not base64 encoded
cannot decode as base64 utf-8: 12345
stderr | tests/unit/utils/text.spec.js > decodeBase64Url() > returns blank if not base64 encoded
cannot decode as base64 utf-8: 12345
✓ tests/unit/utils/background.spec.js (28) 6773ms
✓ tests/unit/utils/login.spec.js (34)
✓ tests/unit/utils/rdap.spec.js (10) 320ms
✓ tests/unit/utils/date.spec.js (73)
✓ tests/unit/utils/axios.spec.js (11)
✓ tests/unit/utils/text.spec.js (74)
✓ tests/unit/utils/model.spec.js (22)
✓ tests/unit/utils/env.spec.js (15)
✓ tests/unit/utils/session.spec.js (13)
✓ tests/unit/utils/timeline.spec.js (40)
✓ tests/unit/stores/user.spec.js (10)
✓ tests/unit/utils/download.spec.js (17)
✓ tests/unit/stores/mfa.spec.js (10)
✓ tests/unit/utils/misc.spec.js (26)
✓ tests/unit/utils/error.spec.js (5)
✓ tests/unit/utils/form.spec.js (8)
✓ tests/unit/utils/message.spec.js (6)
✓ tests/unit/utils/pageantry.spec.js (16)
✓ tests/unit/utils/storage.spec.js (4)
Test Files 19 passed (19)
Tests 422 passed (422)
Start at 01:35:22
Duration 11.85s (transform 618ms, setup 4ms, collect 3.48s, tests 7.74s, environment 13.61s, prepare 2.65s)
% Coverage report from v8
-----------------------|---------|----------|---------|---------|-----------------------------------------------------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-----------------------|---------|----------|---------|---------|-----------------------------------------------------------------
All files | 95.37 | 91.33 | 95.23 | 95.37 |
components/app | 100 | 100 | 100 | 100 |
app-confirm-form.vue | 100 | 100 | 100 | 100 |
app-message.vue | 100 | 100 | 100 | 100 |
app-modal-form.vue | 100 | 100 | 100 | 100 |
stores | 99.14 | 80.21 | 100 | 99.14 |
features.js | 100 | 75 | 100 | 100 | 28
mfa.js | 100 | 100 | 100 | 100 |
prefs.js | 83.33 | 66.66 | 100 | 83.33 | 8-10
session.js | 100 | 86.53 | 100 | 100 | 135-141,193-197
user.js | 100 | 52.63 | 100 | 100 | 34-36,39-40,45,49,57-58
utils | 94.68 | 92.74 | 94.47 | 94.68 |
axios.js | 89.44 | 86.36 | 83.33 | 89.44 | 45-57,94-97,131-133,199-201
background.js | 95.45 | 96.96 | 100 | 95.45 | 14-21
date.js | 100 | 99.11 | 100 | 100 | 75
download.js | 97.19 | 94 | 92.85 | 97.19 | 16-21,157,159
env.js | 98.29 | 100 | 92.3 | 98.29 | 77-78
error.js | 80.21 | 56.09 | 88.88 | 80.21 | 102-123,133,137,139,141,143,145,149,163,165,167,169,172,183-185
form.js | 82.53 | 100 | 66.66 | 82.53 | 29-39
login.js | 98.84 | 91.8 | 100 | 98.84 | 232,234-235
message.js | 100 | 100 | 100 | 100 |
misc.js | 100 | 100 | 100 | 100 |
model.js | 80.45 | 83.33 | 91.66 | 80.45 | 40-73,83,85,87,89,91,93,95,97,99
pageantry.js | 100 | 100 | 100 | 100 |
rdap.js | 100 | 95.45 | 100 | 100 | 15
storage.js | 100 | 83.33 | 100 | 100 | 19
text.js | 94.87 | 100 | 92.59 | 94.87 | 303-315,323-327
timeline.js | 100 | 91.25 | 100 | 100 | 78,159,217,233,236,272-273
-----------------------|---------|----------|---------|---------|-----------------------------------------------------------------
Watch
To run all the tests every time you make a change to the source code, run the make watch command:
$ make watch
...
Test Files 19 passed (19)
Tests 422 passed (422)
Start at 01:38:43
Duration 13.81s (transform 573ms, setup 3ms, collect 2.57s, tests 7.43s, environment 10.04s, prepare 1.95s)
PASS Waiting for file changes...
press h to show help, press q to quit
For the most part, only affected tests will re-run when you make a change.
Press h while this command is waiting to see several options for filtering the tests that are run.
|
Server
The make dev command will run the dev Vite server, serving a dev build of the app UI:
$ make dev VITE v4.4.8 ready in 816 ms ➜ Local: http://localhost:5173/ ➜ Network: http://10.242.5.12:5173/ ➜ press h to show help
In a different terminal, run the following command to verify the dev server is up and running:
$ curl -sS http://10.242.5.12:5173/ | head
<!DOCTYPE html>
<html lang="en">
<head>
<script type="module" src="/@vite/client"></script>
<meta charset="UTF-8">
<link rel="icon" href="/favicon.ico">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Pro Custodibus</title>
<link rel="apple-touch-icon" href="/img/icons/apple-touch-icon-180x180.png" sizes="180x180">
If you make changes to the app source code, it will for the most part reload automatically live in your web browser. For example, open up the src/components/nav/nav-header.vue file, and change the name of the logo.svg image to some dummy value:
<!-- src/components/nav/nav-header.vue -->
<template>
<header class="header">
<nav-navbar class="container" transparent>
<template #brand>
<nav-navbar-item tag="a" :href="`${env.wwwUrl}/`">
<!-- change this line to use some dummy image name like foo.svg: -->
<img src="@/assets/foo.svg" :alt="env.title" />
</nav-navbar-item>
</template>
<!-- ... -->If you have the app opened up in a web browser, you’ll see the logo image in the page header has disappeared, and replaced with its alt text (“Local Pro Custodibus”).
Shell
To run arbitrary NPM commands (such as npm outdated), start a shell with the dev Docker image with the following command:
$ make shell docker-compose run -u $(id -u) shell /bin/bash Creating app_shell_run ... done node@d6b679ca310c:/srv/app$ npm outdated Package Current Wanted Latest Location Depended by axios-extensions 3.1.7 3.1.7 3.1.6 node_modules/axios-extensions app
Directories
Here’s a quick tour of the directories in the app project:
-
bin: miscellaneous shell scripts -
build: build scripts -
coverage: coverage output -
ee: Enterprise Edition code and tests -
node_modules: standard Node.js dependencies directory -
ops: scripts run on deployed systems -
out: build output -
public: static files served by the app (such asfavicon.ico) -
src: source code-
src/assets: static files directly included by the source code -
src/components: Vue.js components -
src/mixins: Vue.js mixins and composables -
src/pages: Vue.js pages -
src/plugins: Vue.js plugins -
src/stores: Pinia stores -
src/styles: SCSS stylesheets -
src/utils: generic JavaScript utility functions
-
-
tests: unit tests
Agent
Clone the official Git repository at https://git.sr.ht/~arx10/procustodibus-agent. All the development scripts assume that you will use tox for development in an Python virtualenv.
Init
In your local procustodibus-agent/ directory, create a virtualenv with pyenv:
$ pyenv virtualenv 3.10.12 procustodibus-agent
Then activate the virtualenv:
$ pyenv local procustodibus-agent 3.7.17 3.8.17 3.9.17 3.10.12 3.11.4
And install tox into the virtualenv:
(procustodibus-agent) $ pip install tox
Collecting tox
Downloading tox-4.9.0-py3-none-any.whl (152 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 152.8/152.8 kB 2.9 MB/s eta 0:00:00
Collecting packaging>=23.1
Using cached packaging-23.1-py3-none-any.whl (48 kB)
Collecting tomli>=2.0.1
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting cachetools>=5.3.1
Downloading cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Collecting colorama>=0.4.6
Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting pyproject-api>=1.5.3
Downloading pyproject_api-1.5.4-py3-none-any.whl (12 kB)
Collecting pluggy>=1.2
Using cached pluggy-1.2.0-py3-none-any.whl (17 kB)
Collecting platformdirs>=3.10
Downloading platformdirs-3.10.0-py3-none-any.whl (17 kB)
Collecting filelock>=3.12.2
Using cached filelock-3.12.2-py3-none-any.whl (10 kB)
Collecting virtualenv>=20.24.3
Downloading virtualenv-20.24.3-py3-none-any.whl (3.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.0/3.0 MB 44.3 MB/s eta 0:00:00
Collecting chardet>=5.2
Downloading chardet-5.2.0-py3-none-any.whl (199 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.4/199.4 kB 20.2 MB/s eta 0:00:00
Collecting distlib<1,>=0.3.7
Downloading distlib-0.3.7-py2.py3-none-any.whl (468 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.9/468.9 kB 32.4 MB/s eta 0:00:00
Installing collected packages: distlib, tomli, pluggy, platformdirs, packaging, filelock, colorama, chardet, cachetools, virtualenv, pyproject-api, tox
Successfully installed cachetools-5.3.1 chardet-5.2.0 colorama-0.4.6 distlib-0.3.7 filelock-3.12.2 packaging-23.1 platformdirs-3.10.0 pluggy-1.2.0 pyproject-api-1.5.4 tomli-2.0.1 tox-4.9.0 virtualenv-20.24.3
Finally, install the pre-commit and pre-push hooks:
(procustodibus-agent) $ tox -e pre-commit -- install pre-commit: install_deps> python -I -m pip install pre-commit pre-commit: commands[0]> pre-commit install pre-commit installed at .git/hooks/pre-commit pre-commit: OK (3.89=setup[3.67]+cmd[0.22] seconds) congratulations :) (7.41 seconds) (procustodibus-agent) $ tox -e pre-commit -- install -t pre-push pre-commit: commands[0]> pre-commit install -t pre-push pre-commit installed at .git/hooks/pre-push pre-commit: OK (0.18=setup[0.01]+cmd[0.18] seconds) congratulations :) (1.48 seconds)
Test
Simply run the tox command to run the full linting and test suite (which will take several minutes to complete):
(procustodibus-agent) $ tox
.pkg-cpython37: install_requires> python -I -m pip install 'setuptools>=40.8.0' wheel
.pkg-cpython37: _optional_hooks> python /home/me/source/pyenv/versions/3.10.9/envs/procustodibus-agent/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg-cpython37: get_requires_for_build_editable> python /home/me/source/pyenv/versions/3.10.9/envs/procustodibus-agent/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg-cpython37: install_requires_for_build_editable> python -I -m pip install wheel
.pkg-cpython37: build_editable> python /home/me/source/pyenv/versions/3.10.9/envs/procustodibus-agent/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
py37: install_package_deps> python -I -m pip install 'docopt-ng~=0.7' 'inflection~=0.5' 'pynacl~=1.5' pytest pytest-cov pytest-datadir pytest-mock pytest-timeout pytest-watch 'requests~=2.27' 'tabulate~=0.8'
py37: install_package> python -I -m pip install --force-reinstall --no-deps /home/me/procustodibus-agent/.tox/.tmp/package/1/procustodibus_agent-1.3.3-0.editable-py3-none-any.whl
py37: commands[0]> pytest
============================================================================================================= test session starts =============================================================================================================
platform linux -- Python 3.7.16, pytest-7.4.0, pluggy-1.2.0
cachedir: .tox/py37/.pytest_cache
rootdir: /home/me/procustodibus-agent
plugins: mock-3.11.1, datadir-1.4.1, cov-4.1.0, timeout-2.1.0
collected 252 items
test/test_api.py .................................. [ 13%]
...
test/mfa/test_init.py ..................................... [100%]
---------- coverage: platform linux, python 3.10.9-final-0 -----------
Coverage HTML written to dir htmlcov
============================================================================================================= 252 passed in 1.82s =============================================================================================================
coverage: commands[1]> coverage report
Name Stmts Miss Cover
-------------------------------------------------------------
procustodibus_agent/__init__.py 5 0 100%
procustodibus_agent/agent.py 47 31 34%
procustodibus_agent/api.py 155 4 97%
procustodibus_agent/cli.py 24 24 0%
procustodibus_agent/cnf.py 215 7 97%
procustodibus_agent/connectivity.py 77 2 97%
procustodibus_agent/credentials.py 19 19 0%
procustodibus_agent/executor.py 416 24 94%
procustodibus_agent/ip_route.py 36 4 89%
procustodibus_agent/mfa/__init__.py 83 12 86%
procustodibus_agent/mfa/api.py 33 1 97%
procustodibus_agent/mfa/cli.py 29 29 0%
procustodibus_agent/resolve_hostname.py 56 0 100%
procustodibus_agent/wg.py 44 2 95%
procustodibus_agent/wg_cnf.py 174 2 99%
-------------------------------------------------------------
TOTAL 1413 161 89%
coverage: OK ✔ in 9.32 seconds
flake8: install_package_deps> python -I -m pip install cohesion darglint dlint 'docopt-ng~=0.7' flake8 flake8-alfred flake8-bandit flake8-broken-line flake8-bugbear flake8-builtins flake8-class-attributes-order flake8-coding flake8-cognitive-complexity flake8-comprehensions flake8-debugger flake8-docstrings flake8-eradicate flake8-executable flake8-expression-complexity flake8-fixme flake8-functions flake8-isort flake8-logging-format flake8-pep3101 flake8-print flake8-pytest flake8-pytest-style flake8-requirements flake8-rst flake8-spellcheck flake8-string-format 'inflection~=0.5' pep8-naming 'pynacl~=1.5' 'requests~=2.27' 'tabulate~=0.8'
flake8: install_package> python -I -m pip install --force-reinstall --no-deps /home/me/procustodibus-agent/.tox/.tmp/package/6/procustodibus_agent-1.3.3-0.editable-py3-none-any.whl
flake8: commands[0]> flake8
.pkg: _exit> python /home/me/procustodibus-agent/versions/3.10.9/envs/testpyenv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg-cpython37: _exit> python /home/me/procustodibus-agent/versions/3.10.9/envs/testpyenv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg-cpython38: _exit> python /home/me/procustodibus-agent/versions/3.10.9/envs/testpyenv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg-cpython39: _exit> python /home/me/procustodibus-agent/versions/3.10.9/envs/testpyenv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
flake8: OK ✔ in 1 minute 26.16 seconds
build: install_deps> python -I -m pip install setuptools wheel
build: commands[0]> python setup.py -q sdist bdist_wheel
/home/me/procustodibus-agent/.tox/build/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!
********************************************************************************
Please avoid running ``setup.py`` directly.
Instead, use pypa/build, pypa/installer or other
standards-based tools.
See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
********************************************************************************
!!
self.initialize_options()
py37: OK (11.96=setup[9.99]+cmd[1.96] seconds)
py38: OK (11.16=setup[9.34]+cmd[1.83] seconds)
py39: OK (11.37=setup[9.48]+cmd[1.89] seconds)
py310: OK (11.09=setup[9.35]+cmd[1.73] seconds)
py311: OK (11.05=setup[9.32]+cmd[1.72] seconds)
coverage: OK (9.32=setup[6.79]+cmd[2.30,0.23] seconds)
flake8: OK (86.16=setup[14.63]+cmd[71.53] seconds)
build: OK (1.94=setup[1.46]+cmd[0.48] seconds)
congratulations :) (155.31 seconds)
Watch
To run all the tests every time you make a change to the source code, run the tox -e watch command:
(procustodibus-agent) $ tox -e watch cli develop-inst-noop: /home/me/procustodibus-agent cli installed: certifi==2023.7.22,cffi==1.15.1,charset-normalizer==3.2.0,colorama==0.4.6,coverage==7.3.0,docopt==0.6.2,docopt-ng==0.9.0,exceptiongroup==1.1.2,idna==3.4,inflection==0.5.1,iniconfig==2.0.0,packaging==23.1,pluggy==1.2.0,# Editable Git install (procustodibus-agent==1.3.3) with either a deleted local remote or invalid URI:,# 'git@git.sr.ht:~arx10/procustodibus-agent',-e /home/me/procustodibus-agent,pycparser==2.21,PyNaCl==1.5.0,pytest==7.4.0,pytest-cov==4.1.0,pytest-datadir==1.4.1,pytest-mock==3.11.1,pytest-timeout==2.1.0,pytest-watch==4.2.0,requests==2.31.0,tabulate==0.9.0,tomli==2.0.1,urllib3==2.0.4,watchdog==3.0.0 watch run-test-pre: PYTHONHASHSEED='526703176' watch run-test: commands[0] | pytest-watch [Wed Aug 02 18:49:23 2023] Running: py.test ================================================= test session starts ================================================= platform linux -- Python 3.10.6, pytest-7.4.0, pluggy-1.2.0 cachedir: .tox/watch/.pytest_cache rootdir: /home/me/procustodibus-agent plugins: mock-3.11.1, datadir-1.4.1, cov-4.1.0, timeout-2.1.0 collected 252 items test/test_api.py .................................. [ 13%] test/test_cnf.py ............................................. [ 31%] test/test_connectivity.py .............. [ 36%] test/test_executor.py ........................................... [ 53%] test/test_ip_route.py ..... [ 55%] test/test_resolve_hostname.py ......................... [ 65%] test/test_wg.py ......... [ 69%] test/test_wg_cnf.py ................................. [ 82%] test/mfa/test_api.py ....... [ 85%] test/mfa/test_init.py ..................................... [100%] ================================================= 252 passed in 1.07s =================================================
CLI
To run the agent CLI (Command-Line Interface), run the tox -e cli command, followed by -- and the arguments to the agent (eg --help):
(procustodibus-agent) $ tox -e cli -- --help
cli develop-inst-nodeps: /home/me/procustodibus-agent
cli installed: certifi==2023.7.22,cffi==1.15.1,charset-normalizer==3.2.0,colorama==0.4.6,coverage==7.3.0,docopt==0.6.2,docopt-ng==0.9.0,exceptiongroup==1.1.2,idna==3.4,inflection==0.5.1,iniconfig==2.0.0,packaging==23.1,pluggy==1.2.0,# Editable Git install (procustodibus-agent==1.3.3) with either a deleted local remote or invalid URI:,# 'git@git.sr.ht:~axr10/procustodibus-agent',-e /home/me/procustodibus-agent,pycparser==2.21,PyNaCl==1.5.0,pytest==7.4.0,pytest-cov==4.1.0,pytest-datadir==1.4.1,pytest-mock==3.11.1,pytest-timeout==2.1.0,pytest-watch==4.2.0,requests==2.31.0,tabulate==0.9.0,tomli==2.0.1,urllib3==2.0.4,watchdog==3.0.0
cli run-test-pre: PYTHONHASHSEED='337286959'
cli run-test: commands[0] | procustodibus-agent --help
Pro Custodibus Agent.
Synchronizes your WireGuard settings with Pro Custodibus.
Usage:
procustodibus-agent [--config=CONFIG] [--loop=SECONDS]
[-v | -vv | --verbosity=LEVEL]
procustodibus-agent --help
procustodibus-agent --test [--config=CONFIG] [-v | -vv | --verbosity=LEVEL]
procustodibus-agent --version
Options:
-h --help Show this help
--test Run connectivity check
--version Show agent version
-c --config=CONFIG Config file
-l --loop=SECONDS Loop indefinitely, sending ping every SECONDS
--verbosity=LEVEL Log level (ERROR, WARNING, INFO, DEBUG)
-v INFO verbosity
-vv DEBUG verbosity
___________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
cli: commands succeeded
congratulations :)
If you make changes to the agent source code, it will automatically recompile the next time you execute it. For example, open up the procustodibus_agent/cli.py file, and change the main() function to print out some dummy value when the --version option is used:
# procustodibus_agent/cli.py
# ...
def main():
"""CLI Entry point."""
args = docopt(__doc__)
if args["--version"]:
# change this line to print some dummy value like "FOOBAR":
print("FOOBAR")
elif args["--test"]:
check(args["--config"], args["--verbosity"] or args["-v"])
else:
run(
args["--config"],
args["--verbosity"] or args["-v"],
args["--loop"],
)
# ...Run the CLI with the --version option to view the results of your change:
(procustodibus-agent) $ tox -e cli -- --version cli develop-inst-noop: /home/me/procustodibus-agent cli installed: certifi==2023.7.22,cffi==1.15.1,charset-normalizer==3.2.0,colorama==0.4.6,coverage==7.3.0,docopt==0.6.2,docopt-ng==0.9.0,exceptiongroup==1.1.2,idna==3.4,inflection==0.5.1,iniconfig==2.0.0,packaging==23.1,pluggy==1.2.0,# Editable Git install (procustodibus-agent==1.3.3) with either a deleted local remote or invalid URI:,# 'git@git.sr.ht:~arx10/procustodibus-agent',-e /home/me/procustodibus-agent,pycparser==2.21,PyNaCl==1.5.0,pytest==7.4.0,pytest-cov==4.1.0,pytest-datadir==1.4.1,pytest-mock==3.11.1,pytest-timeout==2.1.0,pytest-watch==4.2.0,requests==2.31.0,tabulate==0.9.0,tomli==2.0.1,urllib3==2.0.4,watchdog==3.0.0 cli run-test-pre: PYTHONHASHSEED='127587549' cli run-test: commands[0] | procustodibus-agent --version FOOBAR _______________________________________________________ summary _______________________________________________________ cli: commands succeeded congratulations :)
Install Tests
To run the install tests, first build the Docker container for each:
(procustodibus-agent) $ docker-compose -f test_install/docker-compose.yml build --pull TODO
Then run the install tests:
(procustodibus-agent) $ tox -e py310 test_install TODO
The install tests will take around 15 minutes to complete.
Directories
Here’s a quick tour of the directories in the agent project:
-
build: build working directory -
.builds: SourceHut build scripts -
dist: output of build scripts -
docker: docker build scripts -
etc: service configuration files -
htmlcov: coverage output -
procustodibus_agent: source code -
procustodibus_agent.egg-info: local install directory -
requirements: requirements files -
test: unit tests -
test_install: install tests (run via Docker) -
.tox: tox working directory