Download the Agent

To install the Pro Custodibus agent on a Linux, FreeBSD, OpenBSD, or macOS host, download the agent tarball from this page. After you have downloaded the agent tarball and verified it, copy it to the host and run the installer.

To install the Pro Custodibus agent on a Windows host, download and run the Windows installer. To run the Pro Custodibus agent in a Docker (or other OCI) container, use a pre-built container image.

What is the Pro Custodibus Agent?

A lightweight service that runs on each monitored host and sends WireGuard usage and audit logs to the Pro Custodibus servers. If configured to do so, it can also update the host’s WireGuard and network configuration.

When we use the term “agent”, we typically mean the Pro Custodibus agent.

See the Terminology page for a fuller explanation.

Signing Keys

All tarballs will be signed by one of the following PGP keys:

Verify Signature

To check the integrity of a downloaded tarball, verify it via PGP signature. Run the following commands in a terminal:

Import Signing Keys

Import the following keys, using their primary-key ID, from your default keyserver:

$ gpg --recv-keys EFC1AE969DD8159F
gpg: key EFC1AE969DD8159F: public key "Pro Custodibus Build 1 <build@custodib.us>" imported
gpg: Total number processed: 1
gpg:               imported: 1

If this generates an error (like keyserver receive failed), try importing the keys from the OpenPGP keyserver:

$ gpg --keyserver keys.openpgp.org --recv-keys EFC1AE969DD8159F
gpg: key EFC1AE969DD8159F: public key "Pro Custodibus Build 1 <build@custodib.us>" imported
gpg: Total number processed: 1
gpg:               imported: 1

Double-check the primary-key fingerprints against the signing keys listed above:

$ gpg --fingerprint EFC1AE969DD8159F
pub   rsa3072 2020-10-22 [SC]
      98EE 78FA B506 BEAB C0DA  DE15 EFC1 AE96 9DD8 159F
uid           [ unknown] Pro Custodibus Build 1 <build@custodib.us>
sub   rsa3072 2020-10-22 [E]

If for some reason the fingerprints don’t match, please contact us at security@arcemtene.com and let us know!

Download Signature

Download a tarball (.tar.gz file) and the matching signature (.sig file) from the downloads listed above:

$ wget https://ad.custodib.us/agents/procustodibus-agent-latest.tar.gz
Resolving ad.custodib.us (ad.custodib.us)...
... 'procustodibus-agent-latest.tar.gz' saved [33000/33000]
$ wget https://ad.custodib.us/agents/procustodibus-agent-latest.tar.gz.sig
Resolving ad.custodib.us (ad.custodib.us)...
... 'procustodibus-agent-latest.tar.gz.sig' saved [833/833]

Verify Tarball

Now verify the downloaded tarball against the downloaded signature:

$ gpg --verify procustodibus-agent-latest.tar.gz.sig procustodibus-agent-latest.tar.gz
gpg: Signature made Fri Oct  9 19:53:00 2020 UTC
gpg:                using RSA key 98EE78FAB506BEABC0DADE15EFC1AE969DD8159F
gpg: Good signature from "Pro Custodibus Build 1 <build@custodib.us>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 98EE 78FA B506 BEAB C0DA  DE15 EFC1 AE96 9DD8 159F

The signing-key fingerprint (eg 98EE 78FA B506 BEAB C0DA DE15 EFC1 AE96 9DD8 159F in the above example) should match the fingerprint of one of the signing keys listed above. If it doesn’t match, please contact us at security@arcemtene.com and let us know!

You can skip downloading the signature as a separate file, and just pipe the signature directly from wget (or curl) into the gpg --verify command:

$ wget https://ad.custodib.us/agents/procustodibus-agent-latest.tar.gz.sig -O - | gpg --verify - procustodibus-agent-latest.tar.gz
gpg: Signature made Fri Oct  9 19:53:00 2020 UTC
gpg:                using RSA key 98EE78FAB506BEABC0DADE15EFC1AE969DD8159F
gpg: Good signature from "Pro Custodibus Build 1 <build@custodib.us>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 98EE 78FA B506 BEAB C0DA  DE15 EFC1 AE96 9DD8 159F

Verification Results

Note that whatever warnings or success messages the gpg --verify command outputs, the important thing is that the fingerprint of the key that signed the tarball matches a known fingerprint from this page that you’ve previously recorded (like 98EE 78FA B506 BEAB C0DA DE15 EFC1 AE96 9DD8 159F). The “Good signature” message matters only if this signing-key fingerprint matches a known good fingerprint — only if it does can you be confident that you have a good download.

Unknown system error

If GnuPG reports verify signatures failed: Unknown system error, the signature file you downloaded may be empty. Try re-downloading the signature file. If you open up the signature file in a text editor, the first line of the file should be -----BEGIN PGP SIGNATURE-----.

No valid OpenPGP data found

If GnuPG reports no valid OpenPGP data found, you may have downloaded something other than an signature file. Try re-downloading the signature file. If you open up the signature file in a text editor, the first line of the file should be -----BEGIN PGP SIGNATURE-----.

You also may have reversed the order of the signature file and tarball when calling the gpg --verify command. Make sure you specify the signature file first, and the tarball second, when you call gpg --verify.

No public key

If GnuPG reports Can’t check signature: No public key, you may not have imported the key that signed the tarball. Make sure you’ve imported all the signing keys listed above.

This is the message you would see if an attacker took over our download servers and replaced our files with different files, signed by a different key that the attacker controls. While the most likely reason that you’d see this message is simply that you haven’t imported all our signing keys, if believe you had previously imported all our signing keys, contact support@procustodibus.com to double-check before importing any new keys.

Bad signature

If GnuPG reports BAD signature, you may have a mismatch between the signature file and tarball that you downloaded. Try re-downloading the .tar.gz and .sig files of the same version from the downloads listed above.

This is the message you would see if an attacker took over our download servers and replaced our tarballs with different content, without bothering to replace the signature files. However, the mostly likely reason you’d see this is simply that you’ve downloaded a different tarball version than the version of signature file you downloaded, or that the tarball you downloaded was incomplete or corrupted by an innocuous network error. Try re-downloading the .tar.gz and .sig files again. If the problem persists, contact suppport@procustodibus.com, and we’ll double-check that everything’s okay with our download servers.

Next Steps

Copy the agent tarball to the host and run the installer.

Source Code

The Pro Custodibus agent is open source. You can check out the source code at https://git.sr.ht/~arx10/procustodibus-agent. Note, however, that we only support versions of the agent listed in the downloads section above.

Changelog

1.7.5 - 2024-11-14

Fixed

  • Fixed reading WireGuard config files on Windows when the interface name contained a dot (.).

  • Fixed the following error when running extras scripts on a system using the mawk version of awk: ` awk: line 3: regular expression compile failed (missing operand) `

  • Relaxed minimum required dnspython module version to support older versions of Python.

1.7.4 - 2024-10-14

Fixed

  • Fixed error in several extras scripts with certain versions of awk (however, this was an incomplete fix; 1.7.5 should fix the issue completely).

1.7.3 - 2024-10-08

Fixed

  • Corrected previous fix to account for default HTTP port of 80.

1.7.2 - 2024-10-08

Fixed

  • Fixed using API with custom plain-HTTP URL with DNS hostname (eg Api = http://example.com); would fail with got an unexpected keyword argument 'assert_hostname' message.

1.7.1 - 2024-09-27

Fixed

  • Fixed service start-up on Windows after new install (would fail with cannot import service module message).

1.7.0 - 2024-09-25

Added

  • Additional helper scripts for Windows.

Changed

  • Use external DNS resolver by default (see below).

  • Apply packet mark (aka fwmark) to agent connections on Linux when wg-quick will use the mark to override the default gateway (see below).

  • Installer automatically installs wireguard-tools package.

New DNS Settings

Previously, the agent would use the host operating system’s own DNS resolver. Now, by default, the agent will use the Quad9 resolvers via DoH (DNS over HTTPS) to resolve the IP address of the Pro Custodibus API, as well as any WireGuard endpoint addresses specified by DNS name.

This is particularly useful when a WireGuard interface is itself used for the host’s default route, to ensure that the agent can resolve the IP address needed to connect to the API even when the WireGuard tunnel is not working.

To revert to the previous behavior, and use the host operating system’s DNS resolver, add the following setting to the procustodibus.conf file:

Dns = off

To use a custom DoH resolver (such as Cloudflare’s), add the following settings to the procustodibus.conf file, customizing the Dns setting to specify the DNS servers to contact, and the Doh setting to specify the hostname that the servers' TLS certificate must match:

Dns = 1.1.1.1, 1.0.0.1, 2606:4700:4700::1111, 2606:4700:4700::1001
Doh = cloudflare-dns.com

To use a custom UDP resolver, add the following settings to the procustodibus.conf file, customizing the Dns setting to specify the DNS servers to contact:

Dns = 1.1.1.1, 1.0.0.1, 2606:4700:4700::1111, 2606:4700:4700::1001
DnsProtocol = udp

To use a custom TCP resolver, add the following settings to the procustodibus.conf file, customizing the Dns setting to specify the DNS servers to contact:

Dns = 1.1.1.1, 1.0.0.1, 2606:4700:4700::1111, 2606:4700:4700::1001
DnsProtocol = tcp

The agent will automatically attempt to detect whether to use IPv4 or IPv6 addresses. To force the agent to always prefer IPv4 addresses, add the following setting to the procustodibus.conf file:

ResolveHostnames = ipv4

To force the agent to always prefer IPv6 addresses, add the following setting to the procustodibus.conf file:

ResolveHostnames = ipv6

New FwMark Setting

When a network mask of /0 (aka the default route) is used on Linux for the AllowedIPs setting of a WireGuard peer, WireGuard`s wg-quick helper script will set up some policy routing rules to ensure that the WireGuard interface’s own traffic is not recursively routed through the interface’s tunnel; these rules rely on the interface emitting packets with a specific mark (configurable via the FwMark setting on the interface).

Now when the agent detects this scenario, it will also mark its own packets with the same mark, ensuring that its connections to the Pro Custodibus API (and the agent’s DNS resolver) do not go through the WireGuard tunnel (and so the agent will still work even if the tunnel does not).

To prevent the agent from doing this, add the following setting to the procustodibus.conf file:

FwMark = off

To force the agent to always mark its packets with a specific mark (such as 123), add the following setting to the procustodibus.conf file:

FwMark = 123

1.6.1 - 2024-08-08

Changed

  • Log recent unit events from journalctl when WireGuard up/down fails (when using systemd to manage WireGuard).

  • Always print connection test after reloading configuration.

Fixed

  • Fix error applying clamp_msp extra script.

  • Fix errors applying extra scripts when value blank.

1.6.0 - 2024-08-02

Added

  • New extra helper scripts for WireGuard Pre/Post/Up/Down fields (located in scripts directory).

  • New OneLineFields = true agent configuration setting to force WireGuard configuration fields that may use multiple lines (like Address, AllowedIPs, etc) to instead be written on one single line. Defaults to false.

  • Allow WireSock to be used in place of WireGuard.

Changed

  • Allow PROCUSTODIBUS_LOGGING_LEVEL environment variable to override command line --verbosity (or -v or -vv) flags.

  • Log new WireGuard config at debug level when updating it.

  • Omit logging WireGuard service status on Windows when OK.

Fixed

  • Fix updates to WireGuard DNS setting when search path not specified.

  • Fix configuration reloading on Windows.

  • Resume regular ping loop faster after connectivity interruptions.

  • Gracefully handle errors from missing agent credentials file.

Removed

  • Removed Python 3.7 as a supported version.

1.5.2 - 2024-05-26

Added

  • Installer tests for Alpine 3.20, Fedora 40, and Ubuntu 24.04.

Fixed

  • Installer now can install libsodium on Amazon Linux 2023.2 and newer.

  • Installer when upgrading now correctly detects version of existing agent package with newer versions of Python/Pip.

1.5.1 - 2024-05-03

Added

  • Added option to watch config and reload automatically when changed. To turn on, set the following environment variable:

    PROCUSTODIBUS_WATCH_CONFIG=true

1.5.0 - 2024-04-05

Added

Full Windows support, including:

  • Standard GUI installer.

  • Runs as a standard Windows service.

  • Correctly determines WireGuard interface addresses.

  • Applies queued changes.

Fixed

  • Check if route exists before trying to add or delete it.

1.4.5 - 2024-01-15

Fixed

  • Fixed DNS connectivity check in IPv6-only environments.

1.4.4 - 2023-12-27

Added

  • Make app UI URL customizable (used in WireGuard config file comments). Customize by adding an App setting to the procustodibus.conf file; eg:

    App = https://procustodibus.example.com
  • Tests for Python 3.12.

  • Installer tests for Alpine 3.19, Fedora 39, and Ubuntu 23.10.

Fixed

  • When removing a [Peer] section from a WireGuard config file, also remove any comments directly above the section.

Changed

1.4.3 - 2023-12-14

Fixed

  • Include iptables package in wireguard & agent docker images (missing in 1.4.2).

1.4.2 - 2023-12-10

Changed

  • Update base docker image to Alpine Linux 3.19.

  • Use virtualenv for agent in docker container.

1.4.1 - 2023-10-08

Added

  • Support for writing WireGuard config to directories other than /etc/wireguard.

  • Installer support for Homebrew package manager and launchd supervisor.

Fixed

  • Fixed usage of ifconfig on macOS to determine interface address.

  • Fixed errant SELinux error message on non-Linux platforms.

1.4.0 - 2023-09-22

Requires Pro Custodibus API version 1.1 or newer.

Changed

  • Use server challenge for signature authentication (insteading of signing an URL with a timestamp).

1.3.3 - 2023-05-27

Added

  • Installer tests for Alpine 3.18, Debian Bookworm, Fedora 38, and Ubuntu Lunar.

Fixed

  • Installer now detects and fixes conf files with wrong SELinux context labels.

1.3.2 - 2023-03-16

Added

  • Test execution for Python 3.11.

  • Installer tests for Fedora 37 and Alpine 3.17.

Fixed

  • Installer now ensures libffi headers are installed on platforms where they are needed to build PyNaCl.

  • Installer now checks for Amazon Linux 2023 (was previously Amazon Linux 2022).

Changed

  • Added timeout to Pro Custodibus API requests.

  • Added additional hardening constraints to systemd service unit. This will not be upgraded automatically — see the below “Manually Upgrade Systemd Hardening” section to apply.

Manually Upgrade Systemd Hardening

The additional hardening constraints for the agent’s systemd service unit will only be installed automatically on fresh installs. It will not be upgraded automatically. To apply the additional hardening manually, copy the etc/systemd.service file from the agent tarball to the /etc/systemd/system/procustodibus-agent.service file on the host, reload the configuration, and restart the agent:

$ sudo cp etc/systemd.service /etc/systemd/system/procustodibus-agent.service
$ sudo systemctl daemon-reload
$ sudo systemctl restart procustodibus-agent

1.3.1 - 2022-09-26

Added

  • Allow loop interval override in conf file.

  • Installer warning that libsodium must be installed manually with Amazon Linux 2022.

  • Installer tests for RHEL 9 and derivatives.

Changed

  • Require recent versions of package dependencies.

  • Deprecate Python 3.6.

  • Installer detect and recreate broken virtualenv.

  • Installer upgrade pip and setuptools before install.

1.3.0 - 2022-06-28

Added

Changed

  • Use latest PyNacl Python package, and update installer to install OS packages to build it if necessary.

  • Run daemons with INFO verbosity instead of WARN.

  • Install Python 3.8 on RHEL 7.

1.2.6 - 2022-04-08

Fixed installer upgrade with older versions of Python.

1.2.5 - 2022-03-01

Added

  • Add a few minor hardening directives to systemd service definition.

Fixed

  • Fix endpoint restore with no preshared key.

1.2.4 - 2022-02-04

Added

  • Add installer support for Rocky and AlmaLinux.

Fixed

  • Start up daemon even when API is not available; wait until API is available to start pinging.

  • Avoid having to build new PyNacl package on several older Linux distros during install.

1.2.3 - 2021-12-12

Fixed broken procustodibus-credentials command.

1.2.2 - 2021-11-11

Fixed

  • Installer support for FreeBSD.

  • Use ifconfig instead of ip address when not on Linux.

  • Ignore domain name resolution errors for local hostname.

1.2.1 - 2021-11-09

Added

  • Build scripts for docker images.

Changed

  • Run connectivity check on startup.

  • Set up credentials if setup code available during connectivity check.

  • Log to stdout instead of stderr by default.

1.2.0 - 2021-07-20

Added

  • Added ability to add, update, and delete WireGuard interfaces and endpoints based on desired change queue returned by Pro Custodibus API calls.

  • Added optional configuration file for logging at /etc/wireguard/procustodibus-logging.conf (format here).

Changed

  • Changed ResolveHostname to a global setting, now set in procustodibus.conf instead of WireGuard config files.

Fixed

  • Fixed parsing of IPv6 endpoint addresses.

  • Fixed handling of unknown commands by sysvinit.sh service script.

1.1.2 - 2021-05-16

Fixed parsing of firewall mark when set for an interface.

1.1.1 - 2021-04-29

Changed ping to report configured MTU only, not actual MTU.

1.1.0 - 2021-04-21

Added three new features:

  1. Automatically update each monitored WireGuard endpoint when its hostname resolves to a new IP address. This can be turned off per-endpoint via a ResolveHostname = once setting in the [Peer] section of a WireGuard config file.

  2. Send all WireGuard config settings to Pro Custodibus, including pre/post-up/down commands, and private/preshared keys. The sending of keys can be turned off by setting RedactSecrets = true in the Pro Custodibus config file.

  3. Allow specific WireGuard interfaces to be ignored. To do so, specify the interface names in the UnmanagedInterfaces setting of the Pro Custodibus config file (eg UnmanagedInterfaces = wg0, wg1).

1.0.3 - 2020-11-17

  • Fixed installer to prompt to restart daemon after agent upgrade

  • Fixed systemd service config to wait to start until network is available

1.0.2 - 2020-10-22

  • Fixed agent to keep running even if pings start to fail

  • Fixed installer not to chop off systemd service ExecStart args

1.0.1 - 2020-10-15

  • Fixed overly-restrictive systemd service configuration

  • Fixed documentation links

1.0.0 - 2020-10-09

First stable release of the new agent.