System Update Integration

A system update replaces the complete operating system image of a device. Nexigon Agent can install system updates with Rugix Ctrl or RAUC, two update engines for embedded Linux. Both keep two copies of the system, so a device installs an update next to the running system and can return to it if the new system fails.

This guide shows how to prepare a device image and its releases, the versions of the system that you publish to Nexigon Hub, so that Nexigon Agent can update the device. You then select in Nexigon Hub which release each device runs. The agent installs it and reports the result, and the update engine returns the device to the previous system if the new one does not work.

The recipes for Rugix Bakery and the meta-nexigon layers for Yocto, with Rugix Ctrl or RAUC, perform most of the image setup, as Set Up Rugix OTA Updates shows. This guide explains what they do and what a custom image needs. To roll out releases once a device is set up, follow Deploy System Updates.

How It Works

The update engine and the agent split the work. The update engine writes a bundle, the file that contains an update, to the inactive copy of the system and makes the device boot that copy. Nexigon Agent decides which bundle to install and when.

The agent implements the OTA update contract of Nexigon Hub. You set the desired release in the dev.nexigon.ota.config device property, for example the stable tag of a package. The agent applies this property over defaults from its own configuration, so an image can ship with a default release and be switched to another one later. The agent publishes its progress in the dev.nexigon.ota.status property and records each installation and its outcome as an event in the ota category.

The agent checks for updates once it has connected to Nexigon Hub, periodically, when a deferred update becomes due, and when you request a check. Each check:

1. Commits a newly booted system, if one awaits its commit. Committing confirms to the update engine that the new system works.
2. Determines the name and version of the running system from the image.
3. Resolves the configured release to a package version and stops if the device already runs it. If the release requires that the device first runs an earlier version, a checkpoint, the agent installs the checkpoint first.
4. Waits while a hold, a maintenance window, a retry delay, or a spread defers the update. Control When a Device Updates explains these settings.
5. Passes the download URL of the bundle for the running system to the update engine, which downloads and installs the bundle. Then the agent reboots the device.

After the reboot, the next check reports whether the device runs the installed version or has rolled back to the previous one. The agent reads the running version from the image on every check, so the report is correct even after a rollback or a manual installation. For this to work, each image and its release must agree on the system’s name and version, as described below.

Prepare the Image

The image needs the update engine, an identity for the system, a way to verify bundles, and storage for the agent’s state that survives updates.

Install the Update Engine

For Rugix Ctrl, install rugix-ctrl and set up the system for A/B updates as described in the Rugix documentation. The agent installs bundles with rugix-ctrl update install and reboots with rugix-ctrl system reboot --spare, which boots the new system once on trial without making it the default. The agent commits the new system with rugix-ctrl system commit, which makes it the default.

For RAUC, install rauc with a system configuration for your slots, and enable the rauc-mark-good service, which commits a booted system by marking its slot as good. The agent installs bundles with rauc install and reboots with systemctl reboot. RAUC installs from a URL only with streaming support, which requires bundles in the verity or crypt format and network block device (NBD) support in the kernel.

The agent runs these commands as its own user, which needs the privileges to install updates and reboot the device. The systemd unit in the agent’s installation instructions and recipes runs the agent as root.

Identify the System

Every image carries a system name and a version string. The system name identifies a variant of the system, for example gateway-efi-amd64, and selects the bundle that a device installs from a release. The version string identifies the build, for example 2.1 or build-20260401120000-abc1234. It must be unique for every release, because the agent considers a device with the same version string up to date. It also serves as a tag name in Nexigon Hub, so use only letters, digits, ., _, +, ~, and -, starting with a letter or digit, and at most 128 characters. The agent cannot find the package version of a running system whose version string is not a valid tag name.

With Rugix Ctrl, the agent reads both values from /etc/rugix/system-build-info.json, which Rugix Bakery writes into every image. The file contains the name of the Rugix Bakery system and the version that you pass with --release-version. Build the image that you flash and the bundle of a release with the same version:

./run-bakery bake image gateway-efi-amd64 --release-version 2.1
./run-bakery bake bundle gateway-efi-amd64 --release-version 2.1

Without --release-version, Rugix Bakery derives the version from the build’s source date, which is the same for every build unless you set it.

With RAUC, or without this file, the agent reads IMAGE_ID and IMAGE_VERSION from /etc/os-release, for example:

IMAGE_ID=gateway-efi-amd64
IMAGE_VERSION=2.1

In Yocto, the os-release recipe writes this file. Set the variables globally, for example in local.conf or your distribution configuration, because the recipe does not see variables of the image recipe. Set IMAGE_VERSION anew for every release build, for example from a variable that your release pipeline passes to the build:

IMAGE_ID = "gateway-efi-amd64"
IMAGE_VERSION = "2.1"
OS_RELEASE_FIELDS:append = " IMAGE_ID IMAGE_VERSION"

Verify Bundles

The update engines install only bundles that they can verify. RAUC checks the signature of every bundle against the keyring in its system configuration. Rugix Ctrl checks a bundle in one of two ways:

• Signature: Rugix Ctrl checks the bundle’s signature against the certificates that the image trusts. See Signed Updates.
• Hash: With use-bundle-hash enabled in the agent’s configuration, the agent passes a hash from the bundle’s metadata in Nexigon Hub to Rugix Ctrl. You attach this hash when you publish a release. For a bundle without a hash, Rugix Ctrl checks the signature.

Without a trusted signature or a hash, every installation fails. Bundle Verification compares the two options.

Keep the Agent’s State

An update replaces the root filesystem, including any files that the agent stored there. Keep the agent’s TLS certificate and private key, by default in /etc/nexigon/agent/ssl, and its data directory, by default /var/lib/nexigon/agent, on storage that updates do not replace. The data directory holds the credentials that the agent saves after provisioning. If an update loses these credentials, an agent with provisioning disabled fails to start. If it loses the key, the agent creates a new one with a new certificate, which you must accept in Nexigon Hub before the device can connect again. The output of the fingerprint script must also stay the same, so that the device keeps its device ID.

With Rugix Ctrl, persist both directories with state management:

[[persist]]
directory = "/etc/nexigon/agent/ssl"

[[persist]]
directory = "/var/lib/nexigon/agent"

The nexigon/nexigon-agent recipe for Rugix Bakery installs this file, and the meta-nexigon-rugix layer for Yocto persists the same data. With RAUC, place the files on a data partition, point ssl-cert, ssl-key, and data-path in the agent’s configuration to it, and make sure that the partition is mounted before the agent starts. For Yocto, the NEXIGON_AGENT_SSL_DIR and NEXIGON_AGENT_DATA_DIR variables of meta-nexigon do this.

Roll Back Failed Updates

An update can produce a system that boots but does not work, for example because it cannot reach Nexigon Hub. The update engines handle this case differently.

With Rugix Ctrl, a new system runs on trial until the agent commits it, which the agent does only after it has connected to Nexigon Hub. Until then, the previous system remains the default, so the next reboot returns the device to it. To trigger this reboot, install a watchdog that reboots the device if the running system is still uncommitted after a timeout. Rugix Ctrl calls each copy of the system a boot group. A system is uncommitted while rugix-ctrl system info reports an active boot group, the one that runs, that differs from the default boot group, the one that the device boots normally. The nexigon/nexigon-rugix-watchdog recipe for Rugix Bakery reboots the device after 1800 seconds by default, configurable with its timeout parameter, and the meta-nexigon-rugix layer includes the same watchdog.

With RAUC, the rauc-mark-good service commits a new system when it boots, independently of the agent. The bootloader returns to the previous system only if the new one fails to boot and the bootloader counts boot attempts. To also roll back systems that boot but do not work, replace rauc-mark-good with a service that checks the system before it runs rauc status mark-good, for example by waiting until nexigon-agent status reports the connection state Connected, and that reboots the device if the check fails.

Configure Nexigon Agent

Enable the integration for your update engine in the agent’s configuration, for example for Rugix Ctrl:

[integrations.rugix.system-updates]
enabled = true
path = "my-repo/gateway-os/stable"

[commands]
enabled = true

For RAUC, use [integrations.rauc.system-updates] instead. Only one of the two can be enabled. The integration is part of the unreleased Nexigon Agent version that this documentation describes. Until its release, use a development build, as Set Up Rugix OTA Updates shows for Rugix Bakery.

path selects the default release in the format <repository>/<package>/<version>, where <version> is a tag that the device follows or a version ID that pins one version. The example follows the stable tag of the gateway-os package in the my-repo repository. The dev.nexigon.ota.config property overrides the path. Without a path, the device does not update until the property supplies one, which suits generic images that devices of different products share.

With commands enabled, the web interface requests an immediate check through the nexigon.ota.check command, for example after you deploy a release. Otherwise, the device notices the change at its next check, by default after about four hours. System Updates lists all settings, including the check interval.

Publish a Release

Each release is a package version in a repository on Nexigon Hub. For the agent to install it and recognize it on the device afterward, the package version needs:

• A bundle named <system>.rugixb for Rugix Ctrl or <system>.raucb for RAUC, where <system> is the system name of the image. A release for several system variants contains one bundle for each. To use another filename, set asset in the agent’s configuration or in dev.nexigon.ota.config.
• The metadata field imageVersion with the version string of the image. The agent compares it with the running version to decide whether the device is up to date and whether an update completed.
• A tag equal to the version string. The agent resolves the running version through this tag to find the package version that the device runs, which checkpoints and the web interface rely on. Lock the tag, so that it cannot move to another version.

For example, the following Nexigon CLI commands publish version 2.1 of the gateway-efi-amd64 system in the gateway-os package. Rugix Bakery writes the bundle to build/gateway-efi-amd64/system.rugixb:

nexigon-cli repositories packages create my-repo gateway-os
nexigon-cli repositories versions create my-repo/gateway-os \
    --tag 2.1,locked --metadata '{"imageVersion": "2.1"}'
nexigon-cli repositories assets upload my-repo \
    build/gateway-efi-amd64/system.rugixb
nexigon-cli repositories versions assets add <version-id> <asset-id> \
    gateway-efi-amd64.rugixb
nexigon-cli repositories versions tag <version-id> --tag stable,reassign

The first command creates the package, which is needed only once. The second creates the version with its locked tag and metadata and prints its ID as versionId. The third uploads the bundle and prints its ID as assetId, and the fourth attaches it to the version under the filename that devices look for. The last command assigns the stable tag to the version, and reassign moves the tag if another version has it. Devices that follow stable install the release at their next check.

To verify bundles with a hash, pass the hash as metadata when you attach the bundle, in place of the fourth command. Rugix Bakery writes the hash next to the bundle, to system.rugixb-hash:

hash=$(cat build/gateway-efi-amd64/system.rugixb-hash)
nexigon-cli repositories versions assets add <version-id> <asset-id> \
    gateway-efi-amd64.rugixb --metadata "{\"rugix\": {\"bundleHash\": \"$hash\"}}"

If the repository is private, link it to the device’s project under Project Settings → Repositories, so that the device can download its releases. Publish Software Artifacts explains the commands in detail. The release scripts of the Nexigon template for Rugix Bakery and of meta-nexigon follow these rules and build the image with the same version string.

Verify the Integration

Build an image, boot a device with it, and wait for the agent to connect. Then read the update status on the device:

nexigon-agent device properties get dev.nexigon.ota.status

The command prints the property as JSON, with the status as value. For example, shortened:

{
  "result": "Found",
  "name": "dev.nexigon.ota.status",
  "value": {
    "currentVersion": "2.1",
    "currentVersionId": "pkg_v_7Hq2mWbXc9RtLpZ3sNfE1k",
    "state": "completed",
    "updater": { "name": "rugix", "version": "1.4.0", "bootedSlot": "a" }
  }
}

currentVersion shows the version string of the image. Once you have published the release, currentVersionId shows its package version. updater describes the update engine and its slots. The same information appears under Configure Updates on the device’s page in Nexigon Hub. Before you publish the release that path refers to, the state is failed with an error that the version does not exist. For other errors, inspect the log of the agent:

journalctl -u nexigon-agent -b

Then test an update end to end. Publish a second release and deploy it to the device. The status passes through installing and rebooting and, with Rugix Ctrl, committing, and ends with completed and the new version as currentVersion. The device’s event log records each step in the ota category.

Test the rollback as well, before devices depend on it. For example, publish a release in which Nexigon Agent is disabled and deploy it. With Rugix Ctrl, the new system stays uncommitted, so the watchdog reboots the device into the previous system. With RAUC, the check that replaces rauc-mark-good does the same. Once the agent runs again, it reports the update as failed because it was rolled back. Do not run this test on a RAUC device that still uses rauc-mark-good. That service commits the new system, and without the agent, the device can no longer be updated remotely.

Switch from the Standalone Update Service

Before the agent included this integration, the nexigon/nexigon-rugix-ota recipe of the v0.3 recipe branch for Rugix Bakery installed a separate update service. That service uses the same device properties and the same nexigon.ota.check command, so devices keep their update configuration and status when you switch.

To switch:

1. Remove nexigon/nexigon-rugix-ota from your layer and keep nexigon/nexigon-rugix-watchdog.
2. Install a Nexigon Agent version that includes the integration, currently a development build, and add the configuration above. Set path to the repository, package, and tag that the service used.
3. Publish a release built this way and deploy it.

The old service installs this release. After the reboot, the agent of the new system commits it and takes over all further updates.