9. Add a remote gateway node

Back to overview · Full table of contents · Download full PDF

This chapter: when callers sit in another network zone and the control plane is not there, you can add a remote gateway node in that zone so model and tool calls finish authentication, routing, and forwarding locally, without looping back to the control plane. This chapter states when a standalone node is needed, what to prepare before opening one, how to create the node in the console and install it on the node host, how to confirm it is online, and how to reinstall, change the control plane, and upgrade. The readers are the implementation engineer and the customer system administrator.

9.1 When a standalone node is needed

Applies to: business calls need to enter from another network zone, and the control plane is not there.

The host that holds the control plane is itself the first gateway node, named office-primary (the central node). It is installed and started with the control plane. A single-host deployment does not need a separate node install. Add a remote node only when callers and the control plane are not in the same network zone, and crossing zones is expensive or not allowed.

How a remote node works:

• The node itself finishes authentication, routing, rate limiting, guardrail checks, and billing pre-authorization. The request does not loop back to the control plane.
• The node connects to the control plane on its own to pull configuration and report heartbeats. The control plane never connects to the node.
• When the node is disconnected from the control plane, it continues serving with the last configuration it received. Request records are stored locally on the node first and are sent up automatically after the connection returns.

Note: after a node comes online it does not automatically take public traffic. To make a public access URL resolve to that node, Join ingress pool in the node detail and adjust DNS or the load balancer. That step takes effect only after ingress self-check passes. For a POC that only verifies nearby calls, the business side can use the node address directly.

9.2 Prerequisites

Prerequisite: the control plane has been installed per chapters 4–7 and has passed acceptance; the central node office-primary is online.

Before writing any file, the node installer runs a full host check once: CPU architecture, memory, disk, service manager, execute rights and security modules, clock offset, listen ports, and connectivity to the control plane. When there is a blocking item it prints diagnostic codes and remedies in a summary and writes nothing. Codes in the NODE group are node-related; meanings are on the product documentation site's "Install and operations diagnostic codes" page.

Node install-directory rules

Note: the install directory cannot be changed through an upgrade after first install. Plan the directory against the node host's disk before creating the node.

9.3 Create the node in the console

Prerequisite: signed in to the platform admin console as root.

Procedure

1. Open Gateway › Nodes and click Add node.
2. Fill in the four-step dialog:

3. Click Create and get install command, copy the entire command immediately, and store it in a controlled location.

What you will see

• The dialog shows the full install command, which includes a one-time token.
• The dialog enters Onboarding progress, waiting for the node's first online. After the node install finishes, the page updates without a refresh.

Note: the install command is shown only once. After the dialog is closed it cannot be viewed again. When the token expires (24 hours), is lost, or has been used, click Reissue install command in the node detail to get a new command.

Note: the token is equivalent to the node's join credential. Do not send it in plaintext over instant messaging, email, or a ticket, and do not screenshot it for anyone off the host.

9.4 Install on the node host

Prerequisite: the node host meets section 9.2; the installer and offline package of the same product, same version, and matching the node-host platform have been obtained from the download site and SHA-256-checked per Chapter 2.

Procedure

1. Place the installer and offline package in the same staging directory on the node host, unpack the installer, and rename the offline package to zenova-node.zpkg (the command generated by the console uses this fixed file name).

# Node host · root · write (unpack the installer; rename the offline package)
mkdir -p /root/zenova-install
cd /root/zenova-install
tar xzf <installer>.tar.gz
mv <offline-package>.zpkg zenova-node.zpkg

2. Preflight first. Preflight runs the host check and connectivity to the control plane. It does not consume the token and writes nothing.

# Node host · root · read-only (preflight; does not consume the token)
cd /root/zenova-install
sudo ./zenova install node --package zenova-node.zpkg --control-plane <control-plane-callback-url> --check-only

3. After preflight passes, run the install command the console gave. Control-plane CA parameters follow the command generated by the console: copy it whole, do not delete or change it. When the control plane uses a private CA or an install directory was filled in, the generated command includes the matching flags. The command looks like:

# Node host · root · write (installs the node and consumes the token)
cd /root/zenova-install
sudo ./zenova install node --package zenova-node.zpkg --control-plane <control-plane-callback-url> --token <token>

4. Prefer --token-file: write the token into a file readable only by root, then in the command generated by the console replace --token <token> with --token-file <file> and keep the other flags as they are. --token-file and --token are mutually exclusive.

# Node host · root · write (write the token to a file, then install the node; consumes the token)
cd /root/zenova-install
umask 077
printf '%s' '<token>' > /root/zenova-install/node.token
sudo ./zenova install node --package zenova-node.zpkg --control-plane <control-plane-callback-url> --token-file /root/zenova-install/node.token

5. After a successful install, delete the token file.

# Node host · root · delete (clean the token file)
rm -f /root/zenova-install/node.token

What you will see

• When preflight passes, the screen reports pass and writes nothing. When there is a blocking item it lists diagnostic codes and remedies (follow the actual output).
• On install success the installer prints the result. On failure it prints the reason (for example the token has expired or been used, the node package failed verification, or a system service failed to start) and exits non-zero.

Note: --token-file keeps the token out of the process list and the shell history. With --token, other users on the same host may see the token in the process list during install.

Note: preflight does not validate the token. The token is redeemed only during the real install. Token-expired or already-used errors appear only at that step. Passing preflight does not mean the real install will pass. On a token error, Reissue install command in the node detail and rerun with the new command. Do not run the command that carries the token when preflight failed.

9.5 Confirm it is online

Procedure

1. Return to the console Add node dialog or the node detail and inspect Onboarding progress.
2. Inspect node status on the node host.

# Node host · root · read-only
sudo zenova status

3. Check node information in the node list and the node detail (see below).
4. Have the caller point the Base URL at the node address, complete one real call per section 7.3, and find that call in console Gateway requests by response header x-request-id.
5. When onboarding is abnormal, inspect node logs.

# Node host · root · read-only (view node logs)
sudo zenova logs node-edge --since '30 min ago'
sudo zenova logs node-caddy --since '30 min ago'

What you will see

• Onboarding progress shows three states in order:

• zenova status shows node identity, the control plane it is connected to, and readiness.
• In the node list you can check: Status, Version, Config versions behind, Last heartbeat, Requests in 5 min, Error rate in 5 min, Upload backlog, Local spool.
• The real call returns HTTP 200 and a model reply; console Gateway requests can find that call by x-request-id.

Note: within 2 minutes of disconnect from the control plane, the UI still shows online. After 2 minutes it is marked Offline; the node continues serving with the last configuration.

Note: if the control plane uses a self-signed certificate and the CA was not pasted when the node was created, the node cannot connect to the control plane. The remedy is to add the CA certificate under Edit network in the node detail, then reissue the install command if needed.

9.6 Reinstall, change the control plane, and upgrade

Installing again on a host that already has a node

Running the install command again on a host that already has a node, the installer reports "this host already has a gateway-node install" and stops. It does not overwrite the existing node.

To connect that host to another control plane, or to rejoin with a new token, first create a node on the target control plane (or reissue the install command), then run:

# Node host · root · write (rejoin; consumes the new token)
cd /root/zenova-install
sudo ./zenova install node --package zenova-node.zpkg --reenroll --control-plane <control-plane-callback-url> --token <new-token>

Note: do not delete the node directory (for example /opt/zenova-node) or the system service units by hand to "reinstall". The node directory holds node identity, local spool, and request records not yet uploaded. Deleting them by hand loses those records permanently, and later install and upgrade cannot identify this host's state correctly. Always rejoin with --reenroll above.

Upgrade a node

Prerequisite: the installer and offline package of the target version, matching the node-host platform, have been obtained.

Procedure

1. When the upgrade includes the node package for the node's OS, online nodes are upgraded automatically by the control-plane upgrade; you usually do not operate on the node. A control-plane upgrade (see section 10.5) upgrades online nodes one by one before the console is switched. Use the steps below when: the upgrade package does not contain the node package for the node's OS (the pre-upgrade Gateway nodes check fails and must be finished before the control-plane upgrade), the node was offline during the upgrade, or automatic upgrade failed and a node needs a separate catch-up.
2. Place the new-version installer on the node host and rename the new-version offline package to zenova-node.zpkg.
3. In the console node detail, click Issue upgrade command and copy the command.
4. Run that command with the new-version installer in the staging directory. The command includes --upgrade and looks like:

# Node host · root · write (upgrade the node; keep node identity and data)
cd /root/zenova-install
sudo ./zenova install node --package zenova-node.zpkg --control-plane <control-plane-callback-url> --token <newly-issued-token> --upgrade

5. After the upgrade, confirm the node version and online state per section 9.5, and complete one real call through that node.

What you will see

• The node switches to the new version. The Version column in the node list matches the control-plane target version.
• If the node is not ready within 60 seconds after switching to the new version, it automatically switches back to the old version and continues serving on the old version. Then inspect node-edge logs for the cause, reissue the upgrade command after the fix, and run it. Do not rerun the same command repeatedly.

Note: upgrade keeps node identity, spool, and ledger data. The upgrade command does not take --install-dir; it reuses the directory from first install. Pointing at another directory on the upgrade command is refused.

Note: the offline package for a control-plane upgrade must include the node packages for the platforms of online nodes. Otherwise the pre-upgrade check "node packages required by gateway nodes are complete" fails and lists the nodes and platforms that are missing a package.

Capability notes

Previous: 8. Required post-install work and day-to-day operations · Next: 10. Backup, restore, and upgrade