Skip to main content

Adding Nodes

Adding nodes (servers) to your RunOS cluster is the first step in building your infrastructure. This guide covers both manual node addition and automatic cloud provisioning through Hetzner.

Overview​

A node is a physical or virtual server that becomes part of your Kubernetes cluster. RunOS supports two primary methods for adding nodes:

  • Manual Setup - Add existing servers you already control (bare-metal, cloud VMs, on-premises)
  • Automatic Cloud Provisioning - Let RunOS provision and configure cloud servers for you (currently Hetzner)

Prerequisites​

Before adding a node, ensure you have:

  • An active RunOS account
  • A cluster created in the RunOS console
  • For manual setup: An Ubuntu server (20.04 or 22.04) with SSH access
  • For cloud provisioning: A Hetzner account and API token
  • Minimum server requirements: 2GB RAM, 2 CPU cores, 20GB storage

Method 1: Manual Node Setup​

Use this method when you want to add existing servers or have infrastructure in specific locations.

Step 1: Access the Add Node Interface​

  1. Log in to the RunOS console
  2. Select your cluster from the cluster dropdown
  3. Navigate to Nodes page (or use onboarding if this is your first node)
  4. Click "Add Node" or "Add Server" button

Step 2: Choose Self-Managed Setup​

  1. Select "Self-Managed Setup" option
  2. You'll see instructions for connecting your server

Step 3: Prepare Your Server​

Ensure your server meets these requirements:

Operating System:

  • Ubuntu 20.04 LTS (Focal) - Recommended
  • Ubuntu 22.04 LTS (Jammy) - Supported
  • Fresh installation recommended (no existing Kubernetes)

Network Requirements:

  • Outbound internet access on ports 80, 443
  • Outbound UDP on ports 51820, 51821 (WireGuard VPN)
  • SSH access for initial configuration
  • Static or stable IP address recommended

Hardware Minimum:

  • 2 CPU cores
  • 2GB RAM
  • 20GB storage

Hardware Recommended:

  • 4+ CPU cores
  • 8GB+ RAM
  • 50GB+ SSD storage

Step 4: Run the Installation Command​

RunOS provides a unique installation command for your cluster.

  1. Copy the command from the RunOS console (it looks like this):
curl -sSL https://install.runos.com/node | bash -s -- --token YOUR_UNIQUE_TOKEN
  1. SSH into your server:
ssh user@your-server-ip
  1. Run the installation command as root or with sudo:
sudo su
curl -sSL https://install.runos.com/node | bash -s -- --token YOUR_UNIQUE_TOKEN
  1. Wait for installation to complete (typically 5-10 minutes)

Step 5: Monitor Installation Progress​

The installation process shows progress in your terminal:

Installation phases:

  1. System Preparation (0-15%) - Essential packages and system configuration
  2. Network Configuration (18-30%) - Kernel modules and network parameters
  3. VPN Setup (30-45%) - WireGuard VPN installation and configuration
  4. DNS Configuration (45-55%) - DNS resolvers and dnsmasq setup
  5. Container Runtime (55-70%) - containerd installation
  6. Kubernetes Components (70-95%) - kubelet, kubeadm, kubectl installation
  7. Cluster Join (95-100%) - Node joins the cluster

You can also monitor progress in the RunOS console under Nodes.

Step 6: Verify Node is Online​

Once installation completes:

  1. Check the RunOS console - Node should appear with "Online" status
  2. Verify from terminal (if this is a control plane node):
kubectl get nodes

Your node should show as "Ready" within 1-2 minutes.

What Gets Installed​

During manual setup, RunOS installs and configures:

System Tools:

  • Essential utilities (jq, htop, net-tools)
  • System resource limits configured
  • Swap disabled (Kubernetes requirement)

Networking:

  • WireGuard VPN (wg0 for internal, wg1 for user access)
  • Network bridges and iptables rules
  • DNS resolution (dnsmasq + systemd-resolved)

Container Platform:

  • containerd runtime
  • Kubernetes components (kubelet, kubeadm, kubectl)
  • HAProxy for API load balancing
  • Helm package manager

RunOS Components:

  • Node Agent (runs inside Kubernetes)
  • Cluster Agent (for first node only)
  • mTLS certificates for secure communication

Method 2: Automatic Cloud Provisioning (Hetzner)​

Use this method to let RunOS automatically provision and configure cloud servers for you.

Step 1: Access Cloud Provisioning​

  1. Navigate to Add Node interface
  2. Select "Automatic Cloud Provisioning" option
  3. Choose Hetzner as your cloud provider

Step 2: Configure Hetzner API Access​

If this is your first time using Hetzner provisioning:

  1. Get a Hetzner API token:

    • Log in to your Hetzner Cloud account
    • Navigate to Security → API Tokens
    • Click "Generate API Token"
    • Give it a descriptive name (e.g., "RunOS Cluster")
    • Copy the token (you'll only see it once)

  2. Add token to RunOS:

    • Paste your Hetzner API token in the RunOS console
    • Token is encrypted and stored securely
    • Used only for provisioning servers in your account

Step 3: Configure Server Specifications​

Choose your server configuration:

Server Location:

  • Select datacenter location (Nuremberg, Helsinki, Falkenstein, etc.)
  • Choose region closest to your users for best performance

Server Type: Choose from Hetzner's server types:

  • CX21 (2 vCPU, 4GB RAM) - Development
  • CX31 (2 vCPU, 8GB RAM) - Small production
  • CX41 (4 vCPU, 16GB RAM) - Medium production
  • CX51 (8 vCPU, 32GB RAM) - Large production
  • Dedicated CPU options - For performance-critical workloads

Storage:

  • Default server storage included
  • Additional volumes can be attached after provisioning

Server Name:

  • Provide a descriptive name (e.g., "prod-node-1", "dev-control-plane")
  • Name appears in both RunOS console and Hetzner panel

Firewall Options:

  • Firewalled: Automatically creates a Hetzner firewall with least-privilege rules
    • Blocks all inbound traffic except necessary ports
    • Requires VPN connection (wg1) to access dashboards and cluster services
    • Recommended for production environments
    • More secure, but requires user VPN setup for remote access
  • No Firewall: Server accessible directly via public IP
    • Services exposed directly to internet (less secure)
    • No VPN required for access
    • Only use for development or when behind external firewall

SSH Key Access:

  • Upload your SSH public key to your Hetzner project first
  • Select your key from the list during node provisioning
  • Allows SSH access using your private key (no password needed)
  • Find SSH keys in: Hetzner Cloud Console → Security → SSH Keys
  • You can add multiple keys for team access

Additional Storage:

  • Specify volume size in GB (optional)
  • Creates additional volume beyond the default server storage
  • Attached automatically to the provisioned server
  • Useful for databases, logs, or application data
  • Can be expanded later through Hetzner panel
  • Example: Add 100GB volume for database workloads

Step 4: Review and Provision​

  1. Review configuration summary

    • Server type and specifications
    • Monthly cost estimate
    • Location and network settings

  2. Click "Provision Server"

  3. Wait for provisioning (typically 2-5 minutes)

    • Hetzner creates the server
    • RunOS automatically installs and configures the node
    • Node automatically joins your cluster

Step 5: Verify Provisioning​

Once complete:

  1. RunOS Console - Node appears with "Online" status
  2. Hetzner Panel - Server visible in your Hetzner account
  3. Billing - Server charged to your Hetzner account

Automatic Configuration​

When using Hetzner provisioning, RunOS automatically:

  • Creates the cloud server in your Hetzner account
  • Configures Ubuntu with required packages
  • Installs WireGuard VPN and connects to cluster
  • Joins the node to your Kubernetes cluster
  • Sets up monitoring and management agents
  • Configures firewall and security settings

The entire process is hands-off - no manual SSH or commands required.

Managing Provisioned Servers​

In RunOS Console:

  • View node status and metrics
  • Add/remove nodes
  • Configure node roles (control plane vs worker)

In Hetzner Panel:

  • View billing and invoices
  • Manage server power states
  • Configure networking (floating IPs, etc.)
  • Access rescue mode if needed

Cost Management:

  • Servers billed directly by Hetzner
  • RunOS does not mark up cloud costs
  • You retain full control of your Hetzner account

Important: Server Lifecycle:

  • RunOS does NOT automatically delete Hetzner servers when removing nodes
  • You must manually delete servers from the Hetzner panel
  • This prevents accidental data loss
  • You continue to be billed until servers are deleted from Hetzner
  • To delete: Hetzner Cloud Console → Servers → Select server → Delete

Node Types and Roles​

Kubernetes nodes can serve different roles in your cluster:

Control Plane Nodes​

Purpose: Manage the cluster itself

Responsibilities:

  • Run Kubernetes control plane components (API server, scheduler, controller manager)
  • Store cluster state in etcd database
  • Make scheduling decisions
  • Handle authentication and authorization

Requirements:

  • First node in cluster becomes control plane automatically
  • Additional control plane nodes for high availability
  • Recommended: 2GB+ RAM, 2+ CPU cores

When to add:

  • Your first node (required)
  • For high availability (3 or 5 control plane nodes recommended)

Worker Nodes​

Purpose: Run your applications and services

Responsibilities:

  • Execute pods containing your applications
  • Run databases, caches, and other services
  • Handle application workloads

Requirements:

  • More resources for application workloads
  • Scale based on your needs
  • Recommended: 4GB+ RAM, 2+ CPU cores

When to add:

  • When control plane reaches capacity
  • To distribute workloads
  • For horizontal scaling
  • For high availability

Hybrid Nodes​

Purpose: Both control plane and worker

When used:

  • Small clusters (1-3 nodes)
  • Development environments
  • Cost optimization

Note: RunOS configures nodes as hybrid by default in small clusters.

Adding Multiple Nodes​

High Availability Configuration​

For production clusters, add multiple nodes:

Recommended HA Setup:

  • 3 control plane nodes - Ensures cluster survives 1 node failure
  • 2+ worker nodes - Distributes workload and provides redundancy

Benefits:

  • Cluster survives individual node failures
  • Zero-downtime maintenance
  • Better performance distribution
  • Automatic workload redistribution

Adding Nodes to Existing Cluster​

To expand your cluster:

  1. Access Add Node from the Nodes page
  2. Choose addition method (manual or cloud provisioning)
  3. Follow installation steps for the new node
  4. Node automatically joins existing cluster
  5. Workloads can be scheduled on new node immediately

New nodes automatically:

  • Receive cluster configuration
  • Join the VPN mesh
  • Sync with control plane
  • Become available for workloads

Networking and Connectivity​

VPN Mesh Network​

All nodes connect via WireGuard VPN:

wg0 (Internal Network):

  • IP range: 172.24.0.0/16
  • Purpose: Kubernetes internal communication
  • All nodes automatically configured as peers

wg1 (User Access):

  • IP range: 172.24.200.0/21
  • Purpose: Secure administrator access
  • Optional remote access

Benefits:

  • Works behind NAT/firewalls
  • Encrypted node-to-node communication
  • No public IPs required
  • Automatic peer discovery

DNS Configuration​

Each node is configured with:

  • dnsmasq - Routes cluster DNS queries
  • systemd-resolved - System DNS resolver
  • Automatic service discovery
  • External DNS (Cloudflare, Google DNS)

Applications can resolve:

  • Other services by name (e.g., mysql-service.default)
  • External domains normally
  • Cluster-internal resources

Troubleshooting​

Node Won't Join Cluster​

Symptoms:

  • Installation completes but node stays offline
  • Node doesn't appear in RunOS console
  • kubectl doesn't show the node

Possible causes and solutions:

1. Network connectivity issues

# Test internet connectivity
ping -c 3 8.8.8.8

# Test DNS resolution
nslookup google.com

# Check if ports are blocked
nc -zv api.runos.com 443

2. VPN not connecting

# Check WireGuard status
sudo systemctl status wg-quick@wg0
sudo wg show

# Restart VPN
sudo systemctl restart wg-quick@wg0

3. Time sync issues

# Check system time
timedatectl

# Sync time if needed
sudo timedatectl set-ntp true

4. Installation errors

# Check node agent logs
sudo journalctl -u nodeagent -f

# Check kubelet logs
sudo journalctl -u kubelet -f

Installation Hangs or Fails​

Common issues:

1. Insufficient resources

  • Ensure server has minimum 2GB RAM
  • Check disk space: df -h
  • Monitor during install: htop

2. Network timeout

  • Check internet speed
  • Verify no firewall blocking outbound traffic
  • Try running installation again

3. Package conflicts

  • Use fresh Ubuntu installation
  • Remove existing Kubernetes: sudo kubeadm reset -f
  • Clean and retry installation

Node Shows as "Not Ready"​

Check node status:

kubectl get nodes
kubectl describe node <node-name>

Common causes:

1. Container runtime not running

sudo systemctl status containerd
sudo systemctl restart containerd

2. Kubelet not running

sudo systemctl status kubelet
sudo journalctl -u kubelet -f

3. Network plugin issues

kubectl get pods -n kube-system | grep cilium

Cannot SSH to Node After Installation​

For manually added nodes:

  • SSH access not modified by RunOS
  • Use same credentials as before installation
  • Check if firewall rules changed

For Hetzner provisioned nodes:

  • SSH access configured via Hetzner
  • Use SSH keys configured in Hetzner panel
  • Access via Hetzner console if needed

Security Considerations​

Network Security​

Inbound ports (can be blocked):

  • No inbound ports required for worker nodes behind NAT
  • VPN establishes outbound connections

Outbound ports (must allow):

  • 80, 443 (HTTPS)
  • 51820, 51821 (WireGuard)

Encryption:

  • All node-to-node traffic encrypted via VPN
  • Kubernetes API traffic encrypted with TLS
  • Secrets encrypted at rest

Access Control​

Root access:

  • Installation requires root/sudo
  • Node Agent runs with necessary privileges
  • Follow principle of least privilege

Credentials:

  • mTLS certificates for node authentication
  • Certificates automatically managed
  • Rotate certificates via RunOS console

Firewall Configuration​

For cloud servers:

  • Configure firewall in cloud provider panel
  • Allow outbound 80, 443, 51820, 51821
  • Block unnecessary inbound ports

For on-premises servers:

# Example using ufw (Ubuntu firewall)
sudo ufw allow out 80/tcp
sudo ufw allow out 443/tcp
sudo ufw allow out 51820/udp
sudo ufw allow out 51821/udp
sudo ufw enable

Next Steps​

After adding nodes to your cluster:

  1. Configure your cluster - Install essential services (storage, ingress, certificates)
  2. Deploy services - Add databases, caches, and other infrastructure
  3. Deploy applications - Add your custom applications
  4. Set up monitoring - Configure Grafana dashboards and alerts
  5. Configure backups - Protect your data with automated backups

Quick Reference Commands​

# Check node status
kubectl get nodes

# View node details
kubectl describe node <node-name>

# Check VPN status
sudo wg show

# Check system services
sudo systemctl status kubelet
sudo systemctl status containerd
sudo systemctl status wg-quick@wg0

# View logs
sudo journalctl -u kubelet -f
sudo journalctl -u nodeagent -f

# Test cluster connectivity
kubectl get pods -A
kubectl cluster-info