Software-defined load balancing

Ruckos LB

A custom layer-4/layer-7 load balancer for a high-throughput environment.

  • Software-defined
  • Health-checked
  • Automatic failover
  • Real client address
  • Built in
Spread balancing in Ruckos LB One client opens three connections to the service address. Both hosts announce the address over BGP, so connections arrive at both. Ruckos LB at each host's network card hashes each connection on its own: connection 1 enters host 1 and goes to web-1 there, connection 2 enters host 2 and goes to web-3 there, and connection 3 enters host 1 and goes to web-3 on host 2 through an IP-in-IP tunnel. web-2 has failed its health check and is skipped. Each replica replies straight to the client. Host 1 Host 2 Client 198.51.100.7 Service address 203.0.113.1 : 443 Ruckos LB eBPF at the network card picks a replica Ruckos LB same hash, same choice web-1 connection 1 web-2 failed check web-3 connections 2 and 3 IP-in-IP Connections arrive at every host, and each one is hashed on its own.
Request, through the load balancer Reply, straight to the client
Ruckos LB

Load balancing
as part of the platform

Ruckos LB is part of Ruckos Linux. Create an address for a service and it is balanced: there is no appliance to add and nothing extra to install.

Runs on your servers

Ruckos LB runs on the hosts that run your services. Its capacity grows with the cluster, and it is managed like everything else on the platform.

  • Any enterprise server
  • Nothing extra to install
  • The same API as hosts, VMs and containers

A short path, in the kernel

Connections are balanced by an eBPF program at the network card, before they reach the rest of the system. Replies go straight from the service to the client.

  • eBPF at the network card
  • Replies return directly to the client
  • The service sees the real client address

Layer 4 for everything

Anything that speaks TCP or UDP can sit behind an address: web front ends, APIs, databases and caches.

  • HTTP and gRPC
  • Postgres, Redis and other databases
  • Several ports on one address
How it works

From the client
to a healthy replica

Every host in the cluster balances. A connection that arrives for a service's address is sent to a healthy replica, on that host or on another one. Follow one packet through it.

  1. The client connects to one address

    Each service gets a stable address from a range you set once for the cluster. Every host with a healthy replica announces the address over BGP, so connections arrive across the hosts.

    From198.51.100.7 To203.0.113.1:443
  2. The network card picks a replica

    An eBPF program at the host's network card hashes the connection and picks a healthy replica. A replica on the same host gets the packet as it arrived.

    From198.51.100.7 To203.0.113.1:443 unchanged
  3. Or sends it to another host

    When the replica is on another host, the packet is wrapped and sent there. That host unwraps it and delivers the original to its replica.

    Host to host10.1.1.11 → 10.1.1.12 From198.51.100.7 To203.0.113.1:443
  4. The replica answers the client directly

    The reply leaves with the service address as its source and takes the shortest way out. It does not pass back through the load balancer.

    From203.0.113.1:443 To198.51.100.7
Consistent hashing
Connections spread evenly across replicas, and every host makes the same choice.
Connection tracking
An open connection stays on its replica while that replica lives.
Address range
Set once per cluster and kept clear of the cluster's DHCP range.
Built-in test
See which replica answered each connection, from the console or the CLI.
Balancing

Pick how each service
is balanced

Balancing is set per service and can be changed later. A web tier, an application and a database each get what suits them.

Spread

Each connection goes to any healthy replica, for even load across the service.

For stateless APIs and web tiers

Sticky

Every connection from one client reaches the same replica. It moves only when that replica goes away.

For front ends and apps that keep sessions

Primary

All traffic goes to one replica, the primary. A healthy standby takes over when it fails.

For databases and other single-writer services
Staying up

A failed replica or host
drops out on its own

Each host checks the replicas it runs and acts on the result at once. The service address stays where clients expect it.

Health checks

  • A TCP check every second
  • An HTTP check on a path you name
  • A failing replica stops receiving connections
  • It returns once it passes again

Failover

  • A lost host's addresses move to another host
  • Takeover in seconds
  • Replicas on the other hosts keep serving

Upgrades

  • A draining host hands its addresses over first
  • Replicas upgrade one at a time
  • New connections go to the others meanwhile
  • Databases: standbys first, the primary last
Management

Set it up
in a minute

Set the address range once for the cluster, then create an address for each service from the console or the CLI.

rctl
# the address range, once per cluster
$ rctl weir range 192.168.1.11 10.100.0.1 10.100.0.254

# an address for a service, with its ports
$ rctl weir vip create 192.168.1.11 --service web \
    --port 80 --port 443 --tier frontend

# an HTTP health check, and how it balances
$ rctl weir vip health 192.168.1.11 web /healthz
$ rctl weir vip balancing 192.168.1.11 web sticky

# open 50 connections and see which replica took each
$ rctl weir test 192.168.1.11 web --requests 50
Ruckos products

Built to work together

One operating system underneath all of them.