Debian 13 foundation
Carrier Router now uses Debian 13 (Trixie). A newer base does not mean higher throughput, but brings numerous security fixes and improvements. We will also aim to keep pace with newer kernel releases to maintain system security.
CARRIER ROUTER / OUR DIRECTION
Software first.
Carrier ambition.
Our mission: build a software-first, carrier-grade router capable of competing with dedicated hardware.
Our mission is an engineering goal, not a performance certification.
PROJECT ORIGIN
Carrier Router originates from the VyOS 1.5 development branch (rolling), with substantial subsequent rewrites assisted by GPT-6 Astra. It is an independent project, not an official VyOS release.
BEYOND THE BASELINE
A familiar foundation. A different direction.
Reworked configuration storage
Configuration is stored as independently serialized modules with immutable revisions and an atomically switched current revision. Committed state remains distinct from saved state, with migration-version metadata preserved for upgrades. This is a configuration-store redesign, not a new filesystem.
Commit and recovery controls
Validate with commit check, inspect per-component boot results, and optionally save committed changes on a timer. Auto-save skips unconfirmed commit-confirm transactions. An optional FRR memory ceiling helps contain routing-service memory use; recovery is not a guarantee of uninterrupted forwarding.
Deeper kernel and VPP integration
Extensions include VPP WireGuard source selection, PPPoE handover, interface-name mapping and unified monitoring commands. VyOS also has VPP work; the distinction is these extensions, not exclusive ownership of VPP support. VPP connection monitoring is sampled, not exact accounting.
BGP policy composition
Reuse ordered route-map combinations rather than duplicating every combination as a separate policy. Configuration and forwarding changes still require topology-specific validation.
Versioned upgrade paths
Compatible changes can use cumulative update packages within a supported baseline. Kernel or incompatible native-component changes require a new baseline and may require a reboot or service restart; not every ISO is a reboot-free upgrade.
SOURCE & LICENSES
We keep the internal development repository and full Git history private. For components whose licenses require source disclosure, each ISO release is accompanied by the complete corresponding source, modifications, necessary build materials and license notices as independent downloads. This does not mean publishing only patches where the license requires complete corresponding source.
Corresponding source downloadsBuilt around the way you operate.
Focused extensions to the commands you already know. No new management layer to learn.
THE OPERATOR’S FIELD GUIDE
Less guesswork.
More working commands.
Choose a topic. Understand the change. Take the command with you.
Reference build exp68 · 10 Sep 2026
Unified WAN handover, including dual PPPoE
Use static/static, static/PPPoE or PPPoE/PPPoE members with source-bound ICMP and TCP checks.
01Manage two established PPPoE sessions
Choose this example or the fixed-WAN example below; do not bind the same tunnel to both groups. Both PPP clients must already be configured and connected, in separate VRFs, with an ISP allowing simultaneous sessions. VPP WireGuard needs dataplane vpp on both PPP clients. Replace the documentation health target with a reachable address. This chooses between available sessions; it does not delay the second dial or rotate every 36 hours. Verify access before saving.
configure
set service wan-handover group DUALPPP member pppoe1
set service wan-handover group DUALPPP member pppoe2
set service wan-handover group DUALPPP member-options pppoe1 priority 200
set service wan-handover group DUALPPP member-options pppoe2 priority 100
set service wan-handover group DUALPPP health-check target reach address 192.0.2.1
set service wan-handover group DUALPPP health-check target reach protocol icmp
set service wan-handover group DUALPPP preempt enable
set service wan-handover group DUALPPP preempt-delay 60
set service wan-handover group DUALPPP minimum-residence 120
set interfaces vpp wireguard vppwg0 source-group DUALPPP
compare
commit check
commit02Prefer a fixed WAN and recover deliberately
Higher priority wins. Omitted priorities default to 100, with ties resolved by stored member insertion order. Preemption defaults to disabled. Replace documentation addresses with reachable targets you control; interface and tunnel configuration must already exist.
set service wan-handover group CHT member eth4
set service wan-handover group CHT member pppoe0
set service wan-handover group CHT member-options eth4 priority 200
set service wan-handover group CHT member-options pppoe0 priority 100
set service wan-handover group CHT health-check target reach address 192.0.2.1
set service wan-handover group CHT health-check target reach protocol icmp
set service wan-handover group CHT preempt enable
set service wan-handover group CHT preempt-delay 60
set service wan-handover group CHT minimum-residence 120
set interfaces vpp wireguard vppwg0 source-group CHT03Delay only the first PPPoE dial
PPPoE only; seconds, here 36 hours. Each router boot starts a new initial delay; restarting the supervisor or editing a description does not. Existing connected clients are not stopped, and subsequent reconnects are not delayed. An explicit failure of the active WAN starts a delayed backup early. The timer does not force selection or change priority. Recovery/preemption remains policy-controlled. Carrier loss and explicit PPP disconnect skip scoring and hold timers, but a backup must be usable; no zero-loss or zero-time guarantee.
set service wan-handover group DUALPPP member pppoe2 rotation-interval 129600
commit check
commit
run show wan-handover group DUALPPP04Repair PPPoE spacing after a shared outage (exp90)
Requires exp90 and an already configured group with exactly two persistent PPPoE members sharing the same source-interface, service/concentrator and dataplane. Do not apply to the fixed/mixed WAN example. Both clients reconnect normally after an outage. If actual session start times are less than one hour apart, renew only the inactive member after the active session reaches 36 hours, provided both remain qualified for 60 seconds and configured inner IPv4/IPv6 tunnel checks pass. Avoid the survivor’s expected 72-hour expiry. Unknown evidence blocks renewal; a daemon restart does not repeat the same attempt. Correct spacing does not cause periodic rotation. Direct non-group traffic on the renewed interface may be interrupted. Initial-dial delay remains a separate option.
set service wan-handover group DUALPPP pppoe-restagger interval 129600
set service wan-handover group DUALPPP pppoe-restagger minimum-gap 3600
set service wan-handover group DUALPPP pppoe-restagger stable-time 60
set service wan-handover group DUALPPP pppoe-restagger session-lifetime 259200
commit check
commit
run show wan-handover group DUALPPP05Compare average TCP connection latency
Measures TCP connect time, not TLS or HTTP response time. Compare the same named target on all WANs after collecting ten consecutive successful samples per path; use their moving arithmetic mean, not median. An average improvement of at least 3 ms qualifies; set improvement-ms 1 to honor a 1 ms threshold. Percentage is disabled by default and only adds a second threshold when explicitly enabled. Stability timers still apply and preempt must be enabled. Failed checks reset the affected window, never count as zero latency. ICMP/TCP health does not prove an ISP sends WireGuard UDP over the same path.
set service wan-handover group CHT health-check target web address 198.51.100.1
set service wan-handover group CHT health-check target web protocol tcp
set service wan-handover group CHT health-check target web port 443
set service wan-handover group CHT selection-mode latency
set service wan-handover group CHT latency target web
set service wan-handover group CHT latency window 10
set service wan-handover group CHT latency improvement-ms 3
set service wan-handover group CHT latency improvement-percent 0
set service wan-handover group CHT latency hold-time 3006Use a percentage instead of milliseconds
Replaces the previous ms example. Percentage is measured against the current WAN average: 10 ms to 9 ms meets 10%. With ms omitted and percentage enabled, there is no hidden 3 ms floor. Configuring both thresholds requires both to pass (AND); set improvement-ms 3 again to add that requirement. Explicit zero disables its threshold. Delete improvement-percent to return to ms-only. Sampling and stability settings still apply.
delete service wan-handover group CHT latency improvement-ms
set service wan-handover group CHT latency improvement-percent 1007Attach kernel WireGuard explicitly
Security scope: bind-udp-to-all allows unbound UDP listeners, not only WireGuard, to receive across VRFs. Bind other UDP services explicitly or filter their ingress. TCP is not enabled by this option. The controller does not enable it automatically; VPP WireGuard does not require it.
set vrf bind-udp-to-all
set interfaces wireguard wg0 source-group CHT08Inspect selection and health (exp90.1)
Default output is a table with member roles, link/health, RTT and first-dial countdown. pending / waiting means scheduled dialing, not failure. Automation must explicitly select json; a selected group returns an object, all groups an array. These commands only inspect state and do not trigger a switch or redial.
show wan-handover group CHT
show wan-handover group CHT json
show wan-handover jsonSource-checked against exp90.3
Follow a WAN group from a static route
Inherit the selected member interface and next-hop VRF without hard-coding pppoe0.
01Reference the active group
If replacing an existing interface route, remove its old interface selector in the same candidate transaction before committing.
set protocols static route 100.64.100.64/32 group CHT02Resolve an ambiguous fixed-WAN gateway
Optional: Ethernet normally derives a next-hop from its own VRF routes. Specify a gateway when that choice is ambiguous. PPPoE uses its point-to-point interface and needs no gateway here. This does not configure the member VRF default route or health-check reachability for you.
set service wan-handover group CHT member-options eth4 gateway 192.0.2.254Source-checked against exp90.3
Monitor connections with iftop
Use configured interface names to select kernel traffic or sampled VPP IP flows.
01Kernel interface
Use an interface owned by the kernel.
monitor connections interface eth402VPP WireGuard interface
The selected interface must exist in the VPP configuration. Completion uses configured names rather than internal wg/tap names.
monitor connections vpp interface vppwg0Source-checked against exp90.3
Validate before committing
Check the candidate configuration before applying it, including BGP requirements and route-map references.
01Check, commit, then save
Run commit and save only after reviewing the check result. The example addresses are documentation-only; replace them for your network.
configure
set protocols bgp system-as 65001
set protocols bgp neighbor 192.0.2.2 remote-as 65002
set protocols bgp neighbor 192.0.2.2 address-family ipv4-unicast
commit check
commit
save
exit02Keep explicit confirmation separate
Running commit check never confirms the transaction, and auto-save skips an unconfirmed commit. After committing, independently verify access and network health before continuing.
commit check
commit-confirm 503Confirm only after verifying connectivity
Run these commands only after the independent checks above pass. Otherwise, do not confirm; allow the configured recovery action to run.
confirm
saveSource-checked against exp90.3
Optional configuration auto-save
Choose a save interval in minutes; only successfully committed running configuration is persisted.
01Enable a ten-minute interval
Save explicitly once to retain the timer setting across an immediate reboot. After that, successful commits are saved at the configured interval.
configure
set system config-management auto-save interval 10
commit
save
exit02Inspect status and both running views
The last two commands read the same running configuration, including unsaved commits; only the output format differs.
show configuration auto-save
show configuration
show configuration commands03Disable automatic saving
Deleting the setting stops the timer. Save explicitly to keep it disabled after reboot.
configure
delete system config-management auto-save
commit
save
exitSource-checked against exp90.3
A modern shell, a familiar CLI
GNU Bash 5.3.15 powers the Carrier CLI, with native bracketed paste and the carrier-cli entry point from exp66.
01Check the installed CLI
Run as an administrator in operational mode. Tab completion, ? help and command permissions remain available. Native paste is initialized at login, including for accounts retained from an older image; no manual escape-sequence reset is required.
show version
/bin/carrier-cli --versionSource-checked against exp90.3
The configuration workflow
Make a focused change, review it, then commit and save.
01Review, apply, persist
This example assumes vppwg0 already exists and is configured.
configure
set interfaces vpp wireguard vppwg0 description 'Transit tunnel'
compare
commit
save
exit02Inspect the running system
show version
show system configuration-status
show interfacesSource-checked against exp90.3
Compose BGP route-maps
Attach several reusable policies to one network declaration.
01Define reusable policies
configure
set policy route-map cn rule 10 action permit
set policy route-map cn rule 10 set community add '64512:156'
set policy route-map tw rule 10 action permit
set policy route-map tw rule 10 set community add '64512:158'
set policy route-map jp rule 10 action permit
set policy route-map jp rule 10 set community add '64512:392'
set protocols bgp address-family ipv4-unicast network 203.0.113.0/24 route-map 'cn and tw and jp'
commit
save
exit02Remove tw from this network only
Replace the expression. The tw policy object remains available to other networks. Carrier Router regenerates the internal chain; you do not edit its generated name. Policy evaluation occurs when BGP processes routes, not for each forwarded packet.
configure
set protocols bgp address-family ipv4-unicast network 203.0.113.0/24 route-map 'cn and jp'
commit
save
exitSource-checked against exp90.3
Dual-stack VPP WireGuard
Use IPv4 and IPv6 together without an extra tunnel-address on point-to-point subnets.
01Configure a point-to-point tunnel
Configure the peer with 10.77.0.1/30 and 2001:db8:77::/127, matching keys, endpoint and routing. The inferred peer host routes cover both address families.
configure
set interfaces vpp wireguard vppwg0 address '10.77.0.2/30'
set interfaces vpp wireguard vppwg0 address '2001:db8:77::1/127'
set interfaces vpp wireguard vppwg0 private-key '<LOCAL_PRIVATE_KEY>'
set interfaces vpp wireguard vppwg0 source-interface 'eth0'
set interfaces vpp wireguard vppwg0 peer remote public-key '<PEER_PUBLIC_KEY>'
set interfaces vpp wireguard vppwg0 peer remote endpoint '192.0.2.2'
set interfaces vpp wireguard vppwg0 peer remote port '51820'
set interfaces vpp wireguard vppwg0 peer remote allowed-ips '0.0.0.0/0'
set interfaces vpp wireguard vppwg0 peer remote allowed-ips '::/0'
set interfaces vpp wireguard vppwg0 peer remote persistent-keepalive '25'
commit
save
exit02Verify the tunnel
sudo vppctl show wireguard peer
sudo ping -I vppwg0 -c 3 10.77.0.1
sudo ping -6 -I vppwg0 -c 3 2001:db8:77::Source-checked against exp90.3
Follow a changing source IP
Choose source-interface or source-address. Configure exactly one.
01Switch from a fixed IP to an interface
For an existing source-address configuration. If the address comes from PPPoE, replace eth0 with the configured PPPoE interface name. The source is resolved from the interface and refreshed by network events.
configure
delete interfaces vpp wireguard vppwg0 source-address
set interfaces vpp wireguard vppwg0 source-interface 'eth0'
commit
save
exit02Use a fixed source instead
The fixed address must belong to the router and be reachable through the intended underlay.
configure
delete interfaces vpp wireguard vppwg0 source-interface
set interfaces vpp wireguard vppwg0 source-address '192.0.2.1'
commit
save
exitSource-checked against exp90.3
Separate tunnel and underlay routing
Keep the endpoint in the default table while placing decrypted traffic in a VRF.
01Put tunnel traffic in TENANT
configure
set vrf name TENANT table '100'
set interfaces vpp wireguard vppwg0 vrf 'TENANT'
set interfaces vpp wireguard vppwg0 underlay-vrf 'default'
commit
save
exitSource-checked against exp90.3
One bandwidth view, two dataplanes
Monitor VPP counters or kernel interfaces with Carrier bmon.
01VPP interface
monitor bandwidth vpp interface eth402Kernel interface
monitor bandwidth interface eth003Direct Carrier bmon invocation
braille=0 uses the traditional bar-style graph. The bmon compatibility command is also available in the image.
carrier-bmon --use-bit -i vpp -p eth4 -o curses:braille=0Source-checked against exp90.3
Test two VPP interfaces
Send a bounded UDP stream from eth4 and observe the peer on eth6.
01Receiver · second router
Read the peer MAC from its interface details before starting. Keep this monitor visible while generating traffic on the first router.
monitor bandwidth vpp interface eth602Sender · first router
Repeat with identical packet sizes, rates, CPU allocation and link settings when comparing configurations. Increase the requested rate gradually and measure the received rate and loss; requested rate is not measured throughput.
packet-generator vpp interface eth4 destination-mac 02:00:00:00:00:06 source-address 198.18.0.1 destination-address 198.18.0.2 packet-size 64 rate 10kpps duration 1003Inspect or stop the stream
show vpp packet-generator
stop packet-generator vpp interface eth4Source-checked against exp90.3
Set an FRR memory ceiling
Put an optional limit around the FRR service as a whole.
01Set a ceiling
500MB is an example, not a sizing recommendation for full routes. Use 2GB or another suitable value after measuring your workload.
configure
set service frr maxium-memory-size 500MB
commit
save
exit02Inspect the effective limit
sudo systemctl show frr.service --property=MemoryCurrent,MemoryMax03Remove this optional ceiling
When unset, Carrier Router adds no memory ceiling. Other administrator or host-level limits can still apply.
configure
delete service frr maxium-memory-size
commit
save
exitSource-checked against exp90.3
See which boot component failed
Read per-component results and repair the affected configuration. In exp85, missing OSPF/OSPFv3 interface references are skipped individually during boot, including VRFs; valid interfaces continue running. Partial results retain the original editable settings, and saving stays protected until repair.
01Inspect component results
Look for OK or Failed beside System, SSH, BGP and other configured components. Detailed diagnostics are identified in the command output.
show system configuration-status02Repair a single BGP setting
Example only: the BGP instance and cn/jp policies must exist. Fix the reported cause, not an unrelated field. If save is refused after a degraded boot, complete recovery of the preserved failed sections first; do not bypass the save safeguard.
configure
set protocols bgp address-family ipv4-unicast network 203.0.113.0/24 route-map 'cn and jp'
compare
commit
save
exitSource-checked against exp90.3
Resolve next hops via a default route
IPv4 and IPv6 default-route NHT resolution is enabled unless explicitly opted out.
01Opt out explicitly
configure
set system ip nht no-resolve-via-default
set system ipv6 nht no-resolve-via-default
commit
save
exit02Restore the Carrier Router default
configure
delete system ip nht no-resolve-via-default
delete system ipv6 nht no-resolve-via-default
commit
save
exitSource-checked against exp90.3
No matching topics.
Choose a topic to view its examples.
CARRIER ROUTER / EXP90.3
Download Carrier Router
Experimental exp90.3 · AMD64 · 20 September 2026
Carrier Router exp90.3
784.0 MiB · x86-64 · BIOS / UEFI
Download ISO ↓carrier-router-2026.09.20-exp90.3-generic-amd64.iso
ISO SHA-256
99b5a93e95c99fd4a43319112689ee3d7610c6a7b08e4ae3f8232f55918f1c4bVerify SHA-256 after downloading
Corresponding source
License-required source and necessary build materials are available as standalone archives, without access to internal repositories. Modifications to independent permissively licensed components that do not require source disclosure are not included; their licenses and notices still apply.
Source & build materials
Carrier modifications & build scriptscarrier-modified-sources-exp90.3.tar.gz · 29.4 MiBMatching Debian DNS sourcescarrier-updated-sources-exp90.3.tar.gz · 5.8 MiBOpenVPN 2.7.7 corresponding sourcescarrier-updated-sources-exp82.tar.gz · 2.1 MiBUpdated upstream corresponding sourcescarrier-updated-sources-exp80.tar.gz · 120.6 MiBKernel, drivers & third-party sourcescarrier-third-party-sources-exp66.tar · 340.3 MiBDebian corresponding source setscarrier-debian-sources-exp66.tar · 1097.2 MiBFAQ
A few things worth knowing.
Can I download an ISO here?
Yes. The Downloads section provides the Carrier Router exp90.3 AMD64 ISO, corresponding source archives, license notices and SHA-256 checksums. No account is required. Read the upgrade prerequisites before installing.
Are the examples production-ready defaults?
No. Addresses, keys, communities and interface names are examples. Check the notes for each command and adapt them to your topology.
Does VPP packet-generator replace TRex?
No. It is a built-in traffic-generation tool for controlled interface tests. These examples do not constitute a certified throughput, latency or RFC 2544 benchmark.
Does every extension also apply to kernel interfaces?
No. VPP-specific commands are explicitly labeled. In particular, the source-interface examples here are for VPP WireGuard; do not assume the same CLI exists for kernel WireGuard.
exp90.3 ISO now available
EXP90.3 / DOCSFixes native VPP packet-generator pacing after invalid time deltas. Retains the WAN handover configuration lock fix, readable tables and JSON output. Download the physically tested ISO, corresponding source and SHA-256 checksums. Read the upgrade prerequisites first.