Linux DMVPN Hub

Set-Up
This section describes how the system was initially set up

Base Deployment
The base system was deployed from the AWS Debian image, upgraded to Debian 11 (Bullseye)

The AWS instance needs an elastic IP

You will also need to choose an IP range for the DMVPN tunnels. In the example below, we have used 10.255.193.x

You will also need to choos an IP range for wireguard tunnels. In the example below, we have used 10.255.192.x

We also deployed the google authenticator SSH-Google-Authentictor package for remote access

AWS inbound rules
The following inbound rules are needed for AWS (note that non-ipsec firewalls are limited to known IP addresses)
aws_dmvpn_firewall.png

Additional packages
apt install amazon-ssm-agent charon-systemd frr iptables-persistent strongswan strongswan-charon strongswan-starter strongswan-swanctl wireguard wireguard-tools

Wireguard setup
Incomplete
Wireguard set-up is well documented.

Create server pubic and private keys

Create /etc/wireguard/wg0.conf

[Interface]
Address = 10.255.192.1/24
PrivateKey =
ListenPort = 51820
Run the following commands
wg-quick up wg0 systemctl enable wg-quick@wg0

Network setup
Add the following to /etc/network/interfaces

auto dmvpnlo1
iface dmvpnlo1 inet static
pre-up ip link add $IFACE type dummy || true
address /32
post-down ip link del $IFACE || true

auto dmvpnhub1
iface dmvpnhub1 inet static
pre-up ip tunnel add $IFACE mode gre key 76282 local dev ens5 || true
address 10.255.193.1
netmask 255.255.255.255
mtu 1400
post-down ip tunnel del $IFACE || true
On our system, we also created a second dmvpnhub interface as follows to allow non-ipsec tunnels

# We cannot set this to auto because it causes a failure in cloud-init
# We have put a post-start command into the strongswan service file to start this up later
iface dmvpnhub2 inet static
pre-up ip tunnel add $IFACE mode gre key 76283 local dev ens5 || true
address 10.255.193.2
netmask 255.255.255.255
mtu 1400
post-down ip tunnel del $IFACE || true
IPTables
The following NAT rules allows the server to use the public IP address for ipsec and GRE. It should be possible to ge tthings working without this step, but we have things working and are not going to investigate this further.

Add the following to /etc/iptables/rules.v4

*filter
:INPUT ACCEPT [16793:2176851]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [16427:3499951]
:DMVPNIN - [0:0]
-A INPUT -i dmvpnhub* -j DMVPNIN
-A DMVPNIN -m state --state ESTABLISHED -j ACCEPT
-A DMVPNIN -p icmp -j ACCEPT
-A DMVPNIN -p tcp -m tcp --dport 80 -j ACCEPT
-A DMVPNIN -p tcp -m tcp --dport 443 -j ACCEPT
-A DMVPNIN -j REJECT --reject-with icmp-port-unreachable
COMMIT
*nat
:PREROUTING ACCEPT [273:18798]
:INPUT ACCEPT [304:19950]
:OUTPUT ACCEPT [501:37234]
:POSTROUTING ACCEPT [501:37234]
-A PREROUTING -d /32 -p udp -m udp --dport 500 -j DNAT --to-destination 13.210.251.141
-A PREROUTING -d /32 -p esp -j DNAT --to-destination
-A PREROUTING -d /32 -p gre -j DNAT --to-destination
-A POSTROUTING -s /32 -p udp -m udp --sport 500 -j SNAT --to-source
-A POSTROUTING -s /32 -p esp -j SNAT --to-source
-A POSTROUTING -s /32 -p gre -j SNAT --to-source
-A POSTROUTING -s /32 -p icmp -j SNAT --to-source
COMMIT
Enable the rules as follows:

systemctl enable netfilter-persistent.service
systemctl start netfilter-persistent.service
IPSec Setup
You need to disable the strongswan started service

systemctl disable strongswan-starter.service
systemctl stop strongswan-starter.service
You need to add the following to /etc/ipsec.secrets

%any : PSK ""
Create a file called /etc/swanctl/conf.d/dmvpn.conf as follows:

connections {
dmvpn {
version = 1
pull = no
dpd_delay = 15
dpd_timeout = 30
fragmentation = yes
unique = replace
reauth_time = 8h
proposals = aes256-sha1-modp1024
local {
auth = psk
id =
}
remote {
auth = psk
}
children {
dmvpn {
esp_proposals = aes256-sha1
local_ts = [gre]
remote_ts = dynamic[gre]
inactivity = 90m
rekey_time = 8h
mode = transport
dpd_action = clear
reqid = 1
start_action = start
}
}
}
}
We then need to modify the strongswan systemd startup script /etc/systemd/system/strongswan-swanctl.service

ExecStartPost=/usr/bin/sleep 10
We also added the follwing line to enable the non-ipsec DMVPN tunnel

ExecStartPost=sh -c "/usr/bin/test -d /sys/class/net/dmvpnhub2 || /usr/sbin/ifup dmvpnhub2"
And then disable/enable the appropriate services

systemctl daemon-reload
systemctl enable strongswan.service
systemctl start strongswan.service
systemctl disable strongswan-starter.service
systemctl stop strongswan-starter.service
Routing setup
Edit /etc/frr/daemons

bgpd=yes
nhrpd=yes
Edit the startup config for frr as follows

mkdir /etc/systemd/system/frr.service.d/
vi /etc/systemd/system/frr.service.d/deps.conf
[Unit]
Requires=strongswan.service
After=strongswan.service
systemctl daemon-reload
systemctl enable frr.service
systemctl start frr.service
Run vtysh and set the frr config as follows:

interface dmvpnhub1
ip nhrp holdtime 300
ip nhrp mtu 1400
ip nhrp network-id 2
ip nhrp nhs dynamic nbma
ip nhrp registration no-unique
no link-detect
tunnel protection vici profile dmvpn
And for non-IPSEC DMVPN

interface dmvpnhub2
ip nhrp holdtime 300
ip nhrp mtu 1400
ip nhrp network-id 3
ip nhrp nhs dynamic nbma
ip nhrp registration no-unique
no link-detect
Reboot
Reboot the server, and everything should be ready