Hello all, I tried to implement DMVPN with Quagga nhrpd & Strongswan. The nhrp doesn't work. I have followed the NHRP & DMVPN document from: http://docs.frrouting.org/en/latest/nhrpd.html I have used the patch from: https://gitlab.alpinelinux.org/alpine/aports/-/tree/master/main/strongswan Following are some details: 1. The NHRPD create an ipsec connection that seems to be working well. (ipsec statusall report of an established connection) 2. The nhrp registration request is sent inside the secure channel. Is that correct behavior? 3. On the spoke I get frequent messages of: 2020/09/13 09:03:39 NHRP: Send Registration-Request(3) 20.20.20.12 -> 20.20.20.12 2020/09/13 09:03:41 NHRP: NHS: Register 20.20.20.12 -> 20.20.20.12 (timeout 4) 4. I get the follow show status on the spoke: SF1v# show ip nhrp nhs Iface FQDN NBMA Protocol gre1 30.30.30.11 30.30.30.11 (unspec) SF1v# show ip nhrp cache Iface Type Protocol NBMA Flags Identity gre1 local 20.20.20.12 - - SF1v# show dmvpn Src Dst Flags SAs Identity 30.30.30.12 30.30.30.11 n 1 30.30.30.11 5. I get the follow show status on the HUB: SF1v# show ip nhrp nhs Iface FQDN NBMA Protocol gre1 30.30.30.11 - (unspec) SF1v# show ip nhrp cache Iface Type Protocol NBMA Flags Identity gre1 local 20.20.20.11 - - SF1v# show dmvpn Src Dst Flags SAs Identity 30.30.30.11 30.30.30.12 1 30.30.30.12 ================================================= 6. HUB configuration: ================================================= #=============== IPSEC CONFIGURATION ================= echo " "> /etc/ipsec.conf echo "config setup ">> /etc/ipsec.conf echo "conn dmvpn">> /etc/ipsec.conf echo " authby=secret ">> /etc/ipsec.conf echo " auto=add ">> /etc/ipsec.conf echo " keyexchange=ikev2 ">> /etc/ipsec.conf echo " ike=aes256-aes256-sha256-modp2048 ">> /etc/ipsec.conf echo " esp=aes256-aes256-sha256-modp2048 ">> /etc/ipsec.conf echo " dpdaction=clear ">> /etc/ipsec.conf echo " dpddelay=300s ">> /etc/ipsec.conf echo " left=%any ">> /etc/ipsec.conf echo " leftid=%any ">> /etc/ipsec.conf echo " right=%any ">> /etc/ipsec.conf echo " rightid=%any ">> /etc/ipsec.conf echo " leftprotoport=gre ">> /etc/ipsec.conf echo " rightprotoport=gre ">> /etc/ipsec.conf echo " type=transport ">> /etc/ipsec.conf echo " keyingtries=%forever ">> /etc/ipsec.conf echo "# ipsec.secrets - strongSwan IPsec secrets file" > /etc/ipsec.secrets echo "%any : PSK \"rami\"" >> /etc/ipsec.secrets ipsec rereadall ipsec start #=============== clean config ================= rm /opt/smartswitch/etc/quagga/nhrpd0.conf #=============== interface config ================= ip link add name eth4.20 link eth4 type vlan id 20 ip address add 30.30.30.11/255.255.255.0 dev eth4.20 ip link set dev eth4.20 up ip tunnel add gre1 mode gre key 42 dev eth4.20 ttl 64 ip addr add 20.20.20.11/32 dev gre1 ip link set gre1 up iptables -A FORWARD -i gre1 -o gre1 -m hashlimit --hashlimit-upto 4/minute --hashlimit-burst 1 --hashlimit-mode srcip,dstip --hashlimit-srcmask 24 --hashlimit-dstmask 24 --hashlimit-name loglimit-0 -j NFLOG --nflog-group 1 touch /opt/smartswitch/etc/quagga/nhrpd0.conf nhrpd -f /opt/smartswitch/etc/quagga/nhrpd0.conf -i /var/run/nhrpd0.pid -P 3000 start & # Quagga nhrp config on HUB vtysh configure terminal log syslog debug nhrp common nhrp nflog-group 1 interface gre1 description DMVPN Tunnel Interface ip address 20.20.20.11/32 ip nhrp network-id 1 ip nhrp redirect ip nhrp registration no-unique ip nhrp shortcut # no link-detect tunnel protection vici profile dmvpn tunnel source eth4.20 router bgp 65000 bgp router-id 20.20.20.11 no bgp ebgp-requires-policy neighbor SPOKES peer-group neighbor SPOKES disable-connected-check neighbor 20.20.20.12 remote-as 65001 neighbor 20.20.20.12 peer-group SPOKES address-family ipv4 unicast network 11.11.11.11/24 redistribute nhrp exit-address-family end exit ================================================= 7. SPOKE configuration: ================================================= #=============== IPSEC CONFIGURATION ================= echo " "> /etc/ipsec.conf echo "config setup ">> /etc/ipsec.conf echo "conn dmvpn">> /etc/ipsec.conf echo " authby=secret ">> /etc/ipsec.conf echo " auto=add ">> /etc/ipsec.conf echo " keyexchange=ikev2 ">> /etc/ipsec.conf echo " ike=aes256-aes256-sha256-modp2048 ">> /etc/ipsec.conf echo " esp=aes256-aes256-sha256-modp2048 ">> /etc/ipsec.conf echo " dpdaction=clear ">> /etc/ipsec.conf echo " dpddelay=300s ">> /etc/ipsec.conf echo " left=%any ">> /etc/ipsec.conf echo " leftid=%any ">> /etc/ipsec.conf echo " right=%any ">> /etc/ipsec.conf echo " rightid=%any ">> /etc/ipsec.conf echo " leftprotoport=gre ">> /etc/ipsec.conf echo " rightprotoport=gre ">> /etc/ipsec.conf echo " type=transport ">> /etc/ipsec.conf echo " keyingtries=%forever ">> /etc/ipsec.conf echo "# ipsec.secrets - strongSwan IPsec secrets file" > /etc/ipsec.secrets echo "%any : PSK \"rami\"" >> /etc/ipsec.secrets ipsec rereadall ipsec start #=============== clean config ================= rm /opt/smartswitch/etc/quagga/nhrpd0.conf #=============== interface config ================= ip link add name eth4.20 link eth4 type vlan id 20 ip address add 30.30.30.12/255.255.255.0 dev eth4.20 ip link set dev eth4.20 up ip tunnel add gre1 mode gre key 42 dev eth4.20 ttl 64 ip addr add 20.20.20.12/32 dev gre1 ip link set gre1 up touch /opt/smartswitch/etc/quagga/nhrpd0.conf nhrpd -f /opt/smartswitch/etc/quagga/nhrpd0.conf -i /var/run/nhrpd0.pid -P 3000 start & # quagga nhrp config on spoke vtysh configure terminal log syslog debug nhrp common nhrp nflog-group 1 interface gre1 description DMVPN Tunnel Interface #config of HUB GRE IP ip address 20.20.20.12/32 ip nhrp network-id 1 ip nhrp nhs dynamic nbma 30.30.30.11 ip nhrp redirect ip nhrp registration no-unique ip nhrp shortcut no link-detect tunnel protection vici profile dmvpn tunnel source eth4.20 router bgp 65001 bgp router-id 20.20.20.12 no bgp ebgp-requires-policy neighbor 20.20.20.11 remote-as 65000 neighbor 20.20.20.11 disable-connected-check address-family ipv4 unicast network 12.12.12.12/24 exit-address-family end exit ================================================= 8. Ipsec status on HUB # ipsec statusall Status of IKE charon daemon (strongSwan 5.8.4, Linux 4.19.125, armv7l): uptime: 33 minutes, since Sep 13 09:28:12 2020 malloc: sbrk 778240, mmap 0, used 355704, free 422536 worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 3 loaded plugins: charon pkcs11 aes des rc2 sha2 sha1 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf curve25519 xcbc cmac hmac drbg curl attr kernel-netlink resolve socket-default stroke vici updown xauth-generic led counters Listening IP addresses: 10.10.10.11 11.11.11.11 30.30.30.11 20.20.20.11 Connections: dmvpn: %any...%any IKEv2, dpddelay=300s dmvpn: local: uses pre-shared key authentication dmvpn: remote: uses pre-shared key authentication dmvpn: child: dynamic[gre] === dynamic[gre] TRANSPORT, dpdaction=clear Security Associations (1 up, 0 connecting): dmvpn[1]: ESTABLISHED 32 minutes ago, 30.30.30.11[30.30.30.11]...30.30.30.12[30.30.30.12] dmvpn[1]: IKEv2 SPIs: 942411e640760acf_i c5c66aa6073921f8_r*, pre-shared key reauthentication in 2 hours dmvpn[1]: IKE proposal: AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048 dmvpn{1}: INSTALLED, TRANSPORT, reqid 1, ESP SPIs: c33ae7b3_i cd79d565_o dmvpn{1}: AES_CBC_256/HMAC_SHA2_256_128, 9600 bytes_i, 0 bytes_o, rekeying in 13 minutes dmvpn{1}: 30.30.30.11/32[gre] === 30.30.30.12/32[gre] ================================================= ================================================= 8. Ipsec status on spoke: / # ipsec statusall Status of IKE charon daemon (strongSwan 5.8.4, Linux 4.19.125, armv7l): uptime: 32 minutes, since Sep 13 09:28:20 2020 malloc: sbrk 778240, mmap 0, used 357808, free 420432 worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 4 loaded plugins: charon pkcs11 aes des rc2 sha2 sha1 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf curve25519 xcbc cmac hmac drbg curl attr kernel-netlink resolve socket-default stroke vici updown xauth-generic led counters Listening IP addresses: 10.10.10.12 12.12.12.12 30.30.30.12 20.20.20.12 Connections: dmvpn: %any...%any IKEv2, dpddelay=300s dmvpn: local: uses pre-shared key authentication dmvpn: remote: uses pre-shared key authentication dmvpn: child: dynamic[gre] === dynamic[gre] TRANSPORT, dpdaction=clear Security Associations (1 up, 0 connecting): dmvpn[1]: ESTABLISHED 31 minutes ago, 30.30.30.12[30.30.30.12]...30.30.30.11[30.30.30.11] dmvpn[1]: IKEv2 SPIs: 942411e640760acf_i* c5c66aa6073921f8_r, pre-shared key reauthentication in 2 hours dmvpn[1]: IKE proposal: AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048 dmvpn{1}: INSTALLED, TRANSPORT, reqid 1, ESP SPIs: cd79d565_i c33ae7b3_o dmvpn{1}: AES_CBC_256/HMAC_SHA2_256_128, 0 bytes_i, 9100 bytes_o (91 pkts, 64s ago), rekeying in 12 minutes dmvpn{1}: 30.30.30.12/32[gre] === 30.30.30.11/32[gre] ================================================= Any help would be much appreciated. Best regards, Rami