Site-to-Site Vpn Between Oracle Cloud Infrastructure and a Mikrotik Device

Sat, Mar 25, 2023
7-minute read

Not many homelab enthusiast knows that Oracle still offers a pretty comprehensive package of cloud resources for free. One of those resources are IPSec-based Site-to-Site VPN tunnels, which can help you extend your local network to the cloud in a secure way.

About the OCI Always Free Tier

Here are some of the free services included in the Always Free tier of the Oracle Cloud Infrastructure (OCI) as of 2023 March:

Infrastructure

2 AMD based Compute VMs with 1/8 OCPU** and 1 GB memory each
Arm-based Ampere A1 cores and 24 GB of memory usable as 1 VM or up to 4 VMs with 3,000 OCPU hours and 18,000 GB hours per month
2 Block Volumes Storage, 200 GB total
10 GB Object Storage – Standard
10 GB Object Storage – Infrequent Access
10 GB Archive Storage
Resource Manager: managed Terraform
5 OCI Bastions

Additional Services

Flexible Load Balancer: 1 instance, 10 Mbps
Flexible Network Load Balancer
Outbound Data Transfer: 10 TB per month
Virtual Cloud Networks (VCN): Maximum of 2 VCNs, includes IPv4 and IPv6 support
VCN Flow Logs: Up to 10 GB per month shared across OCI Logging services
Site-to-Site VPN: 50 IPSec connections
Content Management Starter Edition: 5000 assets per month
Certificates: 5 Private CA and 150 private TLS certificates
Email Delivery: 100 emails sent per day

Please note that this list is not complete. For full details please visit https://www.oracle.com/cloud/free/.

According to the list above, you can get two small machines running in the cloud with some block storage completely free. What’s even better, you also get free IPSec connections, which allows you to add some extra computing power to your private network. I personally use these extra machines as kubernetes nodes (kubelets) in my k3s cluster, but there are endless applications for these tiny machines.

The Network

The router I’m using is a Mikrotik RB4011iGS+ v2 with RouterOS v7.8.

Home network: 192.168.0.0/16
VCN network: 10.0.0.0/16
VCN subnet: 10.0.0.0/24
Public IP of the Mikrotik router (fake): 181.182.75.164
In-tunnel IP (Oracle): 10.0.1.1
In-tunnel IP (Mikrotik): 10.0.1.2
IKE identifier of the Mikrotik router: 192.168.0.1

Configuring VPN in OCI

In this article I will assume that you have already set up your account with OCI and that you have already created at least a VM with a Virtual Cloud Network.

Add Customer-Premises Equipment

Navigate to Networking / Customer Connectivity / Customer-premises Equipment
Click on Create CPE
Add a meaningful name (I’ve used Mikrotik)
Enter your router’s public IP address in the Public IP Address box: 181.182.75.164
Select Other as your CPE vendor information

Create a Dynamic Routing Gateway

Note: You might have already generated this when you have created your VCN.

Navigate to Networking / Customer Connectivity / Dynamic Routing Gateways
Click on Create Dynamic Routing Gateway
Add a meaningful name (I’ve used drg)
Click on Create Virtual Cloud Network Attachment
Select your Virtual Cloud Network
Click on Create Virtual Cloud Network Attachment

Create Site-to-Site VPN

Navigate to Networking / Customer Connectivity / Site-to-Site VPN
Click on Create IPSec connection
Add a meaningful name (I’ve used home)
Check This CPE is behind a NAT device so that you can provide a fix CPE IKE Identifier
Add your router’s main address: 192.168.0.1
Select your previously created CPE (Mikrotik)
Select your previously created DRG (drg)
Type the route to your home network (192.168.0.0/16) into the Routes to your on-premises network box and hit Enter
For Routing type select Static routing
(Optional) Add 10.0.1.2/30 in IPv4 inside tunnel interface - CPE
(Optional) Add 10.0.1.1/30 in IPv4 inside tunnel interface - Oracle
Click on Create IPSec connection

Wait until the tunnels are provisioned and take notes of the values in the Oracle VPN IP address fields. Also take notes of the Shared secret values which you can find in the detail view.

Add Static Route to VCN

Navigate to Networking / Virtual cloud networks
Open the default VCN
Click on Route Tables
Open the default entry
Click on Add Route Rules
Select Dynamic Routing Gateway as the Target Type
Add your home network to the Destination CIDR Block box: 192.168.0.0/16
Click on Add Route Rules

Your account should now be ready to accept connections from your Mikrotik device. Let’s set up the router as well.

Setting Up the Mikrotik Device

If you get stuck, you can refer to the relevant RouterOS documentation here.

Cryptography

Add the IPSec profile and proposal configuration which define the available security protocols during Phase 1 and Phase 2 of IKE:

/ip ipsec profile
add dh-group=ecp256,ecp384,modp2048,modp1536,modp1024 enc-algorithm=aes-256,aes-192,aes-128 hash-algorithm=sha384 lifetime=8h name=oracle nat-traversal=no
/ip ipsec proposal
add auth-algorithms=sha256 enc-algorithms=aes-256-cbc,aes-256-gcm,aes-192-cbc,aes-192-gcm,aes-128-cbc,aes-128-gcm lifetime=1h name=oracle pfs-group=modp1536

The supported IPSec parameters are described here.

Peers and Policies

Add a new peer:

/ip ipsec identity
add peer=oracle-1 secret=<INSERT_YOUR_SECRET>

Define the IPSec policy:

/ip ipsec policy
add dst-address=10.0.0.0/16 peer=oracle-1 proposal=oracle src-address=192.168.0.0/16 tunnel=yes

(Optional): We can add a dedicated IP address which lives inside the tunnel. This is very useful for troubleshooting the tunnel itself:

/ip address
add address=10.0.1.2/30 comment="oracle ipsec" interface=bridge1 network=10.0.1.0
/ip ipsec policy
add dst-address=10.0.1.0/30 peer=oracle-1 proposal=oracle src-address=10.0.1.0/30 tunnel=yes

Firewall and NAT

Here we allow processing of the IPSec-related packages, and also allow traffic from OCI to access router-provided services (such as DNS):

/ip firewall filter
...
add action=accept chain=input comment="accept Oracle Cloud input" src-address=10.0.0.0/16
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN log-prefix=dropped
...
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
...
add action=accept chain=forward comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=forward comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec

If you want to allow the use of DNS, don’t forget to set the following:

/ip dns
set allow-remote-requests=yes

Since my local network is behind NAT, all outgoing packages are processed by the default srcnat rule. This broke the tunnel connection so I’ve explicitly excluded the IP range assigned to the VCN:

/ip firewall nat
add action=masquerade chain=srcnat comment="Internet NAT" dst-address=!10.0.0.0/16 ipsec-policy=out,none out-interface-list=WAN

Tunnels

Now we can add the details of the tunnels.

/ip ipsec peer
add address=132.145.250.122/32 local-address=192.168.0.1 name=oracle-1 profile=oracle

While you could setup multiple tunnels here, you would then also had to define load balancing between them which would make this setup a whole lot more complicated.

Testing

If you have specified the in-tunnel IP addresses, you can try testing the tunnel:

[belidzs@MikroTik] > /ping 10.0.1.1
  SEQ HOST                                     SIZE TTL TIME       STATUS                                                                                                                                                                                                                                                                             
    0 10.0.1.1                                   56  64 18ms218us 
    1 10.0.1.1                                   56  64 18ms395us 
    2 10.0.1.1                                   56  64 18ms499us 
    3 10.0.1.1                                   56  64 18ms446us 
    sent=4 received=4 packet-loss=0% min-rtt=18ms218us avg-rtt=18ms389us max-rtt=18ms499us

Looking good, now let’s determine the private IP address of the OCI machine:

❯ ip addr
...
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 02:00:17:06:84:60 brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    inet 10.0.0.193/24 metric 100 brd 10.0.0.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::17ff:fe06:8460/64 scope link
       valid_lft forever preferred_lft forever
...

Try pinging it from a machine on your local network:

❯ ping 10.0.0.193
PING 10.0.0.193 (10.0.0.193): 56 data bytes
64 bytes from 10.0.0.193: icmp_seq=0 ttl=59 time=18,893 ms
64 bytes from 10.0.0.193: icmp_seq=1 ttl=59 time=18,823 ms
^C--- 10.0.0.193 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 18,823/18,858/18,893/0,035 ms

It looks like we have successfully achieved full connectivity between the two networks.

After the tunnel has been set up I have added both free machines to my local DNS resolver and installed k3s on them.

Quirks

Unfortunately my ISP only provides me a dynamic IP address which change quite frequently, so I have to change the CPE configuration on a regular basis.

This shouldn’t be too much of an issue, however for some reason it’s not possible to simply change the IP address of a CPE in OCI, instead you have to delete and recreate it. To make things worse, you can only delete a CPE after you have deleted all linked assets, which means after each IP change I have to delete and recreate both the CPE and the Site-to-Site VPN configuration, and log in to my router to apply the new IP and Shared Key settings.

In an attempt to resolve this issue I’ve created a small Go application which monitors my IP for changes and it automatically recreates the whole configuration using the OCI SDK when required. Since this app is not in a production-ready state, I haven’t released it yet, but maybe I’ll do that in the future.

Regardless, if you have your own homelab and want some extra computing power for free, this offer from Oracle might be perfect for you.

oracle oci mikrotik vpn