Mikrotik on Major Hayden

Add a VLAN on a Mikrotik router

major@mhtx.net (Major Hayden) — Thu, 20 Apr 2023 00:00:00 +0000

If your house is like mine, you have devices that you really trust and then there are those other devices.

My trusted device group includes my work computers, a Synology NAS, and a few other computers. The bucket of untrusted devices includes Chromecasts, TVs, tablets, phones, and whatever random devices that my kids’ friends bring over.

A VLAN helps with traffic segmentation by isolating certain traffic over the same network cable. A router can manage tons of different networks via the same downlink cable(s) to a switch or other equipment. You can tell a switch to only allow certain VLANs through a port or you can have the port only offer one network that happens to be one of your VLANs.

The best analogy for a VLAN is a cable within a cable. It’s almost like being able to add thousands of individual segmented networks in the same ethernet cable.

VLANs are possible via a networking standard called 802.1Q. Network devices add a small 802.1Q header, often called a VLAN tag, to each ethernet packet. These tags offer a way for network devices to filter traffic on a network.

It works well for devices that don’t understand VLANs, too. For example, if you have a device that isn’t VLAN-aware, you can plug it into a switch port that is configured to offer a VLAN network as the native VLAN. That device happily uses the network it is offered via the switch port without knowing that the switch is adding VLAN tags to all traffic that the device creates.

Let’s get a VLAN working on a Mikrotik router.

Adding a VLAN

Mikrotik devices have a great command line interface and I’ll use that for this post.

In this example, my networks are set up like this:

I have a default LAN network: 192.168.10.0/24
My VLAN network is tagged with tag 15: 192.168.15.0/24

The basic building block of any network on a Mikrotik device is an interface. We start by creating a VLAN interface:

/interface vlan \
add interface=bridge name=vlan15 vlan-id=15

My router uses a bridge called bridge (gotta keep things simple), but you may need to use something like ether2 or ether3 if you’re using a physical network interface instead of a bridge.

Now I can add an IP address to my new network interface:

/ip address \
add address=192.168.15.1/24 interface=vlan15 network=192.168.15.0

DHCP sure does make IP address configuration easier, so let’s create an address pool and a DHCP server instance for our VLAN network. Choose whatever range makes sense for you but my default is usually 10-254:

/ip pool \
add name=vlan15 ranges=192.168.15.10-192.168.15.254

Add a DHCP server and a DHCP network configuration:

/ip dhcp-server \
add address-pool=vlan15 interface=vlan15 name=vlan15
/ip dhcp-server network
add address=192.168.15.0/24 dns-server=192.168.15.1 gateway=192.168.15.1

The DHCP server uses our vlan15 address pool for handing out addresses to devices on the VLAN.

Testing the VLAN

I like to give the VLAN a quick test with my desktop PC before I start messing around with the switch configuration. We just need to add a VLAN device via nmcli and verify that DHCP and routing are working.

Let’s add a new interface called VLAN15 to handle traffic tagged with VLAN 15:

# Replace enp7s0 with your ethernet interface name!
$ nmcli con add type vlan ifname VLAN15 con-name VLAN15 dev enp7s0 id 15
Connection 'VLAN15' (f7cd4cdf-d2ce-4dc7-9ed8-f40102ff3e42) successfully added.

Did we get an IP address and a route?

$ ip addr show dev VLAN15
7: VLAN15@enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
 link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
 inet 192.168.15.52/24 brd 192.168.15.255 scope global dynamic noprefixroute VLAN15
 valid_lft 409sec preferred_lft 409sec
 inet6 fe80::7f62:9d9a:dc8c:3fd7/64 scope link noprefixroute 
 valid_lft forever preferred_lft forever
$ ip route show dev VLAN15
192.168.15.0/24 proto kernel scope link src 192.168.15.52 metric 400

Awesome!

Using it with a switch

I have a Mikrotik CRS that I use for my main home switch and it handles VLAN tagging. My goal here is to trunk VLAN 15 from the router down to the switch so that a particular port ONLY exposes the VLAN 15 network to a device. I don’t want that device to have any idea that VLAN 15 even exists. It should think that there’s a regular old LAN network coming through the switch port.

This is called an access port. Devices on the port have no idea that VLAN tagging is happening on the switch, but the switch tags all traffic coming from the port.

In this example, I set up switch port ether18 to take VLAN 15 and make it available as the native VLAN to anything connected to that switch port:

# Translate VLAN 15 to the native VLAN on ether18
# This is creating the access port
/interface ethernet switch ingress-vlan-translation \
add ports=ether18 customer-vid=0 new-customer-vid=15

# Ensure that traffic tagged with VLAN 15 can exit the switch
# through the uplink to the router
/interface ethernet switch egress-vlan-tag \
add tagged-ports=ether1 vlan-id=15

# Add VLAN table entries to show which ports are members of the VLAN
/interface ethernet switch vlan \
add ports=ether1,ether18 vlan-id=15

# Don't allow anyone on port ether18 to tag their traffic with a
# different VLAN ID and circumvent our access port settings
/interface ethernet switch \
set drop-if-invalid-or-src-port-not-member-of-vlan-on-ports=ether1,ether18

At this point, I can connect a device to port ether18 and it gets an IP address via DHCP on the 192.168.15.1 network automatically!

For further reading on these settings, check out Mikrotik’s wiki page of switch configuration examples.

Extra credit

Once you begin segmenting your network, review your router configuration to see how these networks are allowed to communicate with one another. The default on Mikrotik devices is to allow internal networks to freely communicate with each other since that makes everything easier to get started. However, I don’t want my Chromecast to talk to my NAS.

Mikrotik’s IP firewalling capabilities give you lots of methods for limiting access between networks. Be sure to read up on the IP/Firewall/Filter documentation. If you use IPv6 on your network, be sure to review the IPv6/Firewall/Filter docs, too.

Even if you think that you aren’t using IPv6 internally, you might actually be using it. 😉

Monitor a UPS with a Mikrotik router via SNMP

major@mhtx.net (Major Hayden) — Fri, 28 Oct 2022 00:00:00 +0000

Cyberpower UPS units saved me from plenty of issues in the past with power outages. However, although I love the units themselves, I found that the quality of replacement batteries varies widely. This leads me to keep a close watch on my UPS units and test them regularly.

Energy conservation ranks high on my list of priorities, too. I monitor the power draw on my UPS units to know about usage spikes or to review electricity consumption after I make changes.

My Raspberry Pi did a great job of monitoring my UPS for my network devices but it failed after a recent reboot. My network woes from September left me with a Mikrotik hEXs running my home network and I noticed it had a USB port.

Can you monitor a UPS with a Mikrotik device and query its status remotely? You can!

Initial setup

My Cyberpower CP1500AVRLCD has a USB port on the back for monitoring and control. The hEXs router has a USB-A port on the side that can be used for mass storage, LTE modems, and yes – UPS units.

However, UPS monitoring does not come standard with RouterOS 7.x and it must be installed via a separate package. Follow these steps to get started:

Identify the CPU architecture of your Mikrotik. It should be shown on the product page. The Mikrotik hEXs is a MMIPS (microMIPS) device.
Go to the RouterOS download page and download the Extra packages file for your architecture.
Unpack the zip file you downloaded and locate the ups-7.x-mmips.npk package.
Upload the ups-7.x-mmips.npk file via your preferred method. FTP, ssh, and the web interface work well for this.
Reboot your Mikrotik device.

Enable monitoring

After the reboot, your Mikrotik should now have a /system/ups entry on the command line. Let’s add monitoring for our UPS:

[major@hexs] > /system/ups
[major@hexs] /system/ups> add name=ups min-runtime=never port=usbhid1

If you don’t know what your port is called, type in add port= and press TAB to see the available ports. Refer to the Mikrotik System/UPS manual for more help here.

I set the min-runtime to never which means that the Mikrotik will never hibernate even if the UPS power runs low. It uses so little power and it’s so critical for my home network that it should be the last system to go offline during an outage.

All that’s left is to enable read-only SNMP so that we can monitor the UPS remotely. Back to the Mikrotik command line:

[major@hexs] > /snmp/set enabled=yes

This enables unrestricted read-only SNMP access for your entire network without authentication under the community name public. I restrict SNMP access with firewall rules but you may want to consider further restrictions on your SNMP community.

Getting data

From another machine on the network, I dumped all of the SNMP data from the Mikrotik into a file:

$ snmpwalk -v2c -c public 192.168.10.1 | tee -a /tmp/snmpwalk.txt

Then I looked for my UPS’ model name:

$ grep LCD /tmp/snmpwalk.txt 
SNMPv2-SMI::mib-2.33.1.1.2.0 = STRING: "CP1500AVRLCDa"
SNMPv2-SMI::mib-2.47.1.1.1.1.2.262146 = STRING: "CPS CP1500AVRLCDa"

Let’s see if the first entry gives us the data we need:

> $ grep "^SNMPv2-SMI::mib-2.33" /tmp/snmpwalk.txt 
SNMPv2-SMI::mib-2.33.1.1.2.0 = STRING: "CP1500AVRLCDa"
SNMPv2-SMI::mib-2.33.1.1.3.0 = ""
SNMPv2-SMI::mib-2.33.1.2.1.0 = INTEGER: 2
SNMPv2-SMI::mib-2.33.1.2.3.0 = INTEGER: 103
SNMPv2-SMI::mib-2.33.1.2.4.0 = INTEGER: 100
SNMPv2-SMI::mib-2.33.1.2.5.0 = INTEGER: 0
SNMPv2-SMI::mib-2.33.1.2.7.0 = INTEGER: 0
SNMPv2-SMI::mib-2.33.1.3.2.0 = INTEGER: 1
SNMPv2-SMI::mib-2.33.1.3.3.1.2.3 = INTEGER: 0
SNMPv2-SMI::mib-2.33.1.3.3.1.3.3 = INTEGER: 122
SNMPv2-SMI::mib-2.33.1.4.3.0 = INTEGER: 1
SNMPv2-SMI::mib-2.33.1.4.4.1.2.3 = INTEGER: 122
SNMPv2-SMI::mib-2.33.1.4.4.1.5.3 = INTEGER: 8
SNMPv2-SMI::mib-2.33.1.6.1.0 = Gauge32: 0

What the heck do all these numbers mean? A quick trip to a MIB browser shows us that there are a few important items here:

upsOutputPercentLoad is 1.4.4.1.5 (8%)
upsOutputVoltage is 1.4.4.1.2 (122V)
upsEstimatedChargeRemaining is 1.2.4.0 (100%)

These are the three numbers I care most about. However, the percent load of 8% isn’t terribly useful. I’d rather have watts.

Let’s write a script to get the value, and convert the percentage to watts:

#!/bin/bash
set -euo pipefail

# From the CP1500AVRLCDa spec sheet
MAX_LOAD_WATTS=815

# SNMP MIB for load percentage
SNMP_MIB="SNMPv2-SMI::mib-2.33.1.4.4.1.5.3"

# Get the load integer only.
CURRENT_LOAD=$(snmpget -Oqv -v2c -c public 192.168.10.1 $SNMP_MIB)

# Convert the percentage into wattage consumed right now.
CURRENT_WATTS=$(($MAX_LOAD_WATTS * $CURRENT_LOAD / 100))

echo "${CURRENT_WATTS}"

Let’s test the script!

$ ./get_wattage.sh
65

Awesome! 🎉

PXE boot netboot.xyz on a Mikrotik router

major@mhtx.net (Major Hayden) — Fri, 02 Sep 2022 00:00:00 +0000

The first RFCs for PXE, or preboot execution environment, showed up in June 1981 and it’s still a popular tool today. It enables computers to boot up and download some software that runs early in the boot process.

Although PXE has been with us for ages, it’s still extremely relevant today:

Provisioning: Deploying new operating systems to machines is easily automated with PXE.
Rescue: Fix a broken system by booting into a live OS and then make repairs.
Validation: PXE boot a machine into a validation suite that checks hardware or puts it through a burn-in process.
Ephemeral OS: Boot into a live operating system that runs completely in RAM and disappears on reboot.

A good friend of mine started a project to take PXE booting to the next level.

Enter netboot.xyz

Most of the PXE deployments I’ve used in the past were restricted to a company’s internal network for specfic uses. These deployments often served as a provisioning method.

Although the actual mechanisms for booting machines via PXE are not difficult, writing the backend scripts and creating keyboard-friendly menus is challenging. One of my favorite people on the planet, Ant Messerli, started the netboot.xyz project several years ago.

What does netboot.xyz do for you? For one, you don’t need to write your own menu scripts. All you do is PXE boot and use the menus already available on the site. You can even add your own in netboot.xyz’s GitHub repository

The site relies on ipxe, an open source boot firmware. There’s no need to compile your own ipxe binary. netboot.xyz offers pre-built ipxe binaries that already connect you to netboot.xyz on the first boot!

Here’s what you’ll see from netboot.xyz on your first boot:

PXE ingredients

Now that netboot.xyz did the hard part, what’s left? A PXE environment requires a few items:

DHCP server
TFTP server

The DHCP server normally tells the machine about its IP address, gateway, DNS servers and more. However, we need it to provide two extra pieces of information:

The server running a TFTP daemon
The filename to request

The boot process for a machine on your network will go something like this:

Machine makes a DHCP request
Your DHCP server replies with the usual IP information plus a server IP and filename for the PXE software
Machine sets its IP, gateway, netmask, and DNS
Machine downloads the PXE software from the server provided by the DHCP server
PXE software runs on the machine

Mikrotik PXE configuration

Let’s update the DHCP server configuration first. Log into your Mikrotik router via ssh and add configuration to your DHCP server’s network configuration

[major@hexs] > /ip/dhcp-server/
[major@hexs] /ip/dhcp-server> 
[major@hexs] /ip/dhcp-server/network> print
Columns: ADDRESS, GATEWAY, DNS-SERVER
# ADDRESS GATEWAY DNS-SERVER 
0 192.168.10.0/24 192.168.10.1 192.168.10.1
[major@hexs] /ip/dhcp-server/network> set next-server=192.168.10.1 boot-file-name=pxeboot numbers=0

Great. Now our DHCP server will tell new machines where to find their PXE image. Now we need to get our PXE boot image. Most of my machines support UEFI, so I use the UEFI DHCP image. Upload the image to the mikrotik however you prefer. I normally use FTP or the web interface.

My PXE image is stored on the Mikrotik as /netboot.xyz/netboot.xyz.efi. Now we can set configure the Mikrotik’s built-in FTP server:

[major@hexs] > /ip tftp
add ip-addresses=192.168.10.0/24 real-filename=\
 /netboot.xyz/netboot.xyz.efi req-filename=.*
[major@hexs] > /ip tftp settings
set max-block-size=8192

These settings enable TFTP access for anything on my LAN. Also, my PXE image from netboot.xyz is returned no matter what is in the request.

Now it’s time for a quick test! On Fedora, you can install a tftp client by running dnf install tftp.

❯ tftp 192.168.10.1 -v -m binary -c get pxeboot
mode set to octet
Connected to 192.168.10.1 (192.168.10.1), port 69
getting from 192.168.10.1:pxeboot to pxeboot [octet]
Received 1074688 bytes in 0.6 seconds [14810621 bit/s]

Awesome! Let’s make sure we downloaded everything correctly:

# Check the software downloaded from TFTP
❯ sha256sum pxeboot 
ef4b7d62d360bd8b58a3e83dfa87f8c645d459340554ce4ad66c0ef341fc3653 pxeboot

# Check our original file
❯ sha256sum ~/Downloads/netboot.xyz.efi
ef4b7d62d360bd8b58a3e83dfa87f8c645d459340554ce4ad66c0ef341fc3653 /home/major/Downloads/netboot.xyz.efi

Now your systems on your local network can PXE boot using netboot.xyz! During the boot routine, you may need to press a key (usually F12, F11, F2, or maybe DEL) to bring up a boot selection menu. Pick the PXE or network boot option (choose IPv4 if asked) and boot!

During the boot, your machine will boot the locally downloaded PXE image and it will automatically call out to netboot.xyz for menu selections. Scroll through the menus, choose your image, and enjoy! 🤓

Adventures with GRE and IPSec on Mikrotik routers

major@mhtx.net (Major Hayden) — Wed, 27 May 2015 13:46:28 +0000

I recently picked up a RB850GX2 from my favorite Mikrotik retailer, r0c-n0c. It’s a dual-core PowerPC board with five ethernet ports and some decent performance for the price.

I still have the RB493G in a colocation and I usually connect my home and the colo via OpenVPN or IPSec. Networking is not one of my best skills and I’m always looking to learn more about it when I can. I decided to try out a GRE tunnel on top of IPSec this time around. Combining GRE and IPSec allows you to simplify connectivity between two network segments through an encrypted tunnel.

The Setup

The LAN in my colo and at home is fairly simple: a /24 of RFC1918 space behind a Mikrotik doing NAT. My goal was to get a tunnel up between both environments so that I could reach devices behind my colo firewall from home and vice versa. I do plenty of ssh back and forth along with backups from time to time.

In this example, here’s the current network configuration:

Home: 192.168.50.0/24 on the LAN, 1.1.1.1 as the public IP
Colo: 192.168.150.0/24 on the LAN, 2.2.2.2 as the public IP

I want devices on 192.168.50.0/24 to talk to 192.168.150.0/24 and vice versa. Let’s get the GRE tunnel up first.

GRE

Plain GRE tunnels aren’t encrypted, but I prefer to set them up first to test connectivity prior to adding IPSec into the mix. IPSec can be a challenge to configure the first time around.

I’ll first create a GRE interface at home:

/interface gre
add !keepalive local-address=1.1.1.1 name=home-to-colo remote-address=2.2.2.2

We’ll do the same on the colo router:

/interface gre
add !keepalive local-address=2.2.2.2 name=colo-to-home remote-address=1.1.1.1

You can check to see if the GRE tunnel is running from either router:

/interface gre print

Look for the R in the flags column.

If you’ve made it this far, you now have a GRE tunnel configured but we can’t pass any traffic across it yet. We need to add some IP’s to both sides and configure some routes.

IP’s and Routes

You have some freedom here to choose the IP addresses for both ends of your tunnel but don’t choose anything that interferes with your current LAN IP addresses. In my case, I’ll choose 10.10.10.1/30 and 10.10.10.2/30 for both ends of the tunnel.

I’ll give the 10.10.10.2 address to the home firewall:

/ip address
add address=10.10.10.2/30 interface=home-to-colo network=10.10.10.0

And I’ll give the 10.10.10.1 address to the colo firewall:

/ip address
add address=10.10.10.1/30 interface=colo-to-home network=10.10.10.0

At this point, systems at home can ping 10.10.10.1 (the colo router’s GRE tunnel endpoint) and systems at the colo can ping 10.10.10.2 (the home router’s GRE tunnel endpoint). That’s great because we will use these IP’s to route our LAN traffic across the tunnel.

We need to tell the home router how to get traffic from its LAN over to the colo LAN and vice versa. We can do that with the tunnel endpoints we just configured.

Let’s tell the home router to use the colo router’s GRE tunnel endpoint to reach the colo LAN:

/ip route
add distance=1 dst-address=192.168.150.0/24 gateway=home-to-colo

And tell the colo router to use the home router’s GRE endpoint to reach the home LAN:

/ip route
add distance=1 dst-address=192.168.50.0/24 gateway=colo-to-home

We don’t have to tell the router about the tunnel’s IP address since those routes are generated automatically when we added the IP addresses to each side of the GRE tunnel.

If you’ve made it this far, systems in your home LAN should be able to ping the colo LAN and vice versa. If not, go back and double-check your IP addresses on both sides of the tunnel and your routes.

Adding IPSec

BEFORE YOU GO ANY FURTHER, ensure you have some sort of out-of-band access to both routers. If you make a big mistake like I did (more on that later), you’re going to be glad you set up another way to reach your devices!

We have an GRE tunnel without encryption already and that’s allowing us to pass traffic. That’s fine, but it’s not terribly secure to send our packets in that tunnel across a hostile internet. IPSec will allow us to tell both routers that we want packets between the public IP addresses of both routers to be encrypted. The GRE tunnel will take care of actually delivering the packets, however. IPSec isn’t an interface and it can’t be a conduit for networking all by itself.

Have you configured another way to access both routers yet? Seriously, stop now and do that. I mean it.

If you have native IPv6 access (not a IPv6 over IPv4 tunnel!) into each device, that can be a viable backup plan. Another option might be serial cables or a dedicated console connection. You’ll thank me later.

Configuring IPSec is done in three chunks:

Make a proposal: both routers must agree on how to authenticate each other and encrypt traffic
Configure a peer list: both routers need to know how to reach each other and have some shared secrets
Set a policy: both routers need to agree on which packets must be encrypted

We will start with the proposal. The defaults are good for both routers. Add this configuration on both devices:

/ip ipsec proposal
set [ find default=yes ] auth-algorithms=md5 enc-algorithms=aes-128-cbc,twofish

Now our routers agree on what methods they’ll use to encrypt traffic. Feel free to adjust these algorithms later if needed. Let’s tell each router about its peer.

At home:

/ip ipsec peer
add address=2.2.2.2/32 nat-traversal=no secret=letshavefunwithipsec

At the colo:

/ip ipsec peer
add address=1.1.1.1/32 nat-traversal=no secret=letshavefunwithipsec

Both routers now know about each other and they both have the same shared secret (please use a better shared secret in production). All we have left is configuring a policy.

At this point, ensure you’re accessing both routers via an out-of-band method (native IPv6, console, serial, etc). YOU ARE ABOUT TO LOSE CONNECTIVITY TO YOUR REMOTE DEVICE.

At home, we set up a policy that says all traffic between the public addresses of both firewalls must be encrypted (GRE will carry the traffic for us). Ensure that the CIDR portion of the IP address for dst-address/src-address is present!

/ip ipsec policy
add dst-address=2.2.2.2/32 sa-dst-address=2.2.2.2 sa-src-address=1.1.1.1 src-address=1.1.1.1/32 tunnel=yes

We will do something similar on the colo side. Again, ensure that the CIDR portion of the IP address for dst-address/src-address is present!

/ip ipsec policy
add dst-address=1.1.1.1/32 sa-dst-address=1.1.1.1 sa-src-address=2.2.2.2 src-address=2.2.2.2/32 tunnel=yes

You should now be able to ping across your GRE tunnel but it’s encrypted this time! If you find that one of your devices is inaccessible, don’t panic. Disable the policy you just added (set disabled=yes number=[number of your policy]) and review your configuration.

In the policy step, we told both routers that if traffic moves between the src-address and dst-address, we want it encrypted. Also, the sa-src-address and sa-dst-address gives the router a hint to figure out the identity of the peer and what their shared secret is.

Checking our work

You can check your work with something like this on the home router:

[major@Home] > /ip ipsec remote-peers print
 0 local-address=1.1.1.1 remote-address=2.2.2.2 state=established side=initiator established=7h17m10s

If you have a line like that, your IPSec peers can communicate properly. To test the encryption, you have two options. One option is to put a device outside your firewall and dump traffic via a tap or hub.

Another option (albeit less accurate) is to use the profile tool built into RouterOS. Run the following:

/tool profile

You’ll see some output showing where the majority of your CPU is consumed. Now, transfer some large files between systems behind both routers. You can use iperf for this as well if you really want to stress out the network link. When you do that, you should see encrypting in the profile output as a very large consumer of the CPU. If you only see something like gre or ethernet as your top CPU consumers, you may have missed something on your IPSec policy and your traffic is likely not being encrypted. This isn’t true for all routers — it depends on your normal workloads.

How I made a huge mistake

When I was going through this process, I made it through the GRE portion without a hitch. Everything worked well. Once I added IPSec to the mix, I used the GRE tunnel endpoints (10.10.10.1 and 10.10.10.2) as my src-address and dst-address in my IPSec policy. Nothing was getting encrypted and I was getting really frustrated.

I kept reading tutorials on various sites and came to realize that I didn’t need an encryption policy between the tunnel endpoints, I needed a policy between the actual public addresses of the routers. I wasn’t aware that the GRE tunnel would happily keep working between the two public IP addresses even with the IPSec policy in place between the IP addresses.

First mistake: I didn’t access my colo router via an out-of-band path. Second mistake: I applied my IPSec policy on the home router first and was shocked that I lost connectivity to the colo router. That was a quick fix — I just disabled the IPSec policy on the home router and I could access the colo router again.

Just after adjusting the IPSec policy on the colo router to use the public IP addresses, I noticed that connectivity dropped. At this point, I expected that — I set up a policy there but I hadn’t done it on the home router yet. I enabled the policy on the home router and then started pinging. Nothing.

Then came the Pingdom and UptimeRobot alerts for my sites in the colo. Oh crap.

Once I was able to reach the colo router via IPv6 through some other VM’s, I realized what happened. I left the CIDR mask off the src-address and dst-address in the IPSec policy.

Guess what RouterOS chose as a CIDR mask for me? /0. Ouch.

I quickly adjusted those to be /32’s. Within seconds, everything was up again and the GRE tunnel began working. As the Pingdom alerts cleared and my heart rate returned to normal, I figured the best thing I should do is share my story so that others don’t make the same mistake. ;)

HOWTO: Mikrotik OpenVPN server

major@mhtx.net (Major Hayden) — Fri, 01 May 2015 15:33:35 +0000

Mikrotik firewalls have been good to me over the years and they work well for multiple purposes. Creating an OpenVPN server on the device can allow you to connect into your local network when you’re on the road or protect your traffic when you’re using untrusted networks.

Although Miktrotik’s implementation isn’t terribly robust (TCP only, client cert auth is wonky), it works quite well for most users. I’ll walk you through the process from importing certificates through testing it out with a client.

Import certificates

Creating a CA and signing a certificate and key is outside the scope of this post and there are plenty of sites that cover the basics of creating a self-signed certificate. You could also create a certificate signing request (CSR) on the Mikrotik and have that signed by a trusted CA. In my case, I have a simple CA already and I signed a certificate for myself.

Upload your certificate, key, and CA certificate (if applicable) to the Mikrotik. After that, import those files into the Mikrotik’s certificate storage:

 import file-name=firewall.example.com.crt
passphrase:
 certificates-imported: 1
 private-keys-imported: 0
 files-imported: 1
 decryption-failures: 0
 keys-with-no-certificate: 0

[major@home] /certificate> import file-name=firewall.example.com.pem
passphrase:
 certificates-imported: 0
 private-keys-imported: 1
 files-imported: 1
 decryption-failures: 0
 keys-with-no-certificate: 0

[major@home] /certificate> import file-name=My_Personal_CA.crt
passphrase:
 certificates-imported: 1
 private-keys-imported: 0
 files-imported: 1
 decryption-failures: 0
 keys-with-no-certificate: 0

Always import the certificate first, then the key. You should be able to do a /certificate print and see the entries for the files you imported. In the print output, look at the flags column and verify that the line with your certificate has a T and a K. If the K is missing, import the key one more time. If that still doesn’t work, ensure that your certificate and key match.

The default naming conventions used for certificates is a little confusing. You can rename a certificate by running set name=firewall.example.com number=0 (run a /certificate print to get the right number).

OpenVPN server configuration

We’re now ready to do the first steps of the OpenVPN setup on the Mikrotik. You can do this configuration via the Winbox GUI or via the web interface, but I prefer to use the command line. Let’s start:

/interface ovpn-server server
set certificate=firewall.example.com cipher=blowfish128,aes128,aes192,aes256 default-profile=default-encryption enabled=yes

This tells the device that we want to use the certificate we imported earlier along with all of the available ciphers. We’re also selecting the default-encryption profile that we will configure in more detail later. Feel free to adjust your cipher list later on but I recommend allowing all of them until you’re sure that the VPN configuration works.

We’re now ready to add an OpenVPN interface. In Mikrotik terms, you can have multiple OpenVPN server profiles running under the same server. They will all share the same certificate, but each may have different authentication methods or network configurations. Let’s define our first profile:

/interface ovpn-server
add name=openvpn-inbound user=openvpn

There’s now a profile with a username of openvpn. That will be the username that we use to connect to this VPN server.

Secrets

The router needs a way to identify the user we just created. We can define a secret easily:

/ppp secret
add name=openvpn password=vpnsarefun profile=default-encryption

We’ve set a password secret and defined a connection profile that corresponds to the secret.

Profiles

We’ve been referring to this default-encryption profile several times and now it’s time to configure it. This is one of the things I prefer to configure using the Winbox GUI or the web interface since there are plenty of options to review.

The most important part is how you connect the VPN connection into your internal network. You have a few options here. You can configure an IP address that will always be assigned to this connection no matter what. There are upsides and downsides with that choice. You’ll always get the same IP on the inside network but you won’t be able to connect to the same profile with multiple clients.

I prefer to set the bridge option to my internal network bridge (which I call lanbridge). That allows me to use my existing bridge configuration and filtering rules on my OpenVPN tunnels. My configuration looks something like this:

/ppp profile
set 1 bridge=lanbridge local-address=default-dhcp only-one=no remote-address=default-dhcp

I’ve told the router that I want VPN connections to be hooked up to my main bridge and it should get local and remote IP addresses from my default DHCP server. In addition, I’ve also allowed more than one simultaneous connection to this profile.

The other defaults are fairly decent to get started. You can go back and adjust them later if needed.

OpenVPN client

Every client has things configured a bit differently but I’ll be working with a basic OpenVPN configuration file here that should work on most systems (or at least show you what to click in your client GUI).

Here’s my OpenVPN client configuration file:

remote firewall.example.com 1194 tcp-client
persist-key
auth-user-pass /etc/openvpn/firewall-creds.txt
tls-client
pull
ca /home/major/.cert/ca.crt
redirect-gateway def1
dev tun
persist-tun
cert /home/major/.cert/cert.crt
nobind
key /home/major/.cert/key.key

In my configuration, I refer to a /etc/openvpn/firewall-creds.txt file to hold my credentials. You can store the file anywhere (or this might be configurable in a GUI) but it should look like this:

username
password

That’s it - just a two line file with the username, a line feed, and a password.

At this point, you should be able to test your client.

Troubleshooting

Firewall - Ensure that you have a firewall rule set to allow traffic into your OpenVPN port. This could be something as simple as:

/ip firewall filter add chain=input dst-port=1194 protocol=tcp

Certificates - Check that your certificate and key were imported properly and that your client is configured to trust the self-signed certificate or the CA you used.

Compression - For some reason, I have lots of problems if compression is enabled on the client. They range from connection failures to being unable to pass traffic through the tunnel after getting connected. Be sure that anything that mentions compression or LZO is disabled.

Security

There are some security improvements that can be made after configuring everything:

Limit access to your OpenVPN port in your firewall to certain source IP’s
Configure better passwords for your OpenVPN secret
Consider making a separate bridge or network segment for VPN users when they connect and apply filters to it
Adjust the list of ciphers in the default-encryption profile so that only the strongest can be used (may cause some clients to be unable to connect)

HOWTO: Time Warner Cable and IPv6

major@mhtx.net (Major Hayden) — Thu, 11 Sep 2014 14:43:03 +0000

Time Warner has gradually rolled out IPv6 connectivity to their Road Runner customers over the past couple of years and it started appearing on my home network earlier this year. I had some issues getting the leases to renew properly after they expired (TWC’s default lease length appears to be seven days) and there were some routing problems that cropped up occasionally. However, over the past month, things seem to have settled down on TWC’s San Antonio network.

Do you have IPv6 yet?

Before you make any adjustments to your network, I’d recommend connecting your computer directly to the cable modem briefly to see if you can get an IPv6 address via stateless autoconfiguration (SLAAC). You’ll only get one IPv6 address via SLAAC, but we can get a bigger network block later on (keep reading). Check your computer’s network status to see if you received an IPv6 address. If you have one, try accessing ipv6.google.com. You can always check ipv6.icanhazip.com or ipv6.icanhaztraceroute.com as well.

There’s a chance your computer didn’t get an IPv6 address while directly connected to the cable modem. Here are some possible solutions:

Power off the cable modem for 30 seconds, then plug it back in and see if your computer gets an address
Ensure you have one of TWC’s approved modems. (Bear in mind that not all of these modems support IPv6.)
Verify that your computer has IPv6 enabled. (Instructions for Windows, Mac and Linux are available.)

But I want more addresses

If you were able to get an IPv6 address, it’s now time to allocate a network block for yourself and begin using it! We will request an allocation via DHCPv6. Every router is a little different, but the overall concept is the same. Your router will request an allocation on the network and receive that allocation from Time Warner’s network. From there, your router will assign that block to an interface (most likely your LAN, more on that in a moment) and begin handing our IPv6 addresses to devices in your home.

By default, TWC hands out /64 allocations regardless of what you request via DHCPv6. I had some success in late 2013 when I requested a /56 but it appears that allocations of that size aren’t available any longer. Sure, a /64 allocation is gigantic (bigger than the entire IPv4 address space), but getting a /56 would allow you to assign multiple /64 allocations to different interfaces. See the last section of this post on how to get a /56 allocation. Splitting /64’s into smaller subnets is a bad idea.

Let’s talk security

IPv6 eliminates the need for network address translation (NAT). This means that by the time you finish this howto, each device in your network with have a publicly accessible internet address. Also, bear in mind that with almost all network devices, firewall rules and ACL’s that are configured with IPv4 will have no effect on IPv6. This means that you’ll end up with devices on your network with all of their ports exposed to the internet.

In Linux, be sure to use ip6tables (via firewalld, if applicable). For other network devices, review their firewall configuration settings to see how you can filter IPv6 traffic. This is a critical step. Please don’t skip it.

On my Mikrotik device, I have a separate IPv6 firewall interface that I can configure. Here is my default ruleset:

/ipv6 firewall filter
/ipv6 firewall filter
add chain=input connection-state=related
add chain=input connection-state=established
add chain=forward connection-state=established
add chain=input in-interface=lanbridge
add chain=forward connection-state=related
add chain=input dst-port=546 protocol=udp
add chain=input protocol=icmpv6
add chain=forward protocol=icmpv6
add chain=forward out-interface=ether1-gateway
add action=drop chain=input
add action=drop chain=forward

The first five rules ensure that only related or established connections can make it to my internal LAN. I allow UDP 546 for DHCPv6 connectivity and I’m allowing all ICMPv6 traffic to the router and internal devices. Finally, I allow all of my devices inside the network to talk to the internet and block the remainder of the unmatched traffic.

Configuring the router

It’s no secret that I’m a big fan of Mikrotik devices and I’ll guide you through the setup of IPv6 on the Mikrotik in this post. Before starting this step, ensure that your firewall is configured (see previous section).

On the Mikrotik, just add a simple DHCPv6 configuration. I’ll call mine ’twc':

/ipv6 dhcp-client
add add-default-route=yes interface=ether1-gateway pool-name=twc

After that, you should see an allocation pop up within a few seconds (run ipv6 dhcp-client print):

# INTERFACE STATUS PREFIX EXPIRES-AFTER
0 ether1-gat... bound 2605:xxxx:xxxx:xxxx::/64 6d9h15m45s

Check that a new address pool was allocated by running ipv6 pool print:

# NAME PREFIX PREFIX-LENGTH EXPIRES-AFTER
0 D twc 2605:xxxx:xxxx:xxxx::/64 64 6d9h13m33s

You can now assign that address pool to an interface. Be sure to assign the block to your LAN interface. In my case, that’s called lanbridge:

/ipv6 address
add address=2605:xxxx:xxxx:xxxx:: from-pool=twc interface=lanbridge

By default, the Mikrotik device will now begin announcing that network allocation on your internal network. Some of your devices may already be picking up IPv6 addresses via SLAAC! Try accessing the Google or icanhazip IPv6 addresses from earlier in the post.

Checking a Linux machine for IPv6 connectivity is easy. Here’s an example from a Fedora 20 server I have at home:

$ ip -6 addr
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
 inet6 2605:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx/64 scope global mngtmpaddr dynamic
 valid_lft 2591998sec preferred_lft 604798sec
 inet6 2605:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx/64 scope global deprecated mngtmpaddr dynamic
 valid_lft 1871064sec preferred_lft 0sec

If you only see an address that starts with fe80, that’s your link local address. It’s not an address that can be accessed from the internet.

Troubleshooting

If you run into some problems or your router can’t pull an allocation via DHCPv6, try the troubleshooting steps from the first section of this post.

Getting assistance from Time Warner is a real challenge. Everyone I’ve contacted via phone or Twitter has not been able to help and many of them don’t even know what IPv6 is. I was even told “we have plenty of regular IPv4 addresses left, don’t worry” when I asked for help. Even my unusual methods haven’t worked:

@TWC_Help I'll buy one of your engineers a six pack of beer if they can enable IPv6 for my internet connection. ;)

— Major Hayden (@majorhayden) August 9, 2014

My old SBG6580 that was issued by Time Warner wouldn’t ever do IPv6 reliably. I ended up buying a SB6121 and I was able to get IPv6 connectivity fairly easily. The SB6121 only does 172mb/sec down - I’ll be upgrading it if TWC MAXX shows up in San Antonio.

Get a /56

You can get a /56 block of IP addresses from Time Warner by adding prefix-hint=::/56 onto your IPv6 dhcp client configuration. You’ll need to carve out some /64 subnets on your own for your internal network and that’s outside the scope of this post. The prefix hint configuration isn’t available in the graphical interface or on the web (at the time of this post’s writing).

Native IPv6 connectivity in Mikrotik’s RouterOS

major@mhtx.net (Major Hayden) — Wed, 11 Jan 2012 13:30:07 +0000

It’s no secret that I’m a big fan of the Routerboard devices and the RouterOS software from Mikrotik that runs on them. The hardware is solid, the software is stable and feature-rich, and I found a great vendor that ships quickly.

I recently added a RB493G (~ $230 USD) to sit in front of a pair of colocated servers. The majority of the setup routine was the same as with my previous devices except for the IPv6 configuration.

In the past, I’ve set up IPv6 tunnels with Hurricane Electric and it’s been mostly a cut-and-paste operation from the sample configuration in their IPv6 tunnel portal. Setting up native IPv6 involved a little more legwork.

If your provider will give you two /64’s or an entire /48, getting IPv6 connectivity for your WAN/LAN interfaces is simple. However, if you can only get one /64, you’ll have to see if your provider can route it to you via your Mikrotik’s link local interface (I wouldn’t recommend this for many reasons).

I split my Mikrotik into two interfaces: wan and lanbridge. The lanbridge bridge joins all of the LAN ethernet ports (ether2-9 on the RB493G) and the wan interface connects to the upstream switch.

My configuration:

/ipv6 address
add address=2001:DB8:0:1::2/64 advertise=yes disabled=no eui-64=no interface=wan
add address=2001:DB8:0:2::1/64 advertise=yes disabled=no eui-64=no interface=lanbridge
/ipv6 route
add disabled=no distance=1 dst-address=::/0 gateway=2001:DB8:0:1::1 scope=30 \
 target-scope=10
/ipv6 nd
add advertise-dns=no advertise-mac-address=yes disabled=no hop-limit=64 \
 interface=all managed-address-configuration=no mtu=unspecified \
 other-configuration=no ra-delay=3s ra-interval=3m20s-10m ra-lifetime=30m \
 reachable-time=unspecified retransmit-interval=unspecified
/ipv6 nd prefix default
set autonomous=yes preferred-lifetime=1w valid-lifetime=4w2d

Explanation:

/ipv6 address
add address=2001:DB8:0:1::2/64 advertise=yes disabled=no eui-64=no interface=wan
add address=2001:DB8:0:2::1/64 advertise=yes disabled=no eui-64=no interface=lanbridge

These two lines configure the IPv6 addresses for the firewall’s interfaces. My provider’s router holds the 2001:DB8:0:1::1/64 address and routes the remainder of that /64 to me via 2001:DB8:0:1::2/64. The second /64 is on the lanbridge interface and my LAN devices take their IP addresses from that block. My provider routes that second /64 to me via the 2001:DB8:0:1::2/64 IP on my wan interface.

/ipv6 route
add disabled=no distance=1 dst-address=::/0 gateway=2001:DB8:0:1::1 scope=30 \
 target-scope=10

I’ve set a gateway for IPv6 traffic so that the Mikrotik knows where to send internet-bound IPv6 traffic (in this case, to my ISP’s core router).

/ipv6 nd
add advertise-dns=no advertise-mac-address=yes disabled=no hop-limit=64 \
 interface=lanbridge managed-address-configuration=no mtu=unspecified \
 other-configuration=no ra-delay=3s ra-interval=3m20s-10m ra-lifetime=30m \
 reachable-time=unspecified retransmit-interval=unspecified
/ipv6 nd prefix default
set autonomous=yes preferred-lifetime=1w valid-lifetime=4w2d

These last two lines configure the neighbor discovery on my lanbridge interface. This allows my LAN devices to do stateless autoconfiguration (which gives them an IPv6 address as well as the gateway).

Want to read up on IPv6?

Measure traffic flows with Mikrotik’s RouterOS and ntop on Fedora 15

major@mhtx.net (Major Hayden) — Sun, 05 Jun 2011 14:58:26 +0000

It’s no secret that I’m a big fan of the RouterBoard network devices paired with Mikrotik’s RouterOS. I discovered today that these devices offer Cisco NetFlow-compatible statistics gathering which can be directed to a Linux box running ntop. Mikrotik calls it “traffic flow” and it’s much more efficient than setting up a mirrored or spanned port and then using ntop to dump traffic on that interface.

These instructions are for Fedora 15, but they should be pretty similar on most other Linux distributions. Install ntop first:

yum -y install ntop

Adjust /etc/ntop.conf so that ntop listens on something other than localhost:

# limit ntop to listening on a specific interface and port
--http-server 0.0.0.0:3000 --https-server 0.0.0.0:3001

I had to comment out the sched_yield() option to get ntop to start:

# Under certain circumstances, the sched_yield() function causes the ntop web
# server to lock up. It shouldn't happen, but it does. This option causes
# ntop to skip those calls, at a tiny performance penalty.
# --disable-schedyield

Set an admin password for ntop:

ntop --set-admin-password

Once you set the password, you may need to press CTRL-C to get back to a prompt in some ntop versions.

Start ntop:

/etc/init.d/ntop start

Open a web browser and open http://example.com:3000 to access the ntop interface. Roll your mouse over the Plugins menu, then NetFlow, and then click Activate. Roll your mouse over the Plugins menu again, then NetFlow, and then click Configure. Click Add NetFlow Device and fill in the following:

Type “Mikrotik” in the NetFlow Device section and click Set Interface Name.
Type 2055 in the Local Collector UDP Port section and click Set Port.
Type in your router’s IP/netmask in the Virtual NetFlow Interface Network Address section and click Set Interface Address.

Enabling traffic flow on the Mikrotik can be done with just two configuration lines:

/ip traffic-flow
set enabled=yes interfaces=all
/ip traffic-flow target
add address=192.168.10.65:2055 disabled=no version=5

Wait about a minute and then try reviewing some of the data in the ntop interface. Depending on the amount of traffic on your network, you might see data in as little as 10-15 seconds.