Test Fedora 21 at Rackspace with Ansible

Fedora Infinity Logo - Fedora 21Fedora 21 reached Alpha status last month and will reach beta status at the end of October. There are plenty of new features planned for the final release.

You can test Fedora 21 now using Rackspace’s Cloud Servers. I’ve assembled a small Ansible playbook that will automate the upgrade process from Fedora 20 to 21 and it takes around 7-9 minutes to complete.

The Ansible playbook is on GitHub along with instructions: ansible-rax-fedora21

Xen’s XSA-108 patch and Fedora

Xen LogoXen’s latest vulnerability, XSA-108, has generated a lot of buzz over the last week. Most of the attention has come from the reboot notifications from large cloud providers (including my employer).

The vulnerability allows a user within a guest to potentially read memory from another guest or the hypervisor itself. The window of available memory is small but it could be read many times over — much like how the Heartbleed vulnerability was exploited. In some situations, these actions could cause the guest or the hypervisor to crash.

The fix involves a small patch to the Xen hypervisor kernel. The patch is essentially a one-liner since the write operation was merely a no-op already.

Thanks to the efforts of Michael Young, new packages are in testing for Fedora 19, 20, and 21:

If you’d like to test these packages now, you can install koji and download the RPM’s directly:

yum -y install koji
koji download-build --arch=x86_64 xen-4.2.5-3.fc19  # For Fedora 19
koji download-build --arch=x86_64 xen-4.3.3-3.fc20  # For Fedora 20
koji download-build --arch=x86_64 xen-4.4.1-6.fc21  # For Fedora 21

Use yum or rpm to install the new packages. Some servers may need to install all of the downloaded RPM’s or only a portion of them. All of that depends on which Xen-related packages were installed already.

After testing, please leave karma in Bodhi on the appropriate package page.

Apache’s mod_proxy, mod_ssl, and BitTorrent Sync

Merging railway tracksBitTorrent Sync allows you to keep files synchronized between multiple computers or mobile devices. It’s a handy way to do backups, share files with friends, or automate the movement of data from device to device. It comes with a web frontend, called the Web UI, that allows for connections over HTTP or HTTPS.

Using HTTP across the internet to administer Sync seems totally absurd, so I decided to enable HTTPS. I quickly realized two things:

  • My SSL certificates were now specified in Apache and Sync
  • Sync’s Web UI is relatively slow with SSL enabled (especially over higher latency links)

I really wanted to keep things simple by wedging Sync into my existing Apache configuration using mod_proxy.  That was easier said than done since the Web UI has some hard-coded paths for certain assets, like CSS and Javascript.  After quite a bit of trial end error, this configuration works well:

  ProxyPass /btsync http://127.0.0.1:8888
  ProxyPassReverse /btsync http://127.0.0.1:8888
  ProxyHTMLURLMap http://127.0.0.1:8888 /btsync
  Redirect permanent /gui /btsync/gui

The ProxyPass and ProxyPassReverse lines tell Apache where to proxy the requests and it also tells Apache to make requests on behalf of the browser making the request. The ProxyHTMLURLMap directive tells Apache that any requests to /btsync from a client browser should be translated as a request to the root directory (/) of the Web UI. The last line redirects hard-coded requests to /gui up to /btsync/gui instead.

When your configuration is in place, be sure to run a configuration check (httpd -S) and reload the Apache daemon. If you’d like to access your application at a different URI, just replace /btsync in the example configuration with that URI.

Once all this is done, I’m able to access Sync at https://example.com/btsync and Apache handles all of the backend requests properly. On some distributions, you may find that mod_proxy_html isn’t installed by default. You’ll need to install it if you want to use ProxyHTMLURLMap in your configuration. For Fedora users, just install it via yum:

yum install mod_proxy_html

Photo: Old Vintage Railway by Viktor Hanacek

HOWTO: Time Warner Cable and IPv6

IPv6 world launch logoTime Warner has gradually rolled out IPv6 connectivity to their Road Runner customers over the past couple of years and it started appearing on my home network earlier this year.  I had some issues getting the leases to renew properly after they expired (TWC’s default lease length appears to be seven days) and there were some routing problems that cropped up occasionally.  However, over the past month, things seem to have settled down on TWC’s San Antonio network.

Do you have IPv6 yet?

Before you make any adjustments to your network, I’d recommend connecting your computer directly to the cable modem briefly to see if you can get an IPv6 address via stateless autoconfiguration (SLAAC).  You’ll only get one IPv6 address via SLAAC, but we can get a bigger network block later on (keep reading).  Check your computer’s network status to see if you received an IPv6 address.  If you have one, try accessing ipv6.google.com.  You can always check ipv6.icanhazip.com or ipv6.icanhaztraceroute.com as well.

There’s a chance your computer didn’t get an IPv6 address while directly connected to the cable modem.  Here are some possible solutions:

  • Power off the cable modem for 30 seconds, then plug it back in and see if your computer gets an address
  • Ensure you have one of TWC’s approved modems(Bear in mind that not all of these modems support IPv6.)
  • Verify that your computer has IPv6 enabled. (Instructions for Windows, Mac and Linux are available.)

But I want more addresses

If you were able to get an IPv6 address, it’s now time to allocate a network block for yourself and begin using it!  We will request an allocation via DHCPv6.  Every router is a little different, but the overall concept is the same.  Your router will request an allocation on the network and receive that allocation from Time Warner’s network.  From there, your router will assign that block to an interface (most likely your LAN, more on that in a moment) and begin handing our IPv6 addresses to devices in your home.

By default, TWC hands out /64 allocations regardless of what you request via DHCPv6.  I had some success in late 2013 when I requested a /56 but it appears that allocations of that size aren’t available any longer.  Sure, a /64 allocation is gigantic (bigger than the entire IPv4 address space), but getting a /56 would allow you to assign multiple /64 allocations to different interfaces.  Splitting /64’s into smaller subnets is a bad idea.

Let’s talk security

IPv6 eliminates the need for network address translation (NAT).  This means that by the time you finish this howto, each device in your network with have a publicly accessible internet address.  Also, bear in mind that with almost all network devices, firewall rules and ACL’s that are configured with IPv4 will have no effect on IPv6.  This means that you’ll end up with devices on your network with all of their ports exposed to the internet.

In Linux, be sure to use ip6tables (via firewalld, if applicable).  For other network devices, review their firewall configuration settings to see how you can filter IPv6 traffic.  This is a critical step.  Please don’t skip it.

On my Mikrotik device, I have a separate IPv6 firewall interface that I can configure.  Here is my default ruleset:

/ipv6 firewall filter
/ipv6 firewall filter
add chain=input connection-state=related
add chain=input connection-state=established
add chain=forward connection-state=established
add chain=input in-interface=lanbridge
add chain=forward connection-state=related
add chain=input dst-port=546 protocol=udp
add chain=input protocol=icmpv6
add chain=forward protocol=icmpv6
add chain=forward out-interface=ether1-gateway
add action=drop chain=input
add action=drop chain=forward

The first five rules ensure that only related or established connections can make it to my internal LAN. I allow UDP 546 for DHCPv6 connectivity and I’m allowing all ICMPv6 traffic to the router and internal devices. Finally, I allow all of my devices inside the network to talk to the internet and block the remainder of the unmatched traffic.

Configuring the router

It’s no secret that I’m a big fan of Mikrotik devices and I’ll guide you through the setup of IPv6 on the Mikrotik in this post.  Before starting this step, ensure that your firewall is configured (see previous section).

On the Mikrotik, just add a simple DHCPv6 configuration. I’ll call mine ‘twc':

/ipv6 dhcp-client
add add-default-route=yes interface=ether1-gateway pool-name=twc

After that, you should see an allocation pop up within a few seconds (run ipv6 dhcp-client print):

 #    INTERFACE     STATUS        PREFIX                                      EXPIRES-AFTER
 0    ether1-gat... bound         2605:xxxx:xxxx:xxxx::/64                    6d9h15m45s

Check that a new address pool was allocated by running ipv6 pool print:

 #   NAME      PREFIX                                      PREFIX-LENGTH EXPIRES-AFTER
 0 D twc       2605:xxxx:xxxx:xxxx::/64                               64 6d9h13m33s

You can now assign that address pool to an interface. Be sure to assign the block to your LAN interface. In my case, that’s called lanbridge:

/ipv6 address
add address=2605:xxxx:xxxx:xxxx:: from-pool=twc interface=lanbridge

By default, the Mikrotik device will now begin announcing that network allocation on your internal network. Some of your devices may already be picking up IPv6 addresses via SLAAC! Try accessing the Google or icanhazip IPv6 addresses from earlier in the post.

Checking a Linux machine for IPv6 connectivity is easy. Here’s an example from a Fedora 20 server I have at home:

$ ip -6 addr
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 2605:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx/64 scope global mngtmpaddr dynamic
       valid_lft 2591998sec preferred_lft 604798sec
    inet6 2605:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx/64 scope global deprecated mngtmpaddr dynamic
       valid_lft 1871064sec preferred_lft 0sec

If you only see an address that starts with fe80, that’s your link local address. It’s not an address that can be accessed from the internet.

Troubleshooting

If you run into some problems or your router can’t pull an allocation via DHCPv6, try the troubleshooting steps from the first section of this post.

Getting assistance from Time Warner is a real challenge. Everyone I’ve contacted via phone or Twitter has not been able to help and many of them don’t even know what IPv6 is. I was even told “we have plenty of regular IPv4 addresses left, don’t worry” when I asked for help. Even my unusual methods haven’t worked:

My old SBG6580 that was issued by Time Warner wouldn’t ever do IPv6 reliably. I ended up buying a SB6121 and I was able to get IPv6 connectivity fairly easily. The SB6121 only does 172mb/sec down — I’ll be upgrading it if TWC MAXX shows up in San Antonio.

Asus Maximus VI Gene – Error 55

It’s been quite a while since I built a computer but I decided to give it a try for a new hypervisor/NAS box at home. I picked up an Asus Maximus VI Gene motherboard since it had some good parts installed and it seems to work well with Linux. This was my first time doing water cooling for the CPU and I picked up a Seidon 240M after getting some recommendations.

Rubber hits the road

Once everything was in the box and the power was applied, I was stuck with an error code. There’s a two-digit LCD display on the motherboard that rapidly flips between different codes during boot-up. If it stays on a code for a while and you don’t get any display output, you have a problem. For me, this Asus Q code was 55.

The manual says it means that RAM isn’t installed. I pulled out my four sticks of RAM and reseated all of them. I still got the same error. After reading a bunch of forum posts, I ran through a lot of troubleshooting steps:

  • Reseat the RAM
  • Try one stick of RAM and add more until the error comes back
  • Reseat the CPU cooler (at least three times)
  • Reseat the CPU (at least three times)
  • Upgrade the BIOS
  • Clear the CMOS
  • Curse loudly, drink a beer, and come back

I still had error 55 and wasn’t going anywhere fast. After some further testing, I found that if I left the two RAM slots next to the CPU empty, the system would boot. If I put any RAM in the two left RAM slots (A1 and A2), the system wouldn’t boot. Here’s an excerpt from the manual:

Asus Maximus VI Gene Motherboard CPU/memory layout

CPU is on the left. RAM slots are A1, A2, B1, B2, left to right.

Fine-tuning the Google search

I adjusted my Google terms to include “A1 A2 slots” and found more posts talking about CPU coolers being installed incorrectly. Mine had to be correct — I installed it four times! I decided to try re-installing it one last time.

When I removed the CPU cooler from the CPU, I noticed something strange. There are four standoffs around the CPU that the cooler would attach to with screws. Those standoffs screwed into posts that connected to a bracket on the back of the motherboard.

Asus motherboard cpu cooler standoff

The lower two standoffs are highlighted.

I removed the two standoffs that were closest to the A1/A2 RAM slots and noticed something peculiar. One side of the standoff had a black coating that seemed a bit tacky while the other side of the standoff was bare metal. Three of the standoffs had the black side down (against the board) while one had the black side up. I unscrewed that standoff and found that the bare metal side was wedged firmly onto some connections that run from the CPU to the A1/A2 RAM slots. Could this be the issue?

Eureka

After double-checking all of the CPU cooler standoffs and attaching the cooler to the board, I crossed my fingers and hit the power button. The machine shot through POST and I was staring down a Fedora logo that quickly led to a GDM login.

Badly installed cpu cooler standoff

The culprit

I don’t talk about hardware too often on the blog, but I certainly hopes this helps someone else who is desperately trying to find a solution.