Audit RHEL/CentOS 6 security benchmarks with ansible

Ansible logoSecuring critical systems isn’t easy and that’s why security benchmarks exist. Many groups and communities distribute recommendations for securing servers, including NIST, the US Department of Defense (DoD), and the Center for Internet Security (CIS).

Although NIST and DoD are catching up quickly with newer OS releases, I’ve found that the CIS benchmarks are updated very regularly. CIS distributes auditing tools (with paid memberships) that require Java and they’re cumbersome to use, especially on servers where Java isn’t normally installed.

A better way to audit security benchmarks

I set out to create an Ansible playbook that would allow users to audit and (carefully!) remediate servers. The result is on GitHub. Before we go any further, I’d just like to state that I’m not affiliated with CIS in any way and this repository hasn’t been endorsed by CIS. Use it at your own risk.

Getting the playbook onto a machine is easy:

git clone

PLEASE review the README and NOTES files in the GitHub repository prior to running the playbook.


Seriously. I mean it. This playbook could knock production environments offline.

The tasks are split into sections (just like the CIS benchmarks themselves) and each section is split into Level 1 and 2 requirements.

Benchmark levels

Level 1 requirements provide good security improvements without a tremendous amount of intrusion into production workloads. With that said, they can still cause issues.

Level 2 requirements provide stronger security improvements but they can adversely affect production server environments. This is where you find things like SELinux, AIDE (including disabling prelinking), and some kernel tweaks for IPv6.

How to use it

I strongly recommend some dry runs with Ansible’s check mode before trying to modify a production system. Also, you can run the playbook against a freshly-installed system and then deploy your applications on top of it. Find out what breaks and disable certain benchmarks that get in the way.

The entire playbook takes less than a minute to run locally on a Rackspace Performance Cloud Server. Your results may vary over remote ssh connections, but I was seeing the playbooks complete over ssh within three to four minutes.

You can also review the variables file to find all the knobs you need to get more aggressive in your audits. If you spot something potentially destructive that needs a variable added, let me know (or submit a pull request).

It’s open source

The entire repository is licensed under Apache License 2.0, so please feel free to submit issues, pull requests, or patches.

Start Jenkins on Fedora 20

Installing Jenkins on Fedora 20 is quite easy thanks to the available Red Hat packages, but I ran into problems when I tried to start Jenkins. Here are the installation steps I followed:

wget -O /etc/yum.repos.d/jenkins.repo
rpm --import
yum -y install jenkins
systemctl enable jenkins
systemctl start jenkins

Your first error will show up if Java isn’t installed. You can fix that by installing Java:

yum -y install java-1.7.0-openjdk-headless

After installing Java, Jenkins still refused to start. Nothing showed up in the command line or via journalctl -xn, so I jumped into the Jenkins log file (found at /var/log/jenkins/jenkins.log):

Aug 13, 2014 2:21:44 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: jetty-8.y.z-SNAPSHOT
Aug 13, 2014 2:21:46 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: NO JSP Support for , did not find org.apache.jasper.servlet.JspServlet

My Java knowledge is relatively limited, so I tossed the JSP error message into Google. A stackoverflow thread was the first result and it talked about a possible misconfiguration with Jetty. I tried their trick of using the OPTIONS environment variable, but that didn’t work.

Then I realized that there wasn’t a Jetty package installed on my server. Ouch. The installation continues:

yum -y install jetty-jsp

Jenkins could now get off the ground and I saw the familiar log messages that I’m more accustomed to seeing:

Aug 13, 2014 2:24:26 PM hudson.WebAppMain$3 run
INFO: Jenkins is fully up and running

Much of these problems could stem from the fact that Jenkins RPM’s are built to suit a wide array of system versions and the dependencies aren’t configured correctly. My hope is that the Jenkins project for Fedora 21 will alleviate some of these problems and give the user a better experience.

httpry 0.1.8 available for RHEL and CentOS 7

Red Hat Enterprise Linux and CentOS 7 users can now install httpry 0.1.8 in EPEL 7 Beta. The new httpry version is also available for RHEL/CentOS 6 and supported Fedora versions (19, 20, 21 branched, and rawhide).

Configuring EPEL on a RHEL/CentOS server is easy. Follow the instructions on EPEL’s site and install the epel-release RPM that matches your OS release version.

If you haven’t used httpry before, check the output on Jason Bittel’s site. It’s a handy way to watch almost any type of HTTP server and see the traffic in an easier to read (and easier to grep) format.

Quickly post gists to GitHub Enterprise and

GitHub Logo
The gist gem from GitHub allows you to quickly post text into a GitHub gist. You can use it with the public site but you can also configure it to work with a GitHub Enterprise installation.

To get started, add two aliases to your ~/.bashrc:

alias gist="gist -c"
alias workgist="GITHUB_URL= gist -c"

The -c will copy the link to the gist to your keyboard whenever you use the gist tool on the command line. Now, go through the login process with each command after sourcing your updated ~/.bashrc:

source ~/.bashrc
gist --login
(follow the prompts to auth and get an oauth token from
workgist --login
(follow the prompts to auth and get an oauth token from GitHub Enterprise)

You’ll now be able to use both aliases quickly from the command line:

cat boring_public_data.txt | gist
cat | workgist blocked by Websense

UPDATE 2014-08-07: Websense emailed me to say that the site has been reviewed and found to be safe. It may take some time for all of their products to receive the updated classification.

Quite a few emails and IRC messages hit my screen today about being blocked by Websense products. The report on Websense’s site claims shows that the site is part of a bot network: The URL analyzed is currently compromised to serve malicious content to visitors.

Here are some screenshots from the report:

icanhazip blocked by websense
icanhazip blocked by websense

I reached out to Websense on Twitter and via their site. In the report I sent to them, I explained how the site works, gave them a link to the FAQ, and directed them to several blog posts from this site about This response from Websense hit my inbox late today:


The site you submitted has been reviewed and determined to pose security risk. At this time, the site is not safe for browsing and is appropriately classified under the following category:

hxxp:// – Bot Networks

Researcher Notes: according to our findings, this site in question is embedded with Dyzap campaign malware.

For additional details related to this threat, please refer to the following source:

The site will resume its content-based categorization, once it has been determined to no longer be a security risk.

For further investigation, please contact the website administrator.

If you have any questions and/or need any additional information, please let us know.

Thank you for your inquiry,

Websense Labs

Here’s what I know:

  • The application that serves up the icanhazip services is not compromised
  • The virtual machine on which the application resides is not compromised
  • The application is returning valid data with no evidence of serving malware

If Websense wishes to claim that the site is being used by malware, I can certainly believe that. However, if they claim the site is serving malicious content or actively participating in attacks in any way, I’ve found no evidence that supports that claim.

I’ll be reaching out to Websense again for additional details and to clear up the report listing on the website. If anyone knows of a way for me to identify this malware traffic and block it from accessing, please let me know. My GPG key is available.