Writing on Major Hayden

How I write blog posts

major@mhtx.net (Major Hayden) — Wed, 17 Aug 2022 00:00:00 +0000

During a mentoring meeting today at work, my mentee asked me how I make time to write blog posts. I hadn’t really thought about it before, so I joked that I needed to write a blog post on that. That’s so meta.

After thinking about it more, a blog post felt like a good idea. Let’s get right to it.

Why write blog posts anyway?

I’ve written about writing¹ before, especially about why technical people should write more often. Writing about things you know, things you love, and things you want other people to know has plenty of benefits:

It helps you structure your own thoughts about a topic
Talking about a topic makes the knowledge more solid in your brain
You can link posts to people who want to know more about the topic
Eventually people come along, read your posts, and they let you know about it

If there’s one thing that you should take away from this post, it’s this: Write for you – not for anyone else.

Here’s what I mean by that. Write because it benefits you. Write because it makes you happy. Write because it helps you organize your thoughts. Write because you want to leave a small mark on the world when you’re gone.²

Sometimes I write a post and think “Nobody will ever read this.” I write it anyway.

However, there are those times where you write something and a reader gains something from it. A tiny portion of those readers will send you something about the post. I use this as my fuel to keep going and it serves as a reminder that I’m doing something for myself that other people enjoy.

Structure is everything

I try to follow a formula for most of my posts and it helps me organize my thoughts efficiently. Here’s what I do:

Introduction: Start with something brief that helps people determine whether they want to read the post or not. The last thing I want to do is waste someone’s time. If the topic applies to them or seems interesting, great! If not, they get some time back and they can skip the post.
Why: A reader that was hooked on the introduction might be interested in the topic but unsure why it’s needed. Explain how you arrived at your decision point. This could uncover a use case the reader never considered, or it might highlight a blind spot in their thought process.
What: Explain how to do something! Walk through each step and take a pause to explain why each step is needed. Highlight optional steps or areas where extra attention and consideration might be required. Diagrams, command line output, and screenshots deliver tons of value here, so don’t hold back.
Extras: You might carry a topic to a certain point, but a reader might want to go farther. Provide links to documentation or point to areas where a reader might want to keep exploring to do more on the topic. Avoid leaving the reader with a dead end.

Eye of the beholder

You may pass up a blog post opportunity because you think everyone knows enough on the topic already. You might do the same if a topic seems to complex.

My advice: write it anyway.

I’ve written incredibly detailed posts about kerberos setups and silly simple posts about deleting a single iptables rule. They both get decent web traffic. They were both fun to write.

Learn from other writers

I read a lot of blogs regularly. Here are some I bloggers I really enjoy reading:

When I read posts, I think about the topic in the post itself, but I also think about how the author writes. I pick out one or two things I really enjoy from their writing and I begin adding those things to my writing habits.

Some of these habits are complex. Others aren’t.

For example, Derek Sivers’ Writing One Sentence Per Line post was incredibly helpful and simple. I’ve implemented it in my writing and it makes a huge difference.

(Try writing one sentence per line. Trust me. It works.)

Over time, just like everything else, you will find ways to make your writing better while writing more efficiently. Avoid getting discouraged and always keep it fun. Write for you.

Sorry for going meta again. ↩︎
I turned 40 today, so please forgive me for thinking about death. 🎂 😂 ↩︎

Raise the bar with an SBAR

major@mhtx.net (Major Hayden) — Tue, 02 Aug 2022 00:00:00 +0000

You discovered a problem at work. If left unchecked, the problem could affect customers and impact revenue. You cannot ignore it.

What now? Who do you tell? Will they listen? Better yet, will they understand?

I find myself in situations like these constantly. My roles over the years involved problems that demanded discussion, thought, and solutions. Some problems were simple but others required complicated fixes that took months or years.

Communicating with other people about complex problems remains a challenge for me, but I learned a new tool that helps me kick off these discussions and share my recommended solutions as efficiently as possible. The SBAR technique shows up frequently in medical settings but it also works well in IT.

This post covers the nuts and bolts of the SBAR format, how to use it to your advantage, and how to communicate clearly with it.

Components

Every SBAR contains four components. Let’s go through each one.

Situation

Hook the reader with an explanation of the events happening right now. Save the backstory for later and focus on the shortest possible list of current events.

Let’s use a favorite example that every system administrator can relate to: a server is down. Your situation section might look something like this:

Situation

Web01 stopped responding at 3PM today and disrupted web traffic to our website at example.com. We cannot process orders online and customers cannot browse our list of products. The marketing team must update the website by the end of the day with information about next week’s trade show.

The situation section includes several critical pieces of information:

What is happening right now?
Who is affected by the problem right now?
How does the problem affect people right now?

Do you see a pattern? Do you see it right now? This section demands short sentences, active language (more on that later), and a focus on right now. Your efforts here charge up the reader to continue into the next section which is often the longest.

Background

Now that you won your reader’s attention, start giving some backstory. Let’s continue our example above about our unresponsive server:

Background

Our datacenter technicians replaced the power supply on web01 last week following a power surge. They found a broken case fan after replacing the power supply and they replaced that as well. Fred called the server vendor about the failures and they warned us that we may have a damaged motherboard following the power surge. Jennifer ordered a replacement motherboard which arrives this Friday.

The server crashed abruptly and rebooted two more times since the maintenance. Normally the server rebooted without any problems, but it did not come back online after today’s crash.

This section adds color and detail about the events that led up to the current situation. It answers that question that executives enjoy asking: “How did we get here?”

Avoid two main pitfalls here:

Stick to facts, not opinions or your intepretation. (That’s the next section!)
Keep the facts pertinent and relevant. If the fact doesn’t help directly explain the current situation at hand, leave it out. Toss extra details into an appendix if needed.

The reader now understands the events happening right now and they know how we got here. Now your experience and expertise comes into play as you bring all of the data together.

Assessment

This remains the toughest section for me with any SBAR. You must thread the needle of tying all the facts together without making a recommendation to the reader.

Imagine the last action movie you saw where the villain has the ability to do something terrible. After the main character explains what the villian has, how dangerous it is, and how vulnerable the good guys are, everyone looks at them and asks: “Okay, give it to me straight: How bad is this?”

Going back to our server down example:

Assessment

We have very low confidence that web01 can host our website reliably at this critical time for our company. The power surge likely damaged the motherboard, but the replacement does not arrive for three more days. While we hoped to have web02 ready for production this week, it still needs more work.

Take all the facts you have and share your thoughts on where they stand. Readers without your experience or level of familiarity with the situation appreciate this section since it ties the first two sections together. Clearly state the root cause of the current situation, the severity, and the impact if left unresolved. Your goal here is to get the reader to ask the right question for the next section: “What do we do now?”

Recommendation

Now it’s time to plot a course forward out of the situation and back to happier times. Make clear recommendations that include a what, why, and how. Assume that every recommendation must survive cross-examination from other engineers and executives.

Our recommendations for fixing our server could include:

Recommendation

Deploy a temporary web server at our public cloud provider to bridge the gap while web01 is awaiting a hardware replacement. Our deployment scripts already work in cloud instances and the our cost for the temporary instance is within our IT budget.

Finish provisioning web02 and configure high availability services so that it can run in active-active mode with web01 as soon as the replacement motherboard arrives.

Repair web01 on Friday and run burn-in tests during the week of the trade show to ensure it operates reliably under load.

Schedule a maintenance after the trade show to tie web01 and web02 together as an H/A pair and move the website traffic back from the cloud provider.

Order a spare parts package from the server vendor to have on hand for future issues with either server.

Be specific and detailed here. Keep the following things in mind:

Think about potential objections (time, cost, etc) and address those in the recommendation itself.
Consider work items beyond the current situation to prevent it from happening again later.
If you have a team working on it, consider multiple simultaneous workstreams and what you could accomplish with each.

Your reader wants to know what to do, so don’t be bashful here. You are an expert and you know the situation well. Tell the reader what must be done, why it must be done, and how to do it (at a high level).

Extra credit

Now that you know all the sections of an SBAR, you’re ready to write one! I learned plenty of tips along the way¹ and these should help you improve your SBAR skills.

Use active language

The SBAR format provides an efficient method for communicating complex problems and recommendations with other people. Using active language keeps the readers attention and allows you to speak more directly and forcefully.

Take the following example:

The server is down because a cart was rolling down the hallway and it hit the server after a datacenter technician forgot to set the brake.

Ugh. That’s a mouthful. How about this instead:

A datacenter technician left a cart unattended on the ramp without setting the brake. The cart rolled down and knocked the server off the rack, taking it offline.

Active language keeps the reader’s attention by keeping the subject followed by a verb. There’s no question of who did what and how that caused a problem. Remove any passive voice that you find and look for sentences without a subject followed by a verb.

Use an appendix

Sometimes an assessment leads to multiple options for a solution. Add these potential solutions to an appendix and allow readers to review those if they need more detail. This reduces clutter in the main part of the SBAR but allows detail-oriented readers to pull out more context without bothering you.

Appendixes are great places for charts, diagrams, and command line output that reinforce your recommendations.

Keep it collaborative

As you write the SBAR, invite others to collaborate with you. My company uses Google Docs heavily and it allows me to bring in more contributors to improve various parts of the document.

This also helps when you’re ready to present your work. Readers can comment and ask questions right in the document. You can add extra details or answer the questions right in the SBAR document.

Wrapping up

The SBAR process offers you a great opportunity to improve your communication skills as an engineer, especially when you communicate with less technical people. You can learn what motivates different people within your organization and tailor your communication so that it matches the things they care about.

These soft skills can also take your career to the next level. Highly technical people who communicate well about complex topics are always in demand.

I learned tips from making mistakes. 🤭 ↩︎

Reprint: Stop Disabling SELinux!

major@mhtx.net (Major Hayden) — Fri, 19 Apr 2013 05:52:23 +0000

This article appeared in SC Magazine and I’ve posted it here as well. For those of you who were left wanting more from my previous SELinux post, this should help. If it doesn’t help, leave a comment. ;)

The push to cloud transforms the way we apply information security principles to systems and applications. Perimeters of the past, secured heavily with traditional network devices in the outermost ring, lose effectiveness day by day. Shifting the focus to “defense in depth” brings the perimeter down to the individual cloud instances running your application. Security-Enhanced Linux, or SELinux, forms an effective part of that perimeter.

SELinux operates in the realm of mandatory access control, or MAC. The design of MAC involves placing constraints on what a user (a subject) can do to a particular object (a target) on the system. In contrast, discretionary access control, or DAC, allows a user with certain access to use discretion to limit or allow access to certain files, directories, or devices. You can set any file system permissions that you want but SELinux can override them with ease at the operating system level.

Consider a typical server running a web application. An attacker compromises the web application and executes malicious code via the web server daemon itself. SELinux has default policies that prevent the daemon from initiating communication on the network. That limits the attacker’s options to attack other services or servers.

In addition, SELinux sets policies on which files and directories the web server can access, regardless of any file system permissions. This protection limits the attacker’s access to other sensitive parts of the file system even if the administrator set the files to be readable to the world.

This is where SELinux shines. Oddly enough, this is the point where many system administrators actually disable SELinux on their systems.

Troubleshooting these events, called AVC denials, without some helpful tools is challenging and frustrating. Each denial flows into to your audit log as a cryptic message. Most administrators will check the usual suspects, like firewall rules and file system permissions. As frustration builds, they disable SELinux and notice that their application begins working as expected. SELinux remains disabled and hundreds of helpful policies lie dormant solely because one policy caused a problem.

Disabling SELinux without investigation frustrated me to the point where I started a site at stopdisablingselinux.com. The site is a snarky response to Linux administrators who reach for the disable switch as soon as SELinux gets in their way.

All jokes aside, here are some helpful tips to use SELinux effectively:

Use the setroubleshoot helpers to understand denials

Working through denials is easy with the setroubleshoot-server package. When a denial occurs, you still receive a cryptic log message in your audit logs. However, you also receive a message via syslog that is very easy to read. Your server can email you these messages as well. The message contains guidance about adjusting SELinux booleans, setting contexts, or generating new SELinux policies to work around a really unusual problem. When I say guidance, I mean that the tools give you commands to copy and paste to adjust your policies, booleans and contexts.

Review SELinux booleans for quick adjustments

Although the myriad of SELinux user-space tools isn’t within the scope of this article, getsebool and togglesebool deserve a mention. Frequently adjusted policies are controlled by booleans that are toggled on and off with togglesebool. Start with getsebool –a for a full list of booleans and then use togglesebool to enable or disable the policy.

Quickly restore file or directory contexts

Shuffling files or directories around a server can cause SELinux denials due to contexts not matching their original values. This happens to me frequently if I move a configuration file from one system to another. Correcting the context problem involves one of two simple commands. The restorecon command applies the default contexts specific to the file or directory. If you have a file in the directory with the correct context, use chcon to fix the context on the wrong file by giving it the path to the file with the correct context.

Here are some additional links with helpful SELinux documentation:

Why technical people should blog (but don’t)

major@mhtx.net (Major Hayden) — Fri, 30 Mar 2012 14:30:45 +0000

I originally wrote this post for the Rackspace Blog but I decided to post it here in case some of my readers might have missed it. Please feel free to leave your comments at the end of the post.

Sometimes people talk to me about posts I’ve written on my blog, or posts they wish I would write. At some point during the discussion, I’ll almost always ask the person why they don’t start up their own blog or contribute to someone else’s. Very few people actually seem interested when I probe them about writing posts on technical topics.

My mother was always the one who told me (and her students) that everyone has a story. She said that writing could be therapeutic in ways you probably won’t consider until you’ve written something that someone else enjoys. Just as software developers exist to write software for their users, writers exist to write stories for their readers. There’s nothing that says technical people can’t become excellent writers who inspire others to learn and share their knowledge with others.

The goal of this post is to encourage technical people to enjoy writing, write efficiently and feel comfortable doing it. I’ll roll through some of the most common responses I’ve received about why technical people don’t blog about what they know.

I don’t think I’m really an expert on anything. I’m not an authority on any topic I can think of.

I’m leading off with this response because it’s the most critical to refute. If you don’t take away anything else from this post, let it be this: you don’t need to be an expert on a topic to write about it.

You can find examples of this by rolling through some of the posts on my blog. I’d consider myself to be an expert on one, maybe two topics, but I’ve written over 450 posts in the span of just over five years. I certainly didn’t write all of those about the one or two topics I know best.

Write about what you know and don’t be afraid to do a little research to become an authority on something. A great example of this was my post, entitled “Kerberos for haters.” I had almost no expertise in Kerberos. In fact, I couldn’t even configure it properly for my RHCA exam! However, I did a ton of research and began to understand how most of the pieces fit together. Many other people were just as confused and I decided to pack all of the knowledge I had about Kerberos into a blog post. Positive and negative feedback rolled in and it was obvious that my post taught some readers, inspired some others and angered a few.

What a great way to lead into the next response:

What if I say something that isn’t correct? I’ll look like an idiot in front of the whole internet!

Been there, done that. Every writer makes errors and comes up with bad assumptions at least once. Readers will call you out on your mistakes (some do it delicately while others don’t) and it’s your duty to correct your post or correct the reader. I’ve written posts with errors, and I’ve gotten a little lazy on my fact-checking from time to time. As my middle school journalism teacher always reminded me, the most important part of a mistake is what you do to clean it up and learn from it.

In short: you’ll make mistakes. As long as you’ve done your due diligence to minimize them and respond to them promptly, your readers should forgive you.

Speaking of errors:

I’m great at a command prompt but my spelling and grammar are awful. I write terribly.

This is easily fixed. If you’re one of those folks who live the do-it-yourself type of lifestyle, pick up a copy of The Elements of Style by Strunk & White. There are free PDF versions online or you can borrow one from your nearest journalist. No matter the situation you’re in, this book has details about where punctuation should and shouldn’t be, how to structure sentences and paragraphs, and how to properly cite your sources (really vital for research posts).

Hauling around a copy of an ultra-dry reference book may not be your thing. If that’s the case, find someone you know who has a knack for writing. You can usually find helpful folks in marketing or corporate communications in most big companies who will take your post and return it covered in red ink ready for corrections (thanks, Garrett!). I’ve even spotted some folks on Fiverr who will do this for as low as $5.

I’ll wrap up with the second most common response:

I don’t know who I’m writing for? What if I write about something simple and the really technical folks think I’m a noob? What if I write something crazy complex and it goes over most people’s heads?

I’ve done both of these. Most Linux system administrators worth their salt know how to add and remove iptables rules, and they’d consider it to be pretty trivial work. Would it surprise you to know that out of over 450 posts, my post about deleting a single iptables rule is in the top five most accessed posts per month? I receive just over 11 percent of my monthly hits to this post. People are either learning from it or they can’t remember how to delete the rule and they want to use the post as a quick reference. Either way, the post is valuable to many people even if I think it’s the simplest topic possible.

On the flip side, I went nuts and wrote up a complete how-to for a redundant cloud hosting configuration complete with LVS, glusterfs, MySQL on DRBD, memcached, haproxy and ldirectord. I thought it would be valuable knowledge to a few folks but that it might sail over the heads of most of my readers. Again, I was wrong. The post is constantly in the top 10 most visited posts on the blog and I’ve probably received more feedback via comments, email and IRC about that post than any other. Once again, a post I thought would be mostly useless turned into a real conversation starter.

Let’s conclude and wrap up. Keep these things in mind if you feel discouraged about writing:

Write about what interests you whether you’re an expert on it or not
Don’t be afraid to fail
Be responsive to your readers
Even if you think nobody will read your post, write it
Always ensure your voice shines through in your writing — this is what makes it special and appealing