Major Haydenhttps://major.io/Recent content on Major HaydenHugoenmajor@mhtx.net (Major Hayden)major@mhtx.net (Major Hayden)All content licensed [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/) πŸ’œFri, 29 Nov 2024 16:33:02 +0000Repairing 4Runner skid plate boltshttps://major.io/p/4runner-skid-plate-bolt-repair/Tue, 08 Oct 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/4runner-skid-plate-bolt-repair/<p>I replaced my old Toyota 4Runner with a new one so I could snag the last run of the 5th generation. It&rsquo;s a tough, reliable vehicle with just enough space for my family and our pets.</p> <p>However, this new one came with the same problem as my old one. All of the bolts that hold in the front skid plate were mostly stripped. Getting them out was difficult and I knew getting them back in would be worse.</p> <p>I&rsquo;ll cover how to fix it in this post. <strong>If you&rsquo;re in a hurry, scroll past the next section!</strong> Otherwise, let&rsquo;s get a little backstory.</p> <h2 id="backstory" class="relative group">Backstory <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#backstory" aria-label="Anchor">#</a></span></h2><p>All <a href="https://en.wikipedia.org/wiki/Toyota_4Runner" target="_blank" rel="noreferrer">4Runner models</a> are made in Japan in the <a href="https://en.wikipedia.org/wiki/Toyota_Motor_Corporation_Tahara_plant" target="_blank" rel="noreferrer">Tahara Plant</a>. The build quality is fantastic and you can tell that they&rsquo;re built with care. You can even find brief notes in Japanese inside the fenders or underneath the car where someone jotted some notes about something.</p> <p>After careful assembly in Japan, they head to various US ports where American workers add on any extra items that come with the trim level. That could include an upgraded exhaust, different wheels, or in my case, skid plates.</p> <p>The skid plate is a sturdy piece of steel that mounts under the front of the vehicle and protects lots of important components from damage. You can see it under the front bumper in this photo:</p> <p> <figure> <picture class="mx-auto my-0 rounded-md" > <img width="2048" height="1449" class="mx-auto my-0 rounded-md" alt="4runner-skid.jpg" loading="lazy" decoding="async" src="https://major.io/p/4runner-skid-plate-bolt-repair/4runner-skid_hu8891988610697267815.jpg" srcset="https://major.io/p/4runner-skid-plate-bolt-repair/4runner-skid_hu4995082311674378694.jpg 330w,/p/4runner-skid-plate-bolt-repair/4runner-skid_hu8891988610697267815.jpg 660w ,/p/4runner-skid-plate-bolt-repair/4runner-skid_hu17357348160087690803.jpg 1024w ,/p/4runner-skid-plate-bolt-repair/4runner-skid_hu11491577878939773027.jpg 1320w " sizes="100vw" /> </picture> <figcaption class="text-center">Skid plate on a white 4Runner</figcaption> </figure> </p> <p>If you&rsquo;ve ever owned a Toyota, you know that the factory is strict about torque applied to various bolts all over the vehicle. All of that gets thrown out the window when the workers at the US ports add on accessories.</p> <p>Based on all the complaints I&rsquo;ve seen across various 4Runner forums, they must use air wrenches or some kind of impact wrench to put on the bolts. If the bolt isn&rsquo;t in straight when they start, it destroys the bolt and causes problems with the mount holes. They also tighten the bolts <em>far past the acceptable torque specs.</em></p> <p>To make matters worse, if you take your car in for an oil change at most places, <strong>they&rsquo;ll have an impact wrench handy to ruin the bolts a bit more for you.</strong></p> <h2 id="root-cause" class="relative group">Root cause <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#root-cause" aria-label="Anchor">#</a></span></h2><p>If you have bolts that are getting stripped in the mount holes or they&rsquo;re getting stuck as you try to bring the bolts in or out, you likely have chunks of metal from the bolts wedged in the threads of the bolt holes.</p> <h2 id="ingredients" class="relative group">Ingredients <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#ingredients" aria-label="Anchor">#</a></span></h2><p>The fix is quite cheap but very tedious. You&rsquo;ll need a few parts to get started:</p> <ul> <li> <p><a href="https://a.co/d/8EkxZxh" target="_blank" rel="noreferrer">Irwin Hanson 12002 T-Handle Tap Wrench (1/4&quot; to 1/2&quot;)</a>: This T-Handle wrench allows you to easily spin the tap screw to clear the metal fragments from your bolt holes. <strong>Don&rsquo;t use a socket set, drill, screwdriver, or anything powerful!</strong> You want to take this <em>slow</em>.</p> </li> <li> <p><a href="https://a.co/d/gy02vOu" target="_blank" rel="noreferrer">Irwin Tap 10-1 25mm Plug</a>: The bolts that go into the holes are M10 bolts with a 25mm thread, so this tap should fit perfectly.</p> </li> <li> <p><a href="https://a.co/d/grHmNR4" target="_blank" rel="noreferrer">Toyota part PT938-00140-AA</a>: This includes four new bolts with spacers and retaining washers to replace your stripped bolts.</p> </li> <li> <p><strong>14mm socket and socket wrench OR a 14mm wrench:</strong> You&rsquo;ll need this for removing the bolts and dealing with the hardware for the skid plate.</p> </li> <li> <p><strong>Some type of lubricant.</strong> I used WD-40, but don&rsquo;t tell anyone. People love to fight about whether WD-40 is a solvent, a grease, or a lubricant. πŸ€·β€β™‚οΈ</p> </li> </ul> <p>What&rsquo;s hilarious is that if you load the Amazon page for the Toyota part, it shows that everyone is buying tap screws and T-Handle wrenches:</p> <p> <figure> <picture class="mx-auto my-0 rounded-md" > <img width="720" height="301" class="mx-auto my-0 rounded-md" alt="buy-together.jpg" loading="lazy" decoding="async" src="https://major.io/p/4runner-skid-plate-bolt-repair/buy-together_hu13269774056314308932.jpg" srcset="https://major.io/p/4runner-skid-plate-bolt-repair/buy-together_hu15963278734882412371.jpg 330w,/p/4runner-skid-plate-bolt-repair/buy-together_hu13269774056314308932.jpg 660w ,/p/4runner-skid-plate-bolt-repair/buy-together.jpg 720w ,/p/4runner-skid-plate-bolt-repair/buy-together.jpg 720w " sizes="100vw" /> </picture> <figcaption class="text-center">Even Amazon knows these bolts are a problem! πŸ˜†</figcaption> </figure> </p> <h2 id="fix-the-bolt-holes" class="relative group">Fix the bolt holes <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#fix-the-bolt-holes" aria-label="Anchor">#</a></span></h2><p>First things first, you&rsquo;ll need to get that skid plate off. <strong>I strongly recommend taking the rear bolts off first.</strong> If the bolts get stuck on the way out, take your time. I found that rocking in the other direction briefly and then trying to loosen them again seemed to work.</p> <p>With the rear bolts out, move to the bolts closest to the front of the car. Put a box or something sturdy underneath the skid plate that allows it to drop when it&rsquo;s loose but prevents it from knocking out one of your teeth when it falls. πŸ€•</p> <p>When you loosen the front bolts, try loosening one of them 4-5 turns and then go to the other one. Keep going back and forth loosening the bolts until they loosen from the frame of the car. There are retaining washers on the top side of the bolt and removing those bolts aggressively will slide the retaining washers right off the bolt.</p> <p>Now you&rsquo;re ready to tap! πŸ‘</p> <p>Start in with the bolt holes in the rear and spray a decent amount of lubricant in the bottom and top of the bolt hole. Get your tap into the T-Handle wrench and slowly start turning it in the bolt hole like you were installing one of the bolts.</p> <p>πŸ›‘ <strong>When you hit resistance, only go 1/2 to 1 turn further.</strong> Then back up 2-3 turns. This means you&rsquo;ve dislodged some metal fragments in the threads!</p> <p> <figure> <picture class="mx-auto my-0 rounded-md" > <img width="3072" height="4080" class="mx-auto my-0 rounded-md" alt="tapping.jpg" loading="lazy" decoding="async" src="https://major.io/p/4runner-skid-plate-bolt-repair/tapping_hu17961248824566662867.jpg" srcset="https://major.io/p/4runner-skid-plate-bolt-repair/tapping_hu6843303268591704604.jpg 330w,/p/4runner-skid-plate-bolt-repair/tapping_hu17961248824566662867.jpg 660w ,/p/4runner-skid-plate-bolt-repair/tapping_hu7491105741113099802.jpg 1024w ,/p/4runner-skid-plate-bolt-repair/tapping_hu8766447198537622698.jpg 1320w " sizes="100vw" /> </picture> <figcaption class="text-center">Working on one of the back holes πŸ’ͺ</figcaption> </figure> </p> <p>After backing up a bit, keep screwing it in further until you hit more resistance. Only go 1/2 to 1 turn more, then back out 2-3 turns. Keep doing this until your tap shows up out of the top side of the bolt hole.</p> <p>With your tap sticking out of the top of the hole, grab a shop towel or paper towel and clear all of the metal filings away from the top of the hole. Then back the tap all the way out and clean your tap screw. It&rsquo;s likely going to be covered in black shavings.</p> <p>If you want to be really thorough, lubricate the hole once more and keep working the tap until the threads feel really smooth. I added some lubricant and fed a <strong>new bolt</strong> in through the top <strong>using finger strength only</strong> until I knew the threads were clear.</p> <p>You can test feed a <strong>new bolt</strong> through the bottom to verify that you&rsquo;ve done a good job. Don&rsquo;t use the old bolts for this. You&rsquo;ll just get more junk in the threads again. 😭</p> <p>Repeat these steps for the other four holes and you should be all set.</p> <h2 id="replace-the-skid-plate" class="relative group">Replace the skid plate <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#replace-the-skid-plate" aria-label="Anchor">#</a></span></h2><p>Be sure to discard the old bolts to avoid causing more problems for yourself. Re-assemble the new front bolts with the spacer and retaining washer just like they were when you took the skid plate down.</p> <p>Get the back bolts going in first and get them in about halfway. Start working on the front bolts after that.</p> <p>When all four bolts are in, grab your torque wrench and tighten them to <strong>22 ft/lbs or 30 Nm</strong> as specified in the <a href="trd_skid_plate_install_instructions.pdf">manual</a>:</p> <p> <figure> <picture class="mx-auto my-0 rounded-md" > <img width="667" height="387" class="mx-auto my-0 rounded-md" alt="skid-plate-torque.png" loading="lazy" decoding="async" src="https://major.io/p/4runner-skid-plate-bolt-repair/skid-plate-torque_hu9736490088889346127.png" srcset="https://major.io/p/4runner-skid-plate-bolt-repair/skid-plate-torque_hu4008253427526604349.png 330w,/p/4runner-skid-plate-bolt-repair/skid-plate-torque_hu9736490088889346127.png 660w ,/p/4runner-skid-plate-bolt-repair/skid-plate-torque.png 667w ,/p/4runner-skid-plate-bolt-repair/skid-plate-torque.png 667w " sizes="100vw" /> </picture> <figcaption class="text-center">Always check the instructions for torque specs! πŸ”§</figcaption> </figure> </p> <p>If you don&rsquo;t own a torque wrench, I wouldn&rsquo;t tighten them much past finger tight with a socket wrench. Then drive to the hardware store and get a basic torque wrench. 😜</p> <h2 id="prevention" class="relative group">Prevention <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#prevention" aria-label="Anchor">#</a></span></h2><p>This is easy but annoying: <strong>always remove the skid plate yourself before any kind of maintenance trip.</strong></p> <p>Yes, this sounds silly, but these mount points are finicky and there&rsquo;s no guarantee that the dreaded impact wrench will not show up again to ruin your bolts. I remove mine before any oil changes or scheduled maintenance at the dealer.</p> <p>Dealers have asked me in the past &ldquo;Where&rsquo;s your skid plate anyway?&rdquo; and I let them know I don&rsquo;t want my bolts stripped. πŸ˜…</p>Spell check in multiple languages with Firefoxhttps://major.io/p/firefox-multi-language-spell-check/Sun, 25 Aug 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/firefox-multi-language-spell-check/<p><strong>Bienvenidos!</strong><sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> I&rsquo;ve been learning Spanish for just over a year and I often type messages in either Spanish or English (my native language) with coworkers and friends. Just like most people, I make spelling mistakes in both languages. πŸ™ƒ</p> <p>Firefox offers a feature for multi-language spell checking and translations but it can be a bit challenging to set up. This post explains how to load languages into Firefox and use them for spell checking.</p> <h2 id="installing-languages" class="relative group">Installing languages <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#installing-languages" aria-label="Anchor">#</a></span></h2><p>Take a trip over to [Dictionaries and Languages Packs] on Mozilla&rsquo;s site. Note that there are <strong>two columns</strong> available to you here:</p> <ul> <li><strong>Language packs</strong> give you the option to change your interface language to something different than your system&rsquo;s default language.</li> <li><strong>Dictionaries</strong> help with checking spelling.</li> </ul> <p>In the second column, click on the language you want to add for checking spelling. In my case, I picked the <a href="https://addons.mozilla.org/en-US/firefox/addon/diccionario-de-espa%C3%B1ol-espa%C3%B1a/" target="_blank" rel="noreferrer">Spanish (Spain) Dictionary</a> along with the <a href="https://addons.mozilla.org/en-US/firefox/addon/spanish-mexico-dictionary/" target="_blank" rel="noreferrer">Spanish (Mexico) Dictionary</a>. Install the dictionaries you want just like any other add-on!</p> <p>Go to the <code>about:addons</code> page in Firefox and you should see your languages under <strong>Languages</strong> and <strong>Dictionaries</strong> on the left side.</p> <h2 id="enable-the-language" class="relative group">Enable the language <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#enable-the-language" aria-label="Anchor">#</a></span></h2><p>Find an input field and right click inside the field. You should see a <strong>Languages</strong> context menu appear. Roll over that menu and a new menu pops out to the side:</p> <p> <figure> <picture class="mx-auto my-0 rounded-md" > <img width="502" height="502" class="mx-auto my-0 rounded-md" alt="context_menu.png" loading="lazy" decoding="async" src="https://major.io/p/firefox-multi-language-spell-check/contextmenu.png" /> </picture> <figcaption class="text-center">Firefox context menu showing multiple languages</figcaption> </figure> </p> <p>Click the checkbox to enable the languages that you want to use with the spell checker. That takes effect immediately!</p> <p>Gracias por leer hasta aquΓ­!<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup> 😜</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>Welcome!&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p>Thank you for reading this far.&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>My meeting hackshttps://major.io/p/meeting-hacks/Thu, 22 Aug 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/meeting-hacks/<p>Ask anyone about the toughest part of their workday and it usually comes down to one thing: meetings. There are plenty of reasons:</p> <ul> <li>The meeting could have been an email</li> <li>Nobody notices when I attend the meeting, but they notice when I don&rsquo;t</li> <li>The meeting is recurring whether there&rsquo;s something important to talk about or not</li> <li>There&rsquo;s no time for questions after everyone presents in a meeting</li> <li>Someone dominates the conversation</li> </ul> <p>This was a central problem in my <a href="https://txlf24-tech-career.major.io/#/" target="_blank" rel="noreferrer">&ldquo;Five tips for a thriving technology career&rdquo;</a> talk that I delivered this year. I wrote a <a href="https://major.io/p/texas-linux-fest-2024-recap/">recap</a> on the blog earlier this summer as well.</p> <p>I came up with some more ideas since then, so let&rsquo;s go!</p> <h2 id="use-headphones-or-earbuds" class="relative group">Use headphones or earbuds <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#use-headphones-or-earbuds" aria-label="Anchor">#</a></span></h2><p>I find it much easier to understand people in meetings when I have the audio closer to my ears. This helps a lot with understanding non-native English speakers or some native English speakers with thick accents. It reduces the noise from various things in my house (kids, pets, appliances) and allows me to focus on the small sounds that are important for understanding someone else.</p> <p>How many times have you been in a meeting with someone who talks constantly without earbuds or headphones and you can&rsquo;t break through with your own voice? Some computers will mute the incoming audio to avoid feedback sounds and you&rsquo;ll totally miss it when someone is trying to get your attention.</p> <p>I was once in a meeting where an attendee spoke at length about a topic that was already covered and multiple people tried to speak to let him know that he could stop. He was completely oblivious. The situation improved a lot recently with the addition of &ldquo;raised hands&rdquo; indicators in most meeting applications, but it&rsquo;s still not perfect.</p> <h2 id="background-music" class="relative group">Background music <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#background-music" aria-label="Anchor">#</a></span></h2><p>If you typically join meetings without earbuds or headphones, then this suggestion isn&rsquo;t for you. <strong>Also, you should go back and re-read the previous section.</strong> ️😜</p> <p>Everyone has their own music preferences, but I find that playing some relaxing music at a low volume really helps me stay focused during meetings. I change the genre of music between different days depending on my mood. No matter what you choose, consider music without vocals to avoid distractions.</p> <p>A good place to start is Lofi Girl&rsquo;s &ldquo;beats to relax/study to&rdquo; playlist. You can listen on <a href="https://open.spotify.com/playlist/0vvXsWCC9xrXsKd4FyS8kM?si=dba7e37978e246bf" target="_blank" rel="noreferrer">Spotify</a> or on <a href="https://www.youtube.com/watch?v=jfKfPfyJRdk" target="_blank" rel="noreferrer">YouTube</a>. Very few songs have vocals, and if they do, it&rsquo;s barely noticeable. I&rsquo;ve found that I can keep this playlist on for hours without getting bored of it.</p> <p>This can be especially helpful for those marathon half or full day meetings. πŸ‘”</p> <h2 id="ask-about-taking-notes" class="relative group">Ask about taking notes <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#ask-about-taking-notes" aria-label="Anchor">#</a></span></h2><p>Most meeting platforms offer transcription and audio/video recording already, but transcriptions are difficult to read and recordings aren&rsquo;t usually fun to watch. I love it when someone takes some concise notes about the points that were raised, who raised them, and who holds the action items to solve them.</p> <p><strong>If nobody&rsquo;s taking notes, ask if you can!</strong></p> <p>It&rsquo;s a great way to ensure you pay attention and the people who missed the meeting will thank you later. Nobody has turned me down yet when I&rsquo;ve asked.</p> <p>This can also be helpful if someone likes to talk over everyone else during the meeting or if someone birdwalks into other topics. Un-mute yourself and ask:</p> <blockquote> <p>Wait, are we still on that previous topic or have we moved to something else?</p> </blockquote> <p>Another favorite question of mine is:</p> <blockquote> <p>Did we get an action item for that previous topic? Who owns that item?</p> </blockquote> <p>Your note taking keeps speakers on track and ensures there is accountability and ownership for problems that need to be solved. It&rsquo;s also a great way to get your name in front of other people during larger meetings.</p> <h2 id="decline-that-meeting" class="relative group">Decline that meeting! <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#decline-that-meeting" aria-label="Anchor">#</a></span></h2><p>This one got the biggest reaction during my talk at Texas Linux Fest. Sometimes you just need to decline a meeting. Certain aspects of a meeting will push me to the &ldquo;No&rdquo; button faster than others, but here&rsquo;s my two biggest red flags:</p> <ul> <li> <p><strong>More than three attendees:</strong> It&rsquo;s difficult to get much done with a meeting that has 25 people in it. If someone sends me a calendar invitation unannounced and there are more than 3-5 people in the meeting, I ask them on Slack what I&rsquo;m expected to bring to the meeting. Often times, I hear <em>&ldquo;Oh, we wanted to be sure you were informed, but there are no action items for you.&rdquo;</em> That&rsquo;s a great time to say: <em>&ldquo;Can you send me the recording or the notes when it&rsquo;s over?&rdquo;</em></p> </li> <li> <p><strong>Missing agenda:</strong> If I&rsquo;m taking time out of my day to meet, I want to know about the meeting&rsquo;s goals. What should we have as we leave the meeting? Will we leave with a plan to do something? A set of decisions? Questions for another team?</p> </li> </ul> <p><strong>You are the only one that can advocate for your own time.</strong> Nobody else is going to do that for you<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. A very talented executive once told me:</p> <blockquote> <p>Time is the most valuable thing you bring to work every day. You can&rsquo;t get more of it, but you can waste it. Your experience and knowledge means nothing if you don&rsquo;t have time to use it. Treat your time as your most precious asset.</p> </blockquote> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>An administrative assistant can help but I&rsquo;ve never had one before. 😜&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>Rub some AI on ithttps://major.io/p/rub-some-ai-on-it/Wed, 21 Aug 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/rub-some-ai-on-it/<p><em>Author&rsquo;s note: This post is all about my personal thoughts on artificial intelligence (AI) and they don&rsquo;t represent the views of any employer or group.</em></p> <hr> <p>You can&rsquo;t escape the clutches of AI lately.</p> <p>It&rsquo;s in my smartphone <a href="https://support.google.com/messages/answer/14599070?hl=en" target="_blank" rel="noreferrer">nestled</a> next to my text messages. It&rsquo;s in my <a href="https://slack.com/features/ai" target="_blank" rel="noreferrer">work chats</a>. It&rsquo;s <a href="https://blog.duolingo.com/large-language-model-duolingo-lessons/" target="_blank" rel="noreferrer">reading my Spanish</a> in Duolingo. It&rsquo;s in my photo albums <a href="https://blog.adobe.com/en/publish/2023/04/18/new-adobe-lightroom-ai-innovations-empower-everyone-edit-like-pro" target="_blank" rel="noreferrer">retouching my images</a>.</p> <p>Sometimes we know that there&rsquo;s AI involved in something and sometimes we don&rsquo;t.</p> <p>However, it seems like so many are in a rush to implement some kind of AI offering without a full idea of why they&rsquo;re doing it. Here an excerpt from the <a href="https://hbr.org/2024/09/ai-wont-give-you-a-new-sustainable-advantage" target="_blank" rel="noreferrer">September 2024 issue</a> of Harvard Business Review that explains it well:</p> <blockquote> <p>Smart early movers in sectors adopting gen AI have certainly captured some of this value in the short term. But relatively soon all surviving companies in those sectors will have applied gen AI, and it won’t be a source of competitive advantage for any one of them, even where its impact on business and business practices will probably be profound. In fact, it will be more likely to remove a competitive advantage than to confer one. <strong>But here’s a silver lining: If you already have a competitive advantage that rivals cannot replicate using AI, the technology may serve to amplify the value you derive from that advantage.</strong></p> </blockquote> <p>AI can help you only if:</p> <ol> <li>You have a product or service your customers value.</li> <li>You can leverage AI for specific improvements to that product or service that make it more valuable.</li> </ol> <h2 id="ai-is-not-valuable-alone" class="relative group">AI is not valuable alone <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#ai-is-not-valuable-alone" aria-label="Anchor">#</a></span></h2><p>I&rsquo;m reminded of a time in the past where I was working hard on OpenStack public clouds. Kubernetes gained more traction day by day. Lots of customers told us: <em>&ldquo;I&rsquo;ve got to get on kubernetes so I can move faster.&rdquo;</em></p> <p>As we asked more about their challenges, they listed lots of things that should look familiar:</p> <ul> <li>Developers throwing code over the wall to Q/E and Q/E delays the release</li> <li>Software passes tests in development and staging, but fails miserably in production</li> <li>Monolithic applications were crushed by load spikes</li> <li>Operations teams struggled to deploy software efficiently and reliably</li> </ul> <p>They had a serious problem with delivering their software, but kubernetes couldn&rsquo;t make any of these better. <strong>Adding kubernetes would just give them two problems instead of one.</strong></p> <p>The running joke whenever someone ran into a problem with a server, a piece of code, or a service was to say <em>&ldquo;Why don&rsquo;t you rub a little kubernetes on it?&rdquo;</em> 🀣</p> <p>I&rsquo;m seeing much of the same with AI as companies scramble to get their hands on the best hardware they can find and access to the highest quality large language models (LLMs) they can find. Cloud budgets are blown wide open. When someone asks about the AI effort, the reply is often: <em>&ldquo;We have to get it before our competitors do, or we&rsquo;re sunk!&rdquo;</em></p> <p>In February 2024, 36% of company earnings reports <a href="https://markets.businessinsider.com/news/stocks/ai-stocks-sp500-4q-tech-earnings-artificial-intelligence-goldman-sachs-2024-2?op=1" target="_blank" rel="noreferrer">mentioned AI</a> &ndash; a record high:</p> <p> <figure> <picture class="mx-auto my-0 rounded-md" > <img width="700" height="441" class="mx-auto my-0 rounded-md" alt="ai_mentions.webp" loading="lazy" decoding="async" src="https://major.io/p/rub-some-ai-on-it/ai_mentions_hu1123981360019847793.webp" srcset="https://major.io/p/rub-some-ai-on-it/ai_mentions_hu10245921421230993916.webp 330w,/p/rub-some-ai-on-it/ai_mentions_hu1123981360019847793.webp 660w ,/p/rub-some-ai-on-it/ai_mentions.webp 700w ,/p/rub-some-ai-on-it/ai_mentions.webp 700w " sizes="100vw" /> </picture> </figure> </p> <p>How many of them are actually doing something meaningful for their employees or customers with AI?</p> <h2 id="work-backwards-from-the-experience" class="relative group">Work backwards from the experience <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#work-backwards-from-the-experience" aria-label="Anchor">#</a></span></h2><p>One of my coworkers, Scott McCarty, wrote a great post on InfoWorld titled <a href="https://www.infoworld.com/article/3482087/what-generative-ai-can-do-for-sysadmins.html" target="_blank" rel="noreferrer">&ldquo;What generative AI can do for sysadmins&rdquo;</a>. What I love most about this article is that Scott remains laser-focused on the <em>experiences</em> and <em>challenges</em> that AI could improve.</p> <p>There are plenty of challenging situations that every sysadmin faces. The worst of these are when you&rsquo;re under incredible pressure to bring a system back into a working state and you need to pick through tons of information to identify the problem. You can sometimes spot these issues easily, such as a failing storage drive in a server. Other situations are much more difficult.</p> <p>The key is to <strong>start with the experience.</strong> Then work backwards from there.</p> <p>As an example, one pattern I often see is companies putting AI chatbots in front of their documentation. Sometimes the chatbot will help you find the right documentation faster, but sometimes it&rsquo;s not much better than a CTRL-F or a quick look at the documentation&rsquo;s table of contents.</p> <p>If your documentation is so complicated that you need to spend the time and money to put an AI chatbot in front of it, why not make your documentation better instead?</p> <p>When something does fail, why not put a link to the documentation in the log message itself? This pattern shows up a lot in modern software lately. If I try to enable a <a href="https://major.io/p/build-tailscale-exit-node-firewalld/">Tailscale exit node</a> but I haven&rsquo;t forwarded packets on an interface, I get quick instructions on the console with a link to documentation that explains it in more detail.</p> <p><strong>You cannot use AI to paper over a poor experience.</strong> Your customers will see right through it.</p> <h2 id="remember-the-human-side" class="relative group">Remember the human side <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#remember-the-human-side" aria-label="Anchor">#</a></span></h2><p>Sometimes companies simply try to take AI much too far and upset the human nature in all of us.</p> <p>A great example of this was Google&rsquo;s awful Olympics ad where it shows a girl&rsquo;s father using Google Gemini to <a href="https://www.cnn.com/2024/08/02/tech/google-olympics-ai-ad-artificial-intelligence/index.html" target="_blank" rel="noreferrer">write a letter to her hero</a>. The reaction at my house when we saw it was: <em>&ldquo;Wait, you&rsquo;re taking the time to write a letter to your hero and you&rsquo;re letting an AI write it? Could that be any more impersonal?&rdquo;</em> If I&rsquo;m writing a letter or email to someone I admire, I&rsquo;m taking the time to write it myself with my own voice.</p> <p>Another example is a Microsoft ad showing someone turning a long document into a long slide deck instead. If nobody wanted to read the document in the first place, why would they want to read your long slide deck? Also, how would they feel if they know you just jammed a document into a LLM to make a slide deck and then held them hostage in a conference room as you walked through a voiceless set of slides?</p> <p>This goes back to the last section, but if you&rsquo;re trying to add AI to replace a human interaction, think that through. Are you papering over a bad experience? Are you looking to cut costs without considering the customer reaction? What&rsquo;s your plan if the AI interactions backfire?</p> <h2 id="so-what-do-we-do" class="relative group">So what do we do? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#so-what-do-we-do" aria-label="Anchor">#</a></span></h2><p>If you work backwards from the customer experience and land on an LLM as the best way to solve a problem or enhance a product, that&rsquo;s great. However, the experience you are enabling should be so good that:</p> <ol> <li>Customers are genuinely delighted with the experience without knowing AI is involved</li> <li>You don&rsquo;t have to mention &ldquo;AI&rdquo; for the experience to feel innovative and delightful</li> <li>You have plans in place for when customers want more from the experience later</li> </ol> <p>Getting hardware or cloud infrastructure together and <a href="https://cfp.fedoraproject.org/flock-2024/talk/HM9Y8U/" target="_blank" rel="noreferrer">running an LLM on top is boring</a>. Even going retrieval augmented generation (RAG) is boring. We will soon live in a world where running an LLM is the same level of difficulty as running a web server or a container. That&rsquo;s not where the value lives.</p> <p>AI isn&rsquo;t the king. <strong>The experience is.</strong> If you forget that, you&rsquo;re just taking a problem and rubbing some AI on it.</p>AMD GPU missing from btophttps://major.io/p/amd-gpu-missing-btop/Tue, 20 Aug 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/amd-gpu-missing-btop/<p>I recently built a new PC for my birthday and I splurged a bit with a new AMD Radeon 7900 XTX GPU. Although I&rsquo;m not a heavy gamer, I&rsquo;m working with <a href="https://en.wikipedia.org/wiki/Large_language_model" target="_blank" rel="noreferrer">LLMs</a> more often and I&rsquo;m interested to do some of this work at home.</p> <p><a href="https://github.com/aristocratos/btop" target="_blank" rel="noreferrer">btop</a> is my go-to tracker for all kinds of data about my system, including CPU usage, memory usage, disk I/O, and network throughput. It&rsquo;s a great way to track down bottlenecks and find out why your CPU fan is spinning at max speed. 😜</p> <p> <figure> <picture class="mx-auto my-0 rounded-md" > <img width="1908" height="1053" class="mx-auto my-0 rounded-md" alt="btop.png" loading="lazy" decoding="async" src="https://major.io/p/amd-gpu-missing-btop/btop_hu10247371322846921946.png" srcset="https://major.io/p/amd-gpu-missing-btop/btop_hu2344920120220028327.png 330w,/p/amd-gpu-missing-btop/btop_hu10247371322846921946.png 660w ,/p/amd-gpu-missing-btop/btop_hu16483154301455967255.png 1024w ,/p/amd-gpu-missing-btop/btop_hu1581830467410715261.png 1320w " sizes="100vw" /> </picture> <figcaption class="text-center">Screenshot of btop running on my system</figcaption> </figure> </p> <h2 id="the-problem" class="relative group">The problem <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#the-problem" aria-label="Anchor">#</a></span></h2><p>My GPU wasn&rsquo;t showing up in btop after rebuilding the system. Normally, there&rsquo;s a bar for the GPU right underneath the CPU usage and it tracks the GPU usage as well as memory usage. Some cards report thermals there, too.</p> <p>The <a href="https://github.com/clbr/radeontop" target="_blank" rel="noreferrer">radeontop</a> tool worked fine and I can see the device in the hardware monitoring subsystem:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">&gt;</span> cat /sys/class/hwmon/hwmon1/name </span></span><span class="line"><span class="cl"><span class="go">amdgpu </span></span></span></code></pre></div><h2 id="the-solution" class="relative group">The solution <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#the-solution" aria-label="Anchor">#</a></span></h2><p>I installed plenty of AMD packages, but I missed a critical one: <code>rocm-smi</code>:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">&gt;</span> dnf install rocm-smi </span></span><span class="line"><span class="cl"><span class="gp">&gt;</span> rocm-smi </span></span><span class="line"><span class="cl"><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go">======================================== ROCm System Management Interface ======================================== </span></span></span><span class="line"><span class="cl"><span class="go">================================================== Concise Info ================================================== </span></span></span><span class="line"><span class="cl"><span class="go">Device Node IDs Temp Power Partitions SCLK MCLK Fan Perf PwrCap VRAM% GPU% </span></span></span><span class="line"><span class="cl"><span class="go"> (DID, GUID) (Edge) (Avg) (Mem, Compute, ID) </span></span></span><span class="line"><span class="cl"><span class="go">================================================================================================================== </span></span></span><span class="line"><span class="cl"><span class="go">0 1 0x744c, 55924 47.0Β°C 21.0W N/A, N/A, 0 218Mhz 96Mhz 0% auto 327.0W 11% 10% </span></span></span><span class="line"><span class="cl"><span class="go">================================================================================================================== </span></span></span><span class="line"><span class="cl"><span class="go">============================================== End of ROCm SMI Log =============================================== </span></span></span></code></pre></div><p>That&rsquo;s the ticket! Now my btop data is complete.</p> <p> <figure> <picture class="mx-auto my-0 rounded-md" > <img width="526" height="308" class="mx-auto my-0 rounded-md" alt="btop-magnified.png" loading="lazy" decoding="async" src="https://major.io/p/amd-gpu-missing-btop/btop-magnified.png" /> </picture> <figcaption class="text-center">btop showing my GPU stats</figcaption> </figure> </p>Running ollama with an AMD Radeon 6600 XThttps://major.io/p/ollama-with-amd-radeon-6600xt/Thu, 08 Aug 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/ollama-with-amd-radeon-6600xt/<p>I&rsquo;m splitting time between two roles at work now and one of the roles has a heavy focus on <a href="https://en.wikipedia.org/wiki/Large_language_model" target="_blank" rel="noreferrer">LLMs</a>. Much like many of you, I&rsquo;ve given ChatGPT a try with questions from time to time. I&rsquo;ve also used GitHub Copilot within Visual Studio Code.</p> <p>They&rsquo;re all great, but I was really hoping to run something locally on my machine at home.</p> <p>Then I stumbled upon a great post on All Things Open titled &ldquo;<a href="https://allthingsopen.org/articles/build-a-local-ai-co-pilot" target="_blank" rel="noreferrer">Build a local AI co-pilot using IBM Granite Code, Ollama, and Continue</a>&rdquo; that started me down a path with <a href="https://ollama.com/" target="_blank" rel="noreferrer">ollama</a>. The ollama project gets you started with a local LLM and makes it easy to serve it for other applications to use.</p> <h2 id="its-so-slow-" class="relative group">It&rsquo;s so slow 🐌 <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#its-so-slow-" aria-label="Anchor">#</a></span></h2><p>When I first began connecting vscode to ollama, I noticed that the responses were incredibly slow. A quick check with <a href="https://github.com/aristocratos/btop" target="_blank" rel="noreferrer">btop</a> showed that my CPU was maxed out at 100% utilization and my GPU was entirely idle. That&rsquo;s not good.</p> <p>My first thought was to check the system journal with <code>sudo journalctl --boot -u ollama</code>. That gets me all the messages from ollama since I last booted the machine.</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">source=images.go:781 msg=&#34;total blobs: 0&#34; </span></span><span class="line"><span class="cl">source=images.go:788 msg=&#34;total unused blobs removed: 0&#34; </span></span><span class="line"><span class="cl">source=routes.go:1155 msg=&#34;Listening on 127.0.0.1:11434 (version 0.3.4)&#34; </span></span><span class="line"><span class="cl">source=payload.go:30 msg=&#34;extracting embedded files&#34; dir=/tmp/ollama1586759388/runners </span></span><span class="line"><span class="cl">source=payload.go:44 msg=&#34;Dynamic LLM libraries [cpu_avx cpu_avx2 cuda_v11 rocm_v60102 cpu]&#34; </span></span><span class="line"><span class="cl">source=gpu.go:204 msg=&#34;looking for compatible GPUs&#34; </span></span><span class="line"><span class="cl">source=amd_linux.go:59 msg=&#34;ollama recommends running the https://www.amd.com/en/support/linux-drivers&#34; error=&#34;amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory&#34; </span></span><span class="line"><span class="cl">source=amd_linux.go:340 msg=&#34;amdgpu is not supported&#34; gpu=0 gpu_type=gfx1032 library=/usr/lib64 supported_types=&#34;[gfx1030 gfx1100 gfx1101 gfx1102]&#34; </span></span><span class="line"><span class="cl">source=amd_linux.go:342 msg=&#34;See https://github.com/ollama/ollama/blob/main/docs/gpu.md#overrides for HSA_OVERRIDE_GFX_VERSION usage&#34; </span></span><span class="line"><span class="cl">source=amd_linux.go:360 msg=&#34;no compatible amdgpu devices detected&#34; </span></span></code></pre></div><p>A couple of things in the output stood out to me:</p> <ul> <li><code>stat /sys/module/amdgpu/version: no such file or directory</code></li> <li><code>msg=&quot;amdgpu is not supported&quot; gpu=0 gpu_type=gfx1032 library=/usr/lib64 supported_types=&quot;[gfx1030 gfx1100 gfx1101 gfx1102]&quot;</code></li> <li><code>&quot;See https://github.com/ollama/ollama/blob/main/docs/gpu.md#overrides for HSA_OVERRIDE_GFX_VERSION usage&quot;</code></li> </ul> <p>Sure enough, the version was missing:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">&gt;</span> stat /sys/module/amdgpu/version </span></span><span class="line"><span class="cl"><span class="go">stat: cannot statx &#39;/sys/module/amdgpu/version&#39;: No such file or directory </span></span></span></code></pre></div><p>And my AMD GPU is indeed an AMD Navi 23 chipset (gfx1032):</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">&gt;</span> lspci <span class="p">|</span> grep -i VGA </span></span><span class="line"><span class="cl"><span class="go">0f:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 23 [Radeon RX 6600/6600 XT/6600M] (rev c7) </span></span></span></code></pre></div><p>I went over to the <a href="https://github.com/ollama/ollama/blob/main/docs/gpu.md#overrides" target="_blank" rel="noreferrer">linked overrides documentation</a> to figure out what <code>HSA_OVERRIDE_GFX_VERSION</code> is all about:</p> <blockquote> <p>Ollama leverages the AMD ROCm library, which does not support all AMD GPUs. In some cases you can force the system to try to use a similar LLVM target that is close. For example The Radeon RX 5400 is gfx1034 (also known as 10.3.4) however, ROCm does not currently support this target. The closest support is gfx1030. You can use the environment variable HSA_OVERRIDE_GFX_VERSION with x.y.z syntax. So for example, to force the system to run on the RX 5400, you would set HSA_OVERRIDE_GFX_VERSION=&ldquo;10.3.0&rdquo; as an environment variable for the server. If you have an unsupported AMD GPU you can experiment using the list of supported types below.</p> </blockquote> <h2 id="the-fix" class="relative group">The fix <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#the-fix" aria-label="Anchor">#</a></span></h2><p>The docs recommended setting <code>HSA_OVERRIDE_GFX_VERSION=&quot;10.3.0&quot;</code> to see if my card will work. Let&rsquo;s edit the systemd unit file for ollama to drop in some additional configuration:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">&gt;</span> sudo systemctl edit ollama.service </span></span></code></pre></div><p>An editor appeared with text in it:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="c1">### Editing /etc/systemd/system/ollama.service.d/override.conf</span> </span></span><span class="line"><span class="cl"><span class="c1">### Anything between here and the comment below will become the contents of the drop-in file</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1">### Edits below this comment will be discarded</span> </span></span></code></pre></div><p>So I added the suggested override along with the path to my AMD ROCm directory:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="c1">### Editing /etc/systemd/system/ollama.service.d/override.conf</span> </span></span><span class="line"><span class="cl"><span class="c1">### Anything between here and the comment below will become the contents of the drop-in file</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">[Service]</span> </span></span><span class="line"><span class="cl"><span class="na">Environment</span><span class="o">=</span><span class="s">&#34;HSA_OVERRIDE_GFX_VERSION=10.3.0&#34;</span> </span></span><span class="line"><span class="cl"><span class="na">Environment</span><span class="o">=</span><span class="s">&#34;ROCM_PATH=/opt/rocm&#34;</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1">### Edits below this comment will be discarded</span> </span></span></code></pre></div><p>Then I can tell systemd to reload the unit and restart ollama:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">&gt;</span> sudo systemctl daemon-reload </span></span><span class="line"><span class="cl"><span class="gp">&gt;</span> sudo systemctl stop ollama </span></span><span class="line"><span class="cl"><span class="gp">&gt;</span> sudo systemctl start ollama </span></span></code></pre></div><p>Back to the system journal for another look:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">source=amd_linux.go:348 msg=&#34;skipping rocm gfx compatibility check&#34; HSA_OVERRIDE_GFX_VERSION=10.3.0 </span></span><span class="line"><span class="cl">source=types.go:105 msg=&#34;inference compute&#34; id=0 library=rocm compute=gfx1032 driver=0.0 name=1002:73ff total=&#34;8.0 GiB&#34; available=&#34;5.9 GiB&#34; </span></span></code></pre></div><p>Success! πŸŽ‰</p> <h2 id="giving-it-another-try" class="relative group">Giving it another try <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#giving-it-another-try" aria-label="Anchor">#</a></span></h2><p>I went back to vscode and tried some code completions, but they were only slightly faster than using the CPU. Each time I&rsquo;d wait for completion, I&rsquo;d watch btop and the GPU would spike, then the CPU, then the GPU spikes again, and so on.</p> <p>After talking with a coworker, it looks like my Radeon 6600 XT is great for games, but it lacks the RAM needed to load the model into the GPU. 😭 From what I&rsquo;ve read, 24GB is the suggested minimum and that&rsquo;s the largest amount of RAM you&rsquo;ll find in most GeForce/Radeon consumer graphics cards.</p>Jellyfin fatal player errorhttps://major.io/p/jellyfin-fatal-player-error/Tue, 02 Jul 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/jellyfin-fatal-player-error/<p>Plex has been a mainstay for serving up media at home but it seems to have changed lately towards a more and more commercial offering. A friend recommended <a href="https://jellyfin.org/" target="_blank" rel="noreferrer">Jellyfin</a> and I deployed it on my Synology NAS in a Docker container.</p> <p>I did a few quick tests in a web browser and everything looked good. But then my Jellyfin android app told me:</p> <blockquote> <p>Playback failed due to a fatal player error</p> </blockquote> <p>Everything looked fine in the browser, so it was time to dig in.</p> <h2 id="checking-the-logs" class="relative group">Checking the logs <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#checking-the-logs" aria-label="Anchor">#</a></span></h2><p>I opened up an ssh connection on the Synology to check the logs and found something unhelpful:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-plain" data-lang="plain"><span class="line"><span class="cl">Jellyfin.Api.Helpers.TranscodingJobHelper: FFmpeg exited with code 1 </span></span></code></pre></div><p>Running a few searches led me down rabbit holes to plenty of GitHub issues. None of them fixed the issue.</p> <h2 id="checking-the-browser-again" class="relative group">Checking the browser again <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#checking-the-browser-again" aria-label="Anchor">#</a></span></h2><p>I went through a few different videos from the Synology and played each. They all looked fine in Firefox until I reached one that seemed to stutter. The frame rate looked as if at least half of the frames were bring dropped.</p> <p>That particular video was in 4K with a high bit rate. Back on the synology, the CPU usage was through the roof.</p> <p>I configured graphics acceleration when I deployed Jellyfin. Perhaps it wasn&rsquo;t working?</p> <h2 id="jellyfin-deployment" class="relative group">Jellyfin deployment <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#jellyfin-deployment" aria-label="Anchor">#</a></span></h2><p>I deployed Jellyfin using the upstream guides with docker-compose:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">jellyfin</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">docker.io/jellyfin/jellyfin:latest</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">container_name</span><span class="p">:</span><span class="w"> </span><span class="l">jellyfin</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">user</span><span class="p">:</span><span class="w"> </span><span class="m">1026</span><span class="p">:</span><span class="m">100</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">network_mode</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;host&#34;</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">devices</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">/dev/dri</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">volumes</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># removed</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">restart</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;unless-stopped&#34;</span><span class="w"> </span></span></span></code></pre></div><p>One of the GitHub issues I stumbled upon suggested being specific about the video devices that are mounted inside the container.</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> ls -al /dev/dri </span></span><span class="line"><span class="cl"><span class="go">total 0 </span></span></span><span class="line"><span class="cl"><span class="go">drwxr-xr-x 2 root root 80 Jun 10 20:23 . </span></span></span><span class="line"><span class="cl"><span class="go">drwxr-xr-x 12 root root 14140 Jun 10 20:24 .. </span></span></span><span class="line"><span class="cl"><span class="go">crw------- 1 root root 226, 0 Jun 10 20:24 card0 </span></span></span><span class="line"><span class="cl"><span class="go">crw-rw---- 1 root videodriver 226, 128 Jun 10 20:24 renderD128 </span></span></span></code></pre></div><p>I adjusted the deployment in <code>docker-compose.yaml</code> and tried again:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">jellyfin</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">docker.io/jellyfin/jellyfin:latest</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">container_name</span><span class="p">:</span><span class="w"> </span><span class="l">jellyfin</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">user</span><span class="p">:</span><span class="w"> </span><span class="m">1026</span><span class="p">:</span><span class="m">100</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">network_mode</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;host&#34;</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">devices</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">/dev/dri/renderD128:/dev/dri/renderD128</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">/dev/dri/card0:/dev/dri/card0</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">volumes</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># removed</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">restart</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;unless-stopped&#34;</span><span class="w"> </span></span></span></code></pre></div><p>I redeployed jellyfin:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> docker-compose up -d jellyfin </span></span></code></pre></div><p>The Android app still had the fatal player error.</p> <h2 id="users-and-groups" class="relative group">Users and groups <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#users-and-groups" aria-label="Anchor">#</a></span></h2><p>Most of my Synology containers use the uid/gid pair of <code>1026:100</code> so allow them to read and write to my storage volume. The <code>/dev/dri/renderD128</code> is owned by the <code>videodriver</code> group:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> grep videodriver /etc/group </span></span><span class="line"><span class="cl"><span class="go">videodriver::937:PlexMediaServer </span></span></span></code></pre></div><p>This likely came from a time when I installed Plex on Synology via one of the Synology applications rather than from a container. <em>(I&rsquo;m not sure, but that&rsquo;s my guess.)</em></p> <p>I added that group to the container:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">jellyfin</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">docker.io/jellyfin/jellyfin:latest</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">container_name</span><span class="p">:</span><span class="w"> </span><span class="l">jellyfin</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">user</span><span class="p">:</span><span class="w"> </span><span class="m">1026</span><span class="p">:</span><span class="m">100</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">network_mode</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;host&#34;</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">group_add</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="s2">&#34;937&#34;</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">devices</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">/dev/dri/renderD128:/dev/dri/renderD128</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">/dev/dri/card0:/dev/dri/card0</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">volumes</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># removed</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">restart</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;unless-stopped&#34;</span><span class="w"> </span></span></span></code></pre></div><p>After redeploying the container, the Android app worked just fine! Also, the video stuttering disappeared when viewing the 4K video from the browser. πŸŽ‰</p>Redirect local ports with firewalldhttps://major.io/p/firewalld-port-redirection/Fri, 28 Jun 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/firewalld-port-redirection/<p>Linux networking and firewalls give us plenty of options for redirecting traffic from one port to another. We can allow people outside our home to reach a web server we run in our internal network. That&rsquo;s called destination NAT, ot <a href="https://en.wikipedia.org/wiki/Network_address_translation#DNAT" target="_blank" rel="noreferrer">DNAT</a>.</p> <p>You can also redirect traffic to different ports on the same host. For example, if you have a daemon listening on port 3000, but you want people to reach that service on port 80, you can redirect traffic from 80 to 3000 on the same host (without network address translation).</p> <p>But how do we do this with <a href="https://firewalld.org/" target="_blank" rel="noreferrer">firewalld</a>? πŸ€”</p> <h2 id="old-school-iptables-methods" class="relative group">Old-school iptables methods <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#old-school-iptables-methods" aria-label="Anchor">#</a></span></h2><p>Let&rsquo;s say you have a service running on port 3000 and you want to expose it to other computers on your same network as port 80. With iptables, you would typically start by enabling IP forwarding:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="go">sudo sysctl -w net.ipv4.ip_forward=1 </span></span></span></code></pre></div><p>Add two iptables rules to handle packets coming in from the outside as well as any locally generated packets:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">#</span> Handle locally-generated packets on the same machine. </span></span><span class="line"><span class="cl"><span class="go">sudo iptables -t nat -A PREROUTING -s 127.0.0.1 -p tcp --dport 80 -j REDIRECT --to 3000` </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">#</span> Handle packets coming from outside the current machine. </span></span><span class="line"><span class="cl"><span class="go">sudo iptables -t nat -A OUTPUT -s 127.0.0.1 -p tcp --dport 80 -j REDIRECT --to 3000` </span></span></span></code></pre></div><p>There&rsquo;s a weird situation that happens on certain machines with certain network configurations where packets are not properly routed when they are destined for the local network adapter. To fix that, set one more sysctl configuration:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="go">sudo sysctl -w net.ipv4.conf.all.route_localnet=1 </span></span></span></code></pre></div><p>Remember to make these sysctl configurations permanent:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="go">sudo mkdir /etc/sysctl.conf.d/ </span></span></span><span class="line"><span class="cl"><span class="go">echo &#34;net.ipv4.ip_forward=1&#34; | sudo tee &gt;&gt; /etc/sysctl.conf.d/redirect.conf </span></span></span><span class="line"><span class="cl"><span class="go">echo &#34;net.ipv4.conf.all.route_localnet&#34; | sudo tee &gt;&gt; /etc/sysctl.conf.d/redirect.conf </span></span></span></code></pre></div><h2 id="why-consider-firewalld" class="relative group">Why consider firewalld? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#why-consider-firewalld" aria-label="Anchor">#</a></span></h2><p>I like firewalld because I can manage lots of settings for different firewall zones and allow access from one zone to another. It also allows me to put certain interfaces in trusted zones so they automatically get more access.</p> <p>Another nice aspect about firewalld is that it supports iptables and nftables backends. You don&rsquo;t have to think about the differences between the backends. All of that is taken care of for you.</p> <h2 id="port-redirections-in-firewalld" class="relative group">Port redirections in firewalld <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#port-redirections-in-firewalld" aria-label="Anchor">#</a></span></h2><p>Let&rsquo;s start by checking our default firewalld zone:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo firewall-cmd --list-all </span></span><span class="line"><span class="cl"><span class="go">FedoraServer (default, active) </span></span></span><span class="line"><span class="cl"><span class="go"> target: default </span></span></span><span class="line"><span class="cl"><span class="go"> ingress-priority: 0 </span></span></span><span class="line"><span class="cl"><span class="go"> egress-priority: 0 </span></span></span><span class="line"><span class="cl"><span class="go"> icmp-block-inversion: no </span></span></span><span class="line"><span class="cl"><span class="go"> interfaces: bond0 eno1 eno2 </span></span></span><span class="line"><span class="cl"><span class="go"> sources: </span></span></span><span class="line"><span class="cl"><span class="go"> services: dhcpv6-client http https </span></span></span><span class="line"><span class="cl"><span class="go"> ports: 51820/udp </span></span></span><span class="line"><span class="cl"><span class="go"> protocols: </span></span></span><span class="line"><span class="cl"><span class="go"> forward: yes </span></span></span><span class="line"><span class="cl"><span class="go"> masquerade: yes </span></span></span><span class="line"><span class="cl"><span class="go"> forward-ports: </span></span></span><span class="line"><span class="cl"><span class="go"> source-ports: </span></span></span><span class="line"><span class="cl"><span class="go"> icmp-blocks: </span></span></span><span class="line"><span class="cl"><span class="go"> rich rules: </span></span></span></code></pre></div><p>This output shows that my external network interfaces are attached to the zone and forwarding is already on in my case. If you see <code>forward: no</code> here, just run this command:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo firewall-cmd --add-forward </span></span><span class="line"><span class="cl"><span class="go">success </span></span></span></code></pre></div><p>Now firewalld will manage your <code>forwarding</code> sysctl variables for you on these interfaces. That&rsquo;s handy. πŸ˜‰</p> <p>Next, let&rsquo;s get the redirect working. We want to take external packets on port 80 and send them to 3000: on the local machine.</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo firewall-cmd <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span><span class="go"> --add-forward-port=port=80:proto=tcp:toport=3000:toaddr=127.0.0.1 </span></span></span><span class="line"><span class="cl"><span class="go">success </span></span></span></code></pre></div><p>In this command, we told firewalld to take 80/tcp from the outside and send it to port 3000 on the local host (127.0.0.1). Let&rsquo;s double check our current configuration:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo firewall-cmd --list-all </span></span><span class="line"><span class="cl"><span class="go">FedoraServer (default, active) </span></span></span><span class="line"><span class="cl"><span class="go"> target: default </span></span></span><span class="line"><span class="cl"><span class="go"> ingress-priority: 0 </span></span></span><span class="line"><span class="cl"><span class="go"> egress-priority: 0 </span></span></span><span class="line"><span class="cl"><span class="go"> icmp-block-inversion: no </span></span></span><span class="line"><span class="cl"><span class="go"> interfaces: bond0 eno1 eno2 </span></span></span><span class="line"><span class="cl"><span class="go"> sources: </span></span></span><span class="line"><span class="cl"><span class="go"> services: dhcpv6-client http https </span></span></span><span class="line"><span class="cl"><span class="go"> ports: 51820/udp </span></span></span><span class="line"><span class="cl"><span class="go"> protocols: </span></span></span><span class="line"><span class="cl"><span class="go"> forward: yes </span></span></span><span class="line"><span class="cl"><span class="go"> masquerade: yes </span></span></span><span class="line"><span class="cl"><span class="go"> forward-ports: </span></span></span><span class="line"><span class="cl"><span class="go"> port=80:proto=tcp:toport=3000:toaddr=127.0.0.1 </span></span></span><span class="line"><span class="cl"><span class="go"> source-ports: </span></span></span><span class="line"><span class="cl"><span class="go"> icmp-blocks: </span></span></span><span class="line"><span class="cl"><span class="go"> rich rules: </span></span></span></code></pre></div><p>Test a connection to port 80 with <code>curl</code> and it should redirect to the service on port 3000.</p> <p>🚨 <strong>If everything works, remember to save the firewalld configuration:</strong></p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo firewall-cmd --runtime-to-permanent </span></span><span class="line"><span class="cl"><span class="go">success </span></span></span></code></pre></div><h2 id="extra-credit" class="relative group">Extra credit <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#extra-credit" aria-label="Anchor">#</a></span></h2><p>We can inspect the nftables rules to see the firewall rules that firewalld set for us. The <a href="https://wiki.archlinux.org/title/Nftables" target="_blank" rel="noreferrer">Arch Linux nftables wiki page</a> is superb for looking up those commands.</p> <p>If we dump the current ruleset, we see the rule we created in firewalld:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo nft list ruleset </span></span><span class="line"><span class="cl"><span class="go">---SNIP--- </span></span></span><span class="line"><span class="cl"><span class="go">chain nat_PRE_FedoraServer_allow { </span></span></span><span class="line"><span class="cl"><span class="go"> meta nfproto ipv4 tcp dport 80 dnat ip to 127.0.0.1:3000 </span></span></span><span class="line"><span class="cl"><span class="go">} </span></span></span><span class="line"><span class="cl"><span class="go">---SNIP--- </span></span></span></code></pre></div>amazon-ec2-utils in Fedorahttps://major.io/p/amazon-ec2-utils-fedora/Wed, 08 May 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/amazon-ec2-utils-fedora/<p>We&rsquo;ve all been in that situation where we see a device in Linux and wonder which physical device it corresponds to. I remember when I built my first NAS and received an alert that a drive had failed. It took me a while to figure out which physical drive actually needed to be replaced.</p> <p>This happens with network devices, too, and I <a href="https://major.io/p/understanding-systemds-predictable-network-device-names/">wrote a post</a> about systemd&rsquo;s predictable network device names back in 2015.</p> <p>Cloud instances often make it even more confusing because storage devices are fully virtualized and show up differently depending on the cloud provider. I recently packaged <a href="https://github.com/amazonlinux/amazon-ec2-utils" target="_blank" rel="noreferrer">amazon-ec2-utils</a> in Fedora to make this a little easier on AWS.</p> <h2 id="the-problem" class="relative group">The problem <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#the-problem" aria-label="Anchor">#</a></span></h2><p>I just built a test instance of Fedora 40 in AWS and the AWS API shows the block device mappings like this:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> aws ec2 describe-instances <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span><span class="go"> --instance-ids i-0687448a184ab0a9e | \ </span></span></span><span class="line"><span class="cl"><span class="go"> jq &#39;.Reservations[0].Instances[0].BlockDeviceMappings&#39; </span></span></span><span class="line"><span class="cl"><span class="go">[ </span></span></span><span class="line"><span class="cl"><span class="go"> { </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;DeviceName&#34;: &#34;/dev/sda1&#34;, </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;Ebs&#34;: { </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;AttachTime&#34;: &#34;2024-05-08T15:24:03+00:00&#34;, </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;DeleteOnTermination&#34;: true, </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;Status&#34;: &#34;attached&#34;, </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;VolumeId&#34;: &#34;vol-0832569729b6c5ea6&#34; </span></span></span><span class="line"><span class="cl"><span class="go"> } </span></span></span><span class="line"><span class="cl"><span class="go"> } </span></span></span><span class="line"><span class="cl"><span class="go">] </span></span></span></code></pre></div><p>However, if I check these devices inside the instance itself, I get something totally different:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">[fedora@f40 ~]$</span> sudo fdisk -l </span></span><span class="line"><span class="cl"><span class="go">Disk /dev/nvme0n1: 10 GiB, 10737418240 bytes, 20971520 sectors </span></span></span><span class="line"><span class="cl"><span class="go">Disk model: Amazon Elastic Block Store </span></span></span><span class="line"><span class="cl"><span class="go">Units: sectors of 1 * 512 = 512 bytes </span></span></span><span class="line"><span class="cl"><span class="go">Sector size (logical/physical): 512 bytes / 512 bytes </span></span></span><span class="line"><span class="cl"><span class="go">I/O size (minimum/optimal): 4096 bytes / 4096 bytes </span></span></span><span class="line"><span class="cl"><span class="go">Disklabel type: gpt </span></span></span><span class="line"><span class="cl"><span class="go">Disk identifier: 9FB58ED7-7581-4469-BEB7-64F069151EAF </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go">Device Start End Sectors Size Type </span></span></span><span class="line"><span class="cl"><span class="go">/dev/nvme0n1p1 2048 206847 204800 100M EFI System </span></span></span><span class="line"><span class="cl"><span class="go">/dev/nvme0n1p2 206848 2254847 2048000 1000M Linux extended boot </span></span></span><span class="line"><span class="cl"><span class="go">/dev/nvme0n1p3 2254848 20971484 18716637 8.9G Linux root (ARM-64) </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go">Disk /dev/zram0: 1.78 GiB, 1909456896 bytes, 466176 sectors </span></span></span><span class="line"><span class="cl"><span class="go">Units: sectors of 1 * 4096 = 4096 bytes </span></span></span><span class="line"><span class="cl"><span class="go">Sector size (logical/physical): 4096 bytes / 4096 bytes </span></span></span><span class="line"><span class="cl"><span class="go">I/O size (minimum/optimal): 4096 bytes / 4096 bytes </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">[fedora@f40 ~]$</span> ls -al /dev/sd* </span></span><span class="line"><span class="cl"><span class="go">ls: cannot access &#39;/dev/sd*&#39;: No such file or directory </span></span></span></code></pre></div><p>One disk isn&rsquo;t so bad, but what if we add more storage? The API tells me one thing:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">&gt;</span> aws ec2 describe-instances --instance-ids i-0687448a184ab0a9e <span class="p">|</span> jq <span class="s1">&#39;.Reservations[0].Instances[0].BlockDeviceMappings&#39;</span> </span></span><span class="line"><span class="cl"><span class="go">[ </span></span></span><span class="line"><span class="cl"><span class="go"> { </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;DeviceName&#34;: &#34;/dev/sda1&#34;, </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;Ebs&#34;: { </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;AttachTime&#34;: &#34;2024-05-08T15:24:03+00:00&#34;, </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;DeleteOnTermination&#34;: true, </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;Status&#34;: &#34;attached&#34;, </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;VolumeId&#34;: &#34;vol-0832569729b6c5ea6&#34; </span></span></span><span class="line"><span class="cl"><span class="go"> } </span></span></span><span class="line"><span class="cl"><span class="go"> }, </span></span></span><span class="line"><span class="cl"><span class="go"> { </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;DeviceName&#34;: &#34;/dev/sde&#34;, </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;Ebs&#34;: { </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;AttachTime&#34;: &#34;2024-05-08T15:38:29.754000+00:00&#34;, </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;DeleteOnTermination&#34;: false, </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;Status&#34;: &#34;attached&#34;, </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;VolumeId&#34;: &#34;vol-0a7ba05c5270d7aa3&#34;, </span></span></span><span class="line"><span class="cl"><span class="go"> &#34;VolumeOwnerId&#34;: &#34;xxx&#34; </span></span></span><span class="line"><span class="cl"><span class="go"> } </span></span></span><span class="line"><span class="cl"><span class="go"> } </span></span></span><span class="line"><span class="cl"><span class="go">] </span></span></span></code></pre></div><p>But then the instance tells me something else entirely:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">[fedora@f40 ~]$</span> sudo fdisk -l </span></span><span class="line"><span class="cl"><span class="go">Disk /dev/nvme0n1: 10 GiB, 10737418240 bytes, 20971520 sectors </span></span></span><span class="line"><span class="cl"><span class="go">Disk model: Amazon Elastic Block Store </span></span></span><span class="line"><span class="cl"><span class="go">Units: sectors of 1 * 512 = 512 bytes </span></span></span><span class="line"><span class="cl"><span class="go">Sector size (logical/physical): 512 bytes / 512 bytes </span></span></span><span class="line"><span class="cl"><span class="go">I/O size (minimum/optimal): 4096 bytes / 4096 bytes </span></span></span><span class="line"><span class="cl"><span class="go">Disklabel type: gpt </span></span></span><span class="line"><span class="cl"><span class="go">Disk identifier: 9FB58ED7-7581-4469-BEB7-64F069151EAF </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go">Device Start End Sectors Size Type </span></span></span><span class="line"><span class="cl"><span class="go">/dev/nvme0n1p1 2048 206847 204800 100M EFI System </span></span></span><span class="line"><span class="cl"><span class="go">/dev/nvme0n1p2 206848 2254847 2048000 1000M Linux extended boot </span></span></span><span class="line"><span class="cl"><span class="go">/dev/nvme0n1p3 2254848 20971484 18716637 8.9G Linux root (ARM-64) </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go">Disk /dev/zram0: 1.78 GiB, 1909456896 bytes, 466176 sectors </span></span></span><span class="line"><span class="cl"><span class="go">Units: sectors of 1 * 4096 = 4096 bytes </span></span></span><span class="line"><span class="cl"><span class="go">Sector size (logical/physical): 4096 bytes / 4096 bytes </span></span></span><span class="line"><span class="cl"><span class="go">I/O size (minimum/optimal): 4096 bytes / 4096 bytes </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go">Disk /dev/nvme1n1: 10 GiB, 10737418240 bytes, 20971520 sectors </span></span></span><span class="line"><span class="cl"><span class="go">Disk model: Amazon Elastic Block Store </span></span></span><span class="line"><span class="cl"><span class="go">Units: sectors of 1 * 512 = 512 bytes </span></span></span><span class="line"><span class="cl"><span class="go">Sector size (logical/physical): 512 bytes / 512 bytes </span></span></span><span class="line"><span class="cl"><span class="go">I/O size (minimum/optimal): 4096 bytes / 4096 bytes </span></span></span></code></pre></div><h2 id="udev-rules-to-the-rescue" class="relative group">udev rules to the rescue <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#udev-rules-to-the-rescue" aria-label="Anchor">#</a></span></h2><p>The amazon-ec2-utils package provides some helpful udev rules and scripts to make it easier to identify these devices. This package is on the way to Fedora as I write this post, but it hasn&rsquo;t reached the stable repos yet. Once it does, you should be able to install it:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo dnf install amazon-ec2-utils </span></span></code></pre></div><p>In the meantime, you can download the latest build and install it on your instance:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo dnf install /usr/bin/koji </span></span><span class="line"><span class="cl"><span class="gp">$</span> koji download-build amazon-ec2-utils-2.2.0-2.fc40 </span></span><span class="line"><span class="cl"><span class="go">Downloading [1/2]: amazon-ec2-utils-2.2.0-2.fc40.src.rpm </span></span></span><span class="line"><span class="cl"><span class="go">[====================================] 100% 24.01 KiB / 24.01 KiB </span></span></span><span class="line"><span class="cl"><span class="go">Downloading [2/2]: amazon-ec2-utils-2.2.0-2.fc40.noarch.rpm </span></span></span><span class="line"><span class="cl"><span class="go">[====================================] 100% 20.53 KiB / 20.53 KiB </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="gp">$</span> sudo dnf install amazon-ec2-utils-2.2.0-2.fc40.noarch.rpm </span></span></code></pre></div><p>The cleanest method to get these new udev rules working is to reboot, but if you&rsquo;re in a hurry, there&rsquo;s an option to reload these rules without a reboot:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo udevadm control --reload-rules </span></span><span class="line"><span class="cl"><span class="gp">$</span> sudo udevadm trigger </span></span></code></pre></div><p>What do we have in <code>/dev/</code> now?</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">[fedora@f40 ~]$</span> ls -al /dev/sd* </span></span><span class="line"><span class="cl"><span class="go">lrwxrwxrwx. 1 root root 7 May 8 15:44 /dev/sda1 -&gt; nvme0n1 </span></span></span><span class="line"><span class="cl"><span class="go">lrwxrwxrwx. 1 root root 9 May 8 15:44 /dev/sda11 -&gt; nvme0n1p1 </span></span></span><span class="line"><span class="cl"><span class="go">lrwxrwxrwx. 1 root root 9 May 8 15:44 /dev/sda12 -&gt; nvme0n1p2 </span></span></span><span class="line"><span class="cl"><span class="go">lrwxrwxrwx. 1 root root 9 May 8 15:44 /dev/sda13 -&gt; nvme0n1p3 </span></span></span><span class="line"><span class="cl"><span class="go">lrwxrwxrwx. 1 root root 7 May 8 15:44 /dev/sde -&gt; nvme1n1 </span></span></span></code></pre></div><p>We can put a filesystem down on the new device using the same name as the API presents:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo dnf install /usr/sbin/mkfs.btrfs </span></span><span class="line"><span class="cl"><span class="gp">$</span> sudo mkfs.btrfs /dev/sde </span></span><span class="line"><span class="cl"><span class="go">btrfs-progs v6.8.1 </span></span></span><span class="line"><span class="cl"><span class="go">See https://btrfs.readthedocs.io for more information. </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go">Performing full device TRIM /dev/sde (10.00GiB) ... </span></span></span><span class="line"><span class="cl"><span class="go">NOTE: several default settings have changed in version 5.15, please make sure </span></span></span><span class="line"><span class="cl"><span class="go"> this does not affect your deployments: </span></span></span><span class="line"><span class="cl"><span class="go"> - DUP for metadata (-m dup) </span></span></span><span class="line"><span class="cl"><span class="go"> - enabled no-holes (-O no-holes) </span></span></span><span class="line"><span class="cl"><span class="go"> - enabled free-space-tree (-R free-space-tree) </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go">Label: (null) </span></span></span><span class="line"><span class="cl"><span class="go">UUID: c2fb9e33-3bf6-4b5b-aa80-44e315f499de </span></span></span><span class="line"><span class="cl"><span class="go">Node size: 16384 </span></span></span><span class="line"><span class="cl"><span class="go">Sector size: 4096 (CPU page size: 4096) </span></span></span><span class="line"><span class="cl"><span class="go">Filesystem size: 10.00GiB </span></span></span><span class="line"><span class="cl"><span class="go">Block group profiles: </span></span></span><span class="line"><span class="cl"><span class="go"> Data: single 8.00MiB </span></span></span><span class="line"><span class="cl"><span class="go"> Metadata: DUP 256.00MiB </span></span></span><span class="line"><span class="cl"><span class="go"> System: DUP 8.00MiB </span></span></span><span class="line"><span class="cl"><span class="go">SSD detected: yes </span></span></span><span class="line"><span class="cl"><span class="go">Zoned device: no </span></span></span><span class="line"><span class="cl"><span class="go">Features: extref, skinny-metadata, no-holes, free-space-tree </span></span></span><span class="line"><span class="cl"><span class="go">Checksum: crc32c </span></span></span><span class="line"><span class="cl"><span class="go">Number of devices: 1 </span></span></span><span class="line"><span class="cl"><span class="go">Devices: </span></span></span><span class="line"><span class="cl"><span class="go"> ID SIZE PATH </span></span></span><span class="line"><span class="cl"><span class="go"> 1 10.00GiB /dev/sde </span></span></span></code></pre></div><p>Being able to know these device names during the instance launch or during storage operations makes it much easier to write automation for these devices. There&rsquo;s no guess work required to translate the device that an instance shows you to what you see via the API.</p>Fix big cursors in Java applications in Waylandhttps://major.io/p/java-big-cursors-wayland/Fri, 26 Apr 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/java-big-cursors-wayland/<p>Scroll through the list of <a href="https://major.io/tags/wayland/">Wayland posts</a> posts on the blog and you&rsquo;ll see that I&rsquo;ve solved plenty of weird problems with Wayland and the <a href="https://swaywm.org/" target="_blank" rel="noreferrer">Sway</a> compositor. Most are pretty easy to fix but some are a bit trickier.</p> <p>Java applications are notoriously unpredictable and Wayland takes unpredictability to the next level. One particular application on my desktop always seems to start with massive cursors.</p> <p>This post is about how I fixed and then discovered something interesting along the way.</p> <h2 id="fixing-big-cursors" class="relative group">Fixing big cursors <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#fixing-big-cursors" aria-label="Anchor">#</a></span></h2><p>I recently moved some investment and trading accounts from TD Ameritrade to <a href="https://tastytrade.com/" target="_blank" rel="noreferrer">Tastytrade</a>. Both offer Java applications that make trading easier, but Tastytrade&rsquo;s application always started with massive cursors.</p> <p>To make matters worse, sometimes the cursor looked lined up on the screen but then the click landed on the wrong buttons in the application! Errors are annoying. Errors that cost you money and time must be fixed. 😜</p> <p>Some web searches eventually led me to Arch Linux&rsquo;s excellent <a href="https://wiki.archlinux.org/title/Wayland" target="_blank" rel="noreferrer">Wayland wiki page</a>. None of the adjustments or environment variables there had any effect on my cursors.</p> <p>I eventually landed on a page that suggested setting <code>XCURSOR_SIZE</code>. I don&rsquo;t remember ever setting that, but it was being set by <em>something</em>:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> <span class="nb">echo</span> <span class="nv">$XCURSOR_SIZE</span> </span></span><span class="line"><span class="cl"><span class="go">24 </span></span></span></code></pre></div><p>One of the suggestions was to decrease it, so I decided to give <code>20</code> a try. That was too big, but <code>16</code> was perfect and it matched all of my other applications:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> <span class="nb">export</span> <span class="nv">XCURSOR_SIZE</span><span class="o">=</span><span class="m">20</span> </span></span><span class="line"><span class="cl"><span class="gp">#</span> /opt/tastytrade/bin/tastytrade </span></span></code></pre></div><p>That works fine when I start my application via the terminal, but how do I set it for the application when I start it from ulauncher in sway? πŸ€”</p> <h2 id="desktop-file" class="relative group">Desktop file <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#desktop-file" aria-label="Anchor">#</a></span></h2><p>The Tastytade RPM comes with a <code>.desktop</code> file for launching the application. I copied that over to my local applications directory:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl">cp /opt/tastytrade/lib/tastytrade-tastytrade.desktop <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> ~/.local/share/applications/ </span></span></code></pre></div><p>Then I opened the copied <code>~/.local/share/applications/tastytrade-tastytrade.desktop</code> file in a text editor:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="k">[Desktop Entry]</span> </span></span><span class="line"><span class="cl"><span class="na">Name</span><span class="o">=</span><span class="s">tastytrade</span> </span></span><span class="line"><span class="cl"><span class="na">Comment</span><span class="o">=</span><span class="s">tastytrade</span> </span></span><span class="line"><span class="cl"><span class="na">Exec</span><span class="o">=</span><span class="s">/opt/tastytrade/bin/tastytrade</span> </span></span><span class="line"><span class="cl"><span class="na">Icon</span><span class="o">=</span><span class="s">/opt/tastytrade/lib/tastytrade.png</span> </span></span><span class="line"><span class="cl"><span class="na">Terminal</span><span class="o">=</span><span class="s">false</span> </span></span><span class="line"><span class="cl"><span class="na">Type</span><span class="o">=</span><span class="s">Application</span> </span></span><span class="line"><span class="cl"><span class="na">Categories</span><span class="o">=</span><span class="s">tastyworks</span> </span></span><span class="line"><span class="cl"><span class="na">MimeType</span><span class="o">=</span> </span></span></code></pre></div><p>I changed the <code>Exec</code> line to be:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="na">Exec</span><span class="o">=</span><span class="s">env XCURSOR_SIZE=16 /opt/tastytrade/bin/tastytrade</span> </span></span></code></pre></div><p>I launched the application again after making that change, but the cursors were still huge! There has to be another way. πŸ€”</p> <h2 id="systemd-does-everything-" class="relative group">systemd does everything πŸ˜† <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#systemd-does-everything-" aria-label="Anchor">#</a></span></h2><p>After more searching and digging, I discovered that systemd has a capability to <a href="https://www.freedesktop.org/software/systemd/man/latest/environment.d.html" target="_blank" rel="noreferrer">set environment variables for user sessions</a>:</p> <blockquote> <p>Configuration files in the environment.d/ directories contain lists of environment variable assignments passed to services started by the systemd user instance. systemd-environment-d-generator(8) parses them and updates the environment exported by the systemd user instance. See below for an discussion of which processes inherit those variables.</p> <p>It is recommended to use numerical prefixes for file names to simplify ordering.</p> <p>For backwards compatibility, a symlink to /etc/environment is installed, so this file is also parsed.</p> </blockquote> <p>Let&rsquo;s give that a try:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> mkdir -p ~/.config/environment.d/ </span></span><span class="line"><span class="cl"><span class="gp">$</span> vim ~/.config/environment.d/wayland.conf </span></span></code></pre></div><p>In the file, I added one line with a comment (because you will soon forget why you added it πŸ˜„):</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl"><span class="c1"># Fix big cursors in Java apps in Wayland</span> </span></span><span class="line"><span class="cl"><span class="nv">XCURSOR_SIZE</span><span class="o">=</span><span class="m">16</span> </span></span></code></pre></div><p><strong>After a reboot, I launched my Java application and boom &ndash; the cursors were perfect!</strong> πŸŽ‰</p> <p>I went back and cleaned up some other hacks I had applied and added them to that <code>wayland.conf</code> file:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl"><span class="c1"># This was important at some point but I&#39;m afraid to remove it.</span> </span></span><span class="line"><span class="cl"><span class="c1"># Note to self: make detailed comments when adding lines here.</span> </span></span><span class="line"><span class="cl"><span class="nv">SDL_VIDEODRIVER</span><span class="o">=</span>wayland </span></span><span class="line"><span class="cl"><span class="nv">QT_QPA_PLATFORM</span><span class="o">=</span>wayland </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># Reduce window decorations for VLC</span> </span></span><span class="line"><span class="cl"><span class="nv">QT_WAYLAND_DISABLE_WINDOWDECORATION</span><span class="o">=</span><span class="s2">&#34;1&#34;</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># Fix weird window handling when Java apps do certain pop-ups</span> </span></span><span class="line"><span class="cl"><span class="nv">_JAVA_AWT_WM_NONREPARENTING</span><span class="o">=</span><span class="m">1</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># Ensure Firefox is using Wayland code (not needed any more)</span> </span></span><span class="line"><span class="cl"><span class="nv">MOZ_ENABLE_WAYLAND</span><span class="o">=</span><span class="m">1</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># Disable HiDPI</span> </span></span><span class="line"><span class="cl"><span class="nv">GDK_SCALE</span><span class="o">=</span><span class="m">1</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># Fix big cursors in Java apps in Wayland</span> </span></span><span class="line"><span class="cl"><span class="nv">XCURSOR_SIZE</span><span class="o">=</span><span class="m">16</span> </span></span></code></pre></div><p>I&rsquo;m told there are some caveats with this solution, especially if your Wayland desktop doesn&rsquo;t use systemd to start. This is working for me with GDM launching Sway on Fedora 40.</p>cloud-init and dhcpcdhttps://major.io/p/fedora-cloud-init-dhcpcd/Thu, 18 Apr 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/fedora-cloud-init-dhcpcd/<p>We&rsquo;re all familiar with the trusty old <code>dhclient</code> on our Linux systems, but <a href="https://github.com/isc-projects/dhcp" target="_blank" rel="noreferrer">it went end-of-life in 2022</a>:</p> <pre tabindex="0"><code>NOTE: This software is now End-Of-Life. 4.4.3 is the final release planned. We will continue to keep the public issue tracker and user mailing list open. You should read this file carefully before trying to install or use the ISC DHCP Distribution. </code></pre><p>Most Linux distributions use <code>dhclient</code> along with cloud-init for the initial dhcp request during the first part of cloud-init&rsquo;s work. I set off to switch Fedora&rsquo;s cloud-init package to <code>dhcpcd</code> instead.</p> <h2 id="whats-new-with-dhcpcd" class="relative group">What&rsquo;s new with dhcpcd? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#whats-new-with-dhcpcd" aria-label="Anchor">#</a></span></h2><p>There are some nice things about <code>dhcpcd</code> that you can find in the <a href="https://github.com/NetworkConfiguration/dhcpcd" target="_blank" rel="noreferrer">GitHub repository</a>:</p> <ul> <li>Very small footprint with almost no dependencies on Fedora</li> <li>It can do DHCP and DHCPv6</li> <li>It can also be a <a href="https://en.wikipedia.org/wiki/Zeroconf" target="_blank" rel="noreferrer">ZeroConf</a> client</li> </ul> <p>The project had its last release back in December 2023 and had commits as recently as this week.</p> <h2 id="but-i-use-networkmanager" class="relative group">But I use NetworkManager <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#but-i-use-networkmanager" aria-label="Anchor">#</a></span></h2><p>That&rsquo;s great! A switch from <code>dhclient</code> to <code>dhcpcd</code> for cloud-init won&rsquo;t affect you.</p> <p>When cloud-init starts, it does an initial dhcp request to get just enough networking to reach the cloud&rsquo;s metadata service. This service provides all kinds of information for cloud-init, including network setup instructions and initial scripts to run.</p> <p>NetworkManager doesn&rsquo;t start taking action until cloud-init has written the network configuration to the system.</p> <h2 id="but-i-use-systemd-networkd" class="relative group">But I use systemd-networkd <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#but-i-use-systemd-networkd" aria-label="Anchor">#</a></span></h2><p>Same as with NetworkManager, this change applies to the <em>very</em> early boot and you won&rsquo;t notice a different when deploying new cloud systems.</p> <h2 id="how-can-i-get-it-right-now" class="relative group">How can I get it right now? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#how-can-i-get-it-right-now" aria-label="Anchor">#</a></span></h2><p>If you&rsquo;re using a recent build of Fedora rawhide (the unstable release under development), you likely have it right now on your cloud instance. Just run <code>journalctl --boot</code>, search for <code>dhcpcd</code>, and you should see these lines:</p> <pre tabindex="0"><code>cloud-init[725]: Cloud-init v. 24.1.4 running &#39;init-local&#39; at Wed, 17 Apr 2024 14:39:36 +0000. Up 6.13 seconds. dhcpcd[727]: dhcpcd-10.0.6 starting kernel: 8021q: 802.1Q VLAN Support v1.8 dhcpcd[730]: DUID 00:01:00:01:2d:b2:9b:a9:06:eb:18:e7:22:dd dhcpcd[730]: eth0: IAID 18:e7:22:dd dhcpcd[730]: eth0: soliciting a DHCP lease dhcpcd[730]: eth0: offered 172.31.26.195 from 172.31.16.1 dhcpcd[730]: eth0: leased 172.31.26.195 for 3600 seconds avahi-daemon[706]: Joining mDNS multicast group on interface eth0.IPv4 with address 172.31.26.195. avahi-daemon[706]: New relevant interface eth0.IPv4 for mDNS. avahi-daemon[706]: Registering new address record for 172.31.26.195 on eth0.IPv4. dhcpcd[730]: eth0: adding route to 172.31.16.0/20 dhcpcd[730]: eth0: adding default route via 172.31.16.1 dhcpcd[730]: control command: /usr/sbin/dhcpcd --dumplease --ipv4only eth0 </code></pre><p>There&rsquo;s also an <a href="https://bodhi.fedoraproject.org/updates/FEDORA-2024-51d7f6b005" target="_blank" rel="noreferrer">update pending for Fedora 40</a>, but it&rsquo;s currently held up by the beta freeze. That should appear as an update as soon as Fedora 40 is released.</p> <p>Keep in mind that if you have a system deployed already, cloud-init won&rsquo;t need to run again. Updating to Fedora 40 will update your cloud-init and pull in <code>dhcpcd</code>, but it won&rsquo;t need to run again since your configuration is already set.</p>Texas Linux Fest 2024 recap 🀠https://major.io/p/texas-linux-fest-2024-recap/Tue, 16 Apr 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/texas-linux-fest-2024-recap/<p>The 2024 <a href="https://2024.texaslinuxfest.org/" target="_blank" rel="noreferrer">Texas Linux Festival</a> just ended last weekend and it was a fun event as always. It&rsquo;s one my favorite events to attend because it&rsquo;s really casual. You have plenty of opportunities to see old friends, meet new people, and learn a few things along the way.</p> <p>I was fortunate enough to have two talks accepted for this year&rsquo;s event. One was focused on containers while the other was a (very belated) addition to my <a href="https://major.io/p/impostor-syndrome-talk-faqs-and-follow-ups/">impostor syndrome talk</a> from 2015.</p> <p>This was also my first time building slides with <a href="https://github.com/webpro/reveal-md" target="_blank" rel="noreferrer">reveal-md</a>, a &ldquo;batteries included&rdquo; package for making <a href="https://revealjs.com/" target="_blank" rel="noreferrer">reveal.js</a> slides. Nothing broke too badly and that was a relief.</p> <h2 id="containers-talk" class="relative group">Containers talk <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#containers-talk" aria-label="Anchor">#</a></span></h2><p>I&rsquo;ve wanted to share more of what I&rsquo;ve done with CoreOS in low-budget container deployments and this seemed like a good time to share it with the world out loud. My talk, <a href="https://txlf24-containers.major.io/#/" target="_blank" rel="noreferrer">Automated container updates with GitHub and CoreOS</a>, walked the audience through how to deploy containers on CoreOS, keep them updated, and update the container image source.</p> <p>My goal was to keep it as low on budget as possible. Much of it was centered around a stack of <a href="https://major.io/p/caddy-porkbun/">caddy</a>, librespeed, and docker-compose. All of it was kept up to date with <a href="https://major.io/p/watchtower/">watchtower</a>.</p> <p>My custom Caddy container needed support for <a href="https://porkbun.com/" target="_blank" rel="noreferrer">Porkbun&rsquo;s</a> DNS API and I used GitHub Actions to build that container and serve it to the internet using GitHub&rsquo;s package hosting. <em>This also gave me the opportunity to share how awesome Porkbun is for registering domains, including their <a href="https://porkbun.com/tld/jobs" target="_blank" rel="noreferrer">customized pig artwork</a> for every TLD imaginable.</em> 🐷</p> <p>We had a great discussion afterwards about how CoreOS <strong>does indeed live on</strong> as <a href="https://fedoraproject.org/coreos/" target="_blank" rel="noreferrer">Fedora CoreOS</a>.</p> <h2 id="tech-career-talk" class="relative group">Tech career talk <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#tech-career-talk" aria-label="Anchor">#</a></span></h2><p>This talk made me nervous because it had a lot of slides to cover, but I also wanted to leave plenty of time for questions. <a href="https://txlf24-tech-career.major.io/#/" target="_blank" rel="noreferrer">Five tips for a thriving technology career</a> built upon my old impostor syndrome talk by sharing some of the things I&rsquo;ve learned over the year that helped me succeed in my career.</p> <p>I managed to end early with time for questions, and boy did the audience have questions! πŸ“£ Some audience members helped me answer some questions, too!</p> <p>We talked a lot about office politics, tribal knowledge, and toxic workplaces. The audience generally agreed that most businesses tried to rub copious amounts of Confluence on their tribal knowledge problem, but it never improved. 😜</p> <p>The room was full with people standing in the back and I&rsquo;m tremendously humbled by everyone who came. I received plenty of feedback afterwards and that&rsquo;s the best gift I could ever get. 🎁</p> <h2 id="other-talks" class="relative group">Other talks <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#other-talks" aria-label="Anchor">#</a></span></h2><p><a href="https://github.com/anitazha" target="_blank" rel="noreferrer">Anita Zhang</a> had an excellent keynote talk on the second day about her unusual path into the world of technology. Her slides were pictures of her dog that lined up with various parts of her story. That was a great idea.</p> <p><a href="https://www.linkedin.com/in/kyle-davis-linux/?originalSubdomain=ca" target="_blank" rel="noreferrer">Kyle Davis</a> offered talks on <a href="https://github.com/valkey-io/valkey" target="_blank" rel="noreferrer">valkey</a> and <a href="https://github.com/bottlerocket-os/bottlerocket" target="_blank" rel="noreferrer">bottlerocket</a>. There was plenty about the redis and valkey story that I didn&rsquo;t know and the context was useful. It looks like you can simply drop valkey into most redis environments without much disruption.</p> <p><a href="https://www.linkedin.com/in/thomascameron/" target="_blank" rel="noreferrer">Thomas Cameron</a> talked about running OKD on Fedora CoreOS in his home lab. There were quite a few steps, but he did a great job of connecting the dots between what needed to be done and why.</p> <h2 id="around-the-exhibit-hall" class="relative group">Around the exhibit hall <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#around-the-exhibit-hall" aria-label="Anchor">#</a></span></h2><p>I helped staff the Fedora/CoreOS booth and we had plenty of questions. Most questions were around the M1 Macbook running <a href="https://asahilinux.org/" target="_blank" rel="noreferrer">Asahi Linux</a> that was on the table. πŸ˜‰</p> <p>There were still quite a few misconceptions around the CentOS Stream changes, as well as how AlmaLinux and Rocky Linux fit into the picture. Our booth was right next to the AlmaLinux booth and I had the opportunity to meet <a href="https://jonathanspw.com/about/" target="_blank" rel="noreferrer">Jonathan Wright</a>. That was awesome!</p> <p><strong>I can&rsquo;t wait for next year&rsquo;s event.</strong></p>Roll your own static blog analyticshttps://major.io/p/static-blog-analytics/Thu, 04 Apr 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/static-blog-analytics/<p>Static blogs come with tons of advantages. They&rsquo;re cheap to serve. You store all your changes in git. People with spotty internet connections can clone your blog and run it locally.</p> <p><strong>However, one of the challenges that I&rsquo;ve run into over the years is around analytics.</strong></p> <p>I could quickly add Google Analytics to the site and call it a day, but is that a good idea? Many browsers have ad blocking these days and the analytics wouldn&rsquo;t even run. For those that don&rsquo;t have an ad blocker, do I want to send more data about them to Google? πŸ™ƒ</p> <p>How about running my own self-hosted analytics platform? That&rsquo;s pretty easy with containers, but most ad blockers know about those, too.</p> <p>This post talks about how to host a static blog in a container behind a <a href="https://caddyserver.com/" target="_blank" rel="noreferrer">Caddy</a> web server. We will use <a href="https://goaccess.io/" target="_blank" rel="noreferrer">goaccess</a> to analyze the log files on the server itself to avoid dragging in an analytics platform.</p> <h2 id="why-do-you-need-analytics" class="relative group">Why do you need analytics? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#why-do-you-need-analytics" aria-label="Anchor">#</a></span></h2><p>Yes, yes, I know this comes from the guy who wrote a post about <a href="https://major.io/p/how-i-write-blog-posts/">writing for yourself</a>, but sometimes I like to know which posts are popular with other people. I also like to know if something&rsquo;s misconfigured and visitors are seeing 404 errors for pages which should be working.</p> <p>It can also be handy to know when someone else is <a href="https://major.io/p/puppy-linux-icanhazip-and-tin-foil-hats/">writing about you</a>, especially when those things are incorrect. πŸ˜‰</p> <p>So my goals here are these:</p> <ul> <li>Get some basic data on what&rsquo;s resonating with people and what isn&rsquo;t</li> <li>Find configuration errors that are leading visitors to error pages</li> <li>Learn more about who is linking to the site</li> <li>Do all this without impacting user privacy through heavy javascript trackers</li> </ul> <h2 id="what-are-the-ingredients" class="relative group">What are the ingredients? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#what-are-the-ingredients" aria-label="Anchor">#</a></span></h2><p>There are three main pieces:</p> <ol> <li>Caddy, a small web server that runs really well in containers</li> <li>This blog, which is written with <a href="https://gohugo.io/" target="_blank" rel="noreferrer">Hugo</a> and <a href="https://github.com/major/major.io" target="_blank" rel="noreferrer">stored in GitHub</a></li> <li>Goaccess, a log analyzer with a capability to do live updates via websockets</li> </ol> <p>Caddy will write logs to a location that goaccess can read. In turn, goaccess will write log analysis to an HTML file that caddy can serve. The HTML file served by caddy will open a websocket to goaccess for live analytics.</p> <h2 id="a-static-blog-in-a-container" class="relative group">A static blog in a container? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#a-static-blog-in-a-container" aria-label="Anchor">#</a></span></h2><p>We can pack a static blog into a very thin container with an extremely lightweight web server. After all, caddy can handle automatic TLS certificate installation, logging, and caching. That just means we need the most basic webserver in the container itself.</p> <p>I was considering a second caddy container with the blog content in it until I stumbled upon a great post by Florin Lipan about <a href="https://lipanski.com/posts/smallest-docker-image-static-website" target="_blank" rel="noreferrer">The smallest Docker image to serve static websites</a>. He went down a rabbit hole to make the smallest possible web server container with busybox.</p> <p>His first stop led to a 1.25MB container, and that&rsquo;s tiny enough for me.<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> 🀏</p> <p>I built a <a href="https://github.com/major/major.io/blob/main/.github/workflows/container.yml" target="_blank" rel="noreferrer">container workflow</a> in GitHub Actions that builds a container, puts the blog in it, and <a href="https://github.com/major/major.io/pkgs/container/major.io" target="_blank" rel="noreferrer">stores that container as a package</a> in the GitHub repository. It all starts with a brief Dockerfile:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Dockerfile" data-lang="Dockerfile"><span class="line"><span class="cl"><span class="k">FROM</span><span class="s"> docker.io/library/busybox:1.36.1</span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">RUN</span> adduser -D static<span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">USER</span><span class="s"> static</span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">WORKDIR</span><span class="s"> /home/static</span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">COPY</span> ./public/ /home/static<span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">CMD</span> <span class="p">[</span><span class="s2">&#34;busybox&#34;</span><span class="p">,</span> <span class="s2">&#34;httpd&#34;</span><span class="p">,</span> <span class="s2">&#34;-f&#34;</span><span class="p">,</span> <span class="s2">&#34;-p&#34;</span><span class="p">,</span> <span class="s2">&#34;3000&#34;</span><span class="p">]</span><span class="err"> </span></span></span></code></pre></div><p>We start with busybox, add a user, put the website content into the user&rsquo;s home directory, and start busybox&rsquo;s <code>httpd</code> server. The container starts up and serves the static content on port 3000.</p> <h2 id="caddy-logs" class="relative group">Caddy logs <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#caddy-logs" aria-label="Anchor">#</a></span></h2><p>Caddy writes its logs in a JSON format and goaccess already knows how to parse caddy logs. Our first step is to get caddy writing some logs. In my case, I have a directory called <code>caddy/logs/</code> in my home directory where those logs are written.</p> <p>I&rsquo;ll mount the log storage into the caddy container and mount one extra directory to hold the HTML file that goaccess will write. Here&rsquo;s my <code>docker-compose.yaml</code> excerpt:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="w"> </span><span class="nt">caddy</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">ghcr.io/major/caddy:main</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">container_name</span><span class="p">:</span><span class="w"> </span><span class="l">caddy</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">ports</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="s2">&#34;80:80/tcp&#34;</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="s2">&#34;443:443/tcp&#34;</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="s2">&#34;443:443/udp&#34;</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">restart</span><span class="p">:</span><span class="w"> </span><span class="l">unless-stopped</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">volumes</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">./caddy/Caddyfile:/etc/caddy/Caddyfile:Z</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">caddy_data:/data</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">caddy_config:/config</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># Caddy writes logs here πŸ‘‡</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">./caddy/logs:/logs:z</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># This is for goaccess to write its HTML file πŸ‘‡</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">./storage/goaccess_major_io:/var/www/goaccess_major_io:z</span><span class="w"> </span></span></span></code></pre></div><p>Now we need to update the <code>Caddyfile</code> to tell caddy where to place the logs and add a <code>reverse_proxy</code> configuration for our new container that serves the blog:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Caddyfile" data-lang="Caddyfile"><span class="line"><span class="cl"><span class="gh">major.io</span> <span class="p">{</span><span class="c1"> </span></span></span><span class="line"><span class="cl"><span class="c1"> # We will set up this container in a moment πŸ‘‡ </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">reverse_proxy</span> <span class="s">major_io:3000</span> <span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="k">lb_try_duration</span> <span class="mi">30s</span> </span></span><span class="line"><span class="cl"> <span class="p">}</span><span class="c1"> </span></span></span><span class="line"><span class="cl"><span class="c1"> </span></span></span><span class="line"><span class="cl"><span class="c1"> # Tell Caddy to write logs to `/logs` which </span></span></span><span class="line"><span class="cl"><span class="c1"> # is `storage/logs` on the host: </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">log</span> <span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="k">output</span> <span class="s">file</span> <span class="s">/logs/major.io-access.log</span> <span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="k">roll_size</span> <span class="s">1024mb</span> </span></span><span class="line"><span class="cl"> <span class="k">roll_keep</span> <span class="mi">20</span> </span></span><span class="line"><span class="cl"> <span class="k">roll_keep_for</span> <span class="mi">720h</span> </span></span><span class="line"><span class="cl"> <span class="p">}</span> </span></span><span class="line"><span class="cl"> <span class="p">}</span> </span></span><span class="line"><span class="cl"><span class="p">}</span> </span></span></code></pre></div><p>Great! We now have the configuration in place for caddy to write the logs and the caddy container can mount the log and analytics storage.</p> <h2 id="enabling-analytics" class="relative group">Enabling analytics <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#enabling-analytics" aria-label="Anchor">#</a></span></h2><p>We&rsquo;re heading back to the <code>docker-compose.yml</code> file once more, this time to set up a goaccess container:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="w"> </span><span class="nt">goaccess_major_io</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">docker.io/allinurl/goaccess:latest</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">container_name</span><span class="p">:</span><span class="w"> </span><span class="l">goaccess_major_io</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">restart</span><span class="p">:</span><span class="w"> </span><span class="l">always</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">volumes</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># Mount caddy&#39;s log files πŸ‘‡</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="s2">&#34;./caddy/logs:/var/log/caddy:z&#34;</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># Mount the directory where goaccess writes the analytics HTML πŸ‘‡</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="s2">&#34;./storage/goaccess_major_io:/var/www/goaccess:rw&#34;</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">command</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;/var/log/caddy/major.io-access.log --log-format=CADDY -o /var/www/goaccess/index.html --real-time-html --ws-url=wss://stats.major.io:443/ws --port=7890 --anonymize-ip --ignore-crawlers --real-os&#34;</span><span class="w"> </span></span></span></code></pre></div><p>This gets us a goaccess container to parse the logs from caddy. We need to update the caddy configuration so that we can reach the goaccess websocket for live updates:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Caddyfile" data-lang="Caddyfile"><span class="line"><span class="cl"><span class="gh">stats.major.io</span> <span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="k">root</span> <span class="nd">*</span> <span class="s">/var/www/goaccess_major_io</span> </span></span><span class="line"><span class="cl"> <span class="k">file_server</span> </span></span><span class="line"><span class="cl"> <span class="k">reverse_proxy</span> <span class="nd">/ws</span> <span class="s">goaccess_major_io:7890</span> </span></span><span class="line"><span class="cl"><span class="p">}</span> </span></span></code></pre></div><p>At this point, we have caddy writing logs in the right place, goaccess can read them, and the analytics output is written to a place where caddy can serve it. We&rsquo;ve also exposed the websocket from goaccess for live updates.</p> <h2 id="serving-the-blog" class="relative group">Serving the blog <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#serving-the-blog" aria-label="Anchor">#</a></span></h2><p>We&rsquo;ve reached the most important part!</p> <p>We added the caddy configuration to reach the blog container earlier, but now it&rsquo;s time to deploy the container itself. As a reminder, this is the container with busybox and the blog content that comes from GitHub Actions.</p> <p>The <code>docker-compose.yml</code> configuration here is <em>very basic</em>:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="w"> </span><span class="nt">major_io</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">ghcr.io/major/major.io:main</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">container_name</span><span class="p">:</span><span class="w"> </span><span class="l">major_io</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">restart</span><span class="p">:</span><span class="w"> </span><span class="l">always</span><span class="w"> </span></span></span></code></pre></div><p>Caddy will connect to this container on port 3000 to serve the blog. (We set port 3000 in the original <code>Dockerfile</code>).</p> <p>At this point, everything should be set to go. Make it live with:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="go">docker-compose up -d </span></span></span></code></pre></div><p>This should bring up the goaccess and blog containers while also restarting caddy. The website should be visible now at <a href="https://major.io/" target="_blank" rel="noreferrer">major.io</a> (and that&rsquo;s how you&rsquo;re reading this today).</p> <h2 id="what-about-new-posts" class="relative group">What about new posts? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#what-about-new-posts" aria-label="Anchor">#</a></span></h2><p>I&rsquo;m glad you asked! That was something I wondered about as well. <strong>How do we get the new blog content down to the container when a new post is written?</strong> πŸ€”</p> <p>As I&rsquo;ve <a href="https://major.io/p/watchtower/">written in the past</a>, I like using <a href="https://containrrr.dev/watchtower/" target="_blank" rel="noreferrer">watchtower</a> to keep containers updated. Watchtower offers an HTTP API interface for webhooks to initiate container updates. We can trigger that update via a simple curl request from GitHub Actions when our container pipeline runs.</p> <p>My <a href="https://github.com/major/major.io/blob/main/.github/workflows/container.yml" target="_blank" rel="noreferrer">container workflow</a> has a brief bit at the end that does this:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Update the blog container</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">if</span><span class="p">:</span><span class="w"> </span><span class="l">github.event_name != &#39;pull_request&#39;</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd"> </span></span></span><span class="line"><span class="cl"><span class="sd"> curl -s -H &#34;Authorization: Bearer ${WATCHTOWER_TOKEN}&#34; \ </span></span></span><span class="line"><span class="cl"><span class="sd"> https://watchtower.thetanerd.com/v1/update</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">env</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">WATCHTOWER_TOKEN</span><span class="p">:</span><span class="w"> </span><span class="l">${{ secrets.WATCHTOWER_TOKEN }}</span><span class="w"> </span></span></span></code></pre></div><p>You can enable this in watchtower with a few new environment variables in your <code>docker-compose.yml</code>:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-YAML" data-lang="YAML"><span class="line"><span class="cl"><span class="w"> </span><span class="nt">watchtower</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># New environment variables πŸ‘‡</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">environment</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">WATCHTOWER_HTTP_API_UPDATE=true</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">WATCHTOWER_HTTP_API_TOKEN=SUPER-SECRET-TOKEN-PASSWORD</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">WATCHTOWER_HTTP_API_PERIODIC_POLLS=true</span><span class="w"> </span></span></span></code></pre></div><p><code>WATCHTOWER_HTTP_API_UPDATE</code> enables the updating via API and <code>WATCHTOWER_HTTP_API_TOKEN</code> sets the token required when making the API request. If you set <code>WATCHTOWER_HTTP_API_PERIODIC_POLLS</code> to <code>true</code>, watchtower will still periodically look for updates to containers even if an API request never appeared. By default, watchtower will stop doing periodic updates if you enable the API.</p> <p>This is working on my site right now and you can view my public blog stats on <a href="https://stats.major.io" target="_blank" rel="noreferrer">stats.major.io</a>. πŸŽ‰</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>Florin went all the way down to 154KB and I was extremely impressed. However, I&rsquo;m not too worried about an extra megabyte here. πŸ˜‰&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>Connect Caddy to Porkbunhttps://major.io/p/caddy-porkbun/Thu, 29 Feb 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/caddy-porkbun/<p>I recently told a coworker about <a href="https://caddyserver.com/" target="_blank" rel="noreferrer">Caddy</a>, a small web and proxy server with a very simple configuration. It also has a handy feature where it manages your TLS certificate for you automatically.</p> <p>However, one problem I had at home with my <a href="https://fedoraproject.org/coreos/" target="_blank" rel="noreferrer">CoreOS</a> deployment is that I don&rsquo;t have inbound network access to handle the certificate verification process. Most automated certificate vendors need to reach your web server to verify that you have control over your domain.</p> <p>This post talks about how to work around this problem with domains registered at <a href="https://porkbun.com/" target="_blank" rel="noreferrer">Porkbun</a>.</p> <h2 id="dns-validation" class="relative group">DNS validation <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#dns-validation" aria-label="Anchor">#</a></span></h2><p>Certificate providers usually default to verifying domains by making a request to your server and retrieving a validation code. If your systems are all behind a firewall without inbound access from the internet, you can use DNS validation instead.</p> <p>The process looks something like this:</p> <ol> <li>You tell the certificate provider the domain names you want on your certificate</li> <li>The certificate provider gives you some DNS records to add wherever you host your DNS records</li> <li>You add the DNS records</li> <li>You get your certificates once the certificate provider verifies the records.</li> </ol> <p>You can do this manually with something like <a href="https://github.com/acmesh-official/acme.sh" target="_blank" rel="noreferrer">acme.sh</a> today, but it&rsquo;s <strong>painful</strong>:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl"><span class="c1"># Make the initial certificate request</span> </span></span><span class="line"><span class="cl">acme.sh --issue --dns -d example.com <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> --yes-I-know-dns-manual-mode-enough-go-ahead-please </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># Add your DNS records manually.</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># Verify the DNS records and issue the certificates.</span> </span></span><span class="line"><span class="cl">acme.sh --issue --dns -d example.com <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> --yes-I-know-dns-manual-mode-enough-go-ahead-please --renew </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># Copy the keys/certificates and configure your webserver.</span> </span></span></code></pre></div><p>We don&rsquo;t want to live this way.</p> <p> <figure> <picture class="mx-auto my-0 rounded-md" > <img width="581" height="355" class="mx-auto my-0 rounded-md" alt="do-not-want.gif" loading="lazy" decoding="async" src="https://major.io/p/caddy-porkbun/do-not-want.gif" /> </picture> </figure> </p> <p>Let&rsquo;s talk about how Caddy can help.</p> <h2 id="adding-porkbun-support-to-caddy" class="relative group">Adding Porkbun support to Caddy <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#adding-porkbun-support-to-caddy" aria-label="Anchor">#</a></span></h2><p>Caddy is a minimal webserver and <a href="https://github.com/caddy-dns/porkbun" target="_blank" rel="noreferrer">Porkbun support</a> doesn&rsquo;t get included by default. However, we can quickly add it via a simple container build:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Dockerfile" data-lang="Dockerfile"><span class="line"><span class="cl"><span class="k">FROM</span><span class="s"> caddy:2.7.6-builder AS builder</span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">RUN</span> xcaddy build <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> --with github.com/caddy-dns/porkbun<span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">FROM</span><span class="s"> caddy:2.7.6</span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">COPY</span> --from<span class="o">=</span>builder /usr/bin/caddy /usr/bin/caddy<span class="err"> </span></span></span></code></pre></div><p>This is a two stage container build where we compile the Porkbun support and then use that new <code>caddy</code> binary in the final container.</p> <p>We&rsquo;re not done yet!</p> <h2 id="automated-caddy-builds-with-updates" class="relative group">Automated Caddy builds with updates <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#automated-caddy-builds-with-updates" aria-label="Anchor">#</a></span></h2><p>I created a <a href="https://github.com/major/caddy" target="_blank" rel="noreferrer">GitHub repository</a> that builds the Caddy container for me and keeps it updated. There&rsquo;s a <a href="https://github.com/major/caddy/blob/main/.github/workflows/docker-publish.yml" target="_blank" rel="noreferrer">workflow to publish a container</a> to GitHub&rsquo;s container repository and I can pull containers from there on my various CoreOS machines.</p> <p>In addition, I use <a href="https://github.com/apps/renovate" target="_blank" rel="noreferrer">Renovate</a> to watch for Caddy updates. New updates come through a <a href="https://github.com/major/caddy/pull/10" target="_blank" rel="noreferrer">regular pull request</a> and I can apply them whenever I want.</p> <p> <figure> <picture class="mx-auto my-0 rounded-md" > <img width="918" height="461" class="mx-auto my-0 rounded-md" alt="Renovate pull request" loading="lazy" decoding="async" src="https://major.io/p/caddy-porkbun/pr_hu1139072803250834900.png" srcset="https://major.io/p/caddy-porkbun/pr_hu9098124362148163369.png 330w,/p/caddy-porkbun/pr_hu1139072803250834900.png 660w ,/p/caddy-porkbun/pr.png 918w ,/p/caddy-porkbun/pr.png 918w " sizes="100vw" /> </picture> <figcaption class="text-center">Example pull request from Renovate</figcaption> </figure> </p> <h2 id="connecting-to-porkbun" class="relative group">Connecting to Porkbun <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#connecting-to-porkbun" aria-label="Anchor">#</a></span></h2><p>We start here by getting an API key to manage the domain at Porkbun.</p> <ol> <li>Log into your <a href="https://porkbun.com/account/domainsSpeedy" target="_blank" rel="noreferrer">Porkbun dashboard</a>.</li> <li>Click <strong>Details</strong> to the right of the domain you want to manage.</li> <li>Look for <strong>API Access</strong> in the leftmost column and turn it on.</li> <li>At the top right of the dashboard, click <strong>Account</strong> and then <strong>API Access</strong>.</li> <li>Add a title for your new API key, such as <em>Caddy</em>, and click <strong>Create API Key</strong>.</li> <li>Save the API key and secrey key that are displayed.</li> </ol> <p>Open up your Caddy configuration file (the <em>Caddyfile</em>) and add some configuration:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-caddyfile" data-lang="caddyfile"><span class="line"><span class="cl"><span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="k">email</span> <span class="s">me@example.com</span><span class="c1"> </span></span></span><span class="line"><span class="cl"><span class="c1"> </span></span></span><span class="line"><span class="cl"><span class="c1"> # Uncomment this next line if you want to get </span></span></span><span class="line"><span class="cl"><span class="c1"> # some test certificates first. </span></span></span><span class="line"><span class="cl"><span class="c1"> # acme_ca https://acme-staging-v02.api.letsencrypt.org/directory </span></span></span><span class="line"><span class="cl"><span class="c1"></span> </span></span><span class="line"><span class="cl"> <span class="k">acme_dns</span> <span class="s">porkbun</span> <span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="k">api_key</span> <span class="s">pk1_******</span> </span></span><span class="line"><span class="cl"> <span class="k">api_secret_key</span> <span class="s">sk1_******</span> </span></span><span class="line"><span class="cl"> <span class="p">}</span> </span></span><span class="line"><span class="cl"><span class="p">}</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="gh">example.com</span> <span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="k">handle</span> <span class="p">{</span> </span></span><span class="line"><span class="cl"> <span class="k">respond</span> <span class="s2">&#34;Hello world!&#34;</span> </span></span><span class="line"><span class="cl"> <span class="p">}</span> </span></span><span class="line"><span class="cl"><span class="p">}</span> </span></span></code></pre></div><p>Save the Caddyfile and restart your Caddy server or container. Caddy will immediately begin requesting your TLS certificates and managing your DNS records for those certificates. This normally finishes in less than 30 seconds or so during the first run.</p> <p>If you don&rsquo;t see the HTTPS endpoint working within a minute or two, be sure to check the Caddy logs. You might have a typo in a Porkbun API key or the domain you&rsquo;re trying to modify doesn&rsquo;t have the <strong>API Access</strong> switch enabled.</p> <div class="flex rounded-md bg-primary-100 px-4 py-3 dark:bg-primary-900"> <span class="pe-3 text-primary-400"> <span class="icon relative inline-block px-1 align-text-bottom"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><path fill="currentColor" d="M256 0C114.6 0 0 114.6 0 256s114.6 256 256 256s256-114.6 256-256S397.4 0 256 0zM256 128c17.67 0 32 14.33 32 32c0 17.67-14.33 32-32 32S224 177.7 224 160C224 142.3 238.3 128 256 128zM296 384h-80C202.8 384 192 373.3 192 360s10.75-24 24-24h16v-64H224c-13.25 0-24-10.75-24-24S210.8 224 224 224h32c13.25 0 24 10.75 24 24v88h16c13.25 0 24 10.75 24 24S309.3 384 296 384z"/></svg> </span> </span> <span class="dark:text-neutral-300">Remember that Porkbun requires you to enable API access for each domain. API access is disabled at Porkbun by default.</span> </div> <p><strong>That&rsquo;s it!</strong> πŸŽ‰</p> <h2 id="renewals" class="relative group">Renewals <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#renewals" aria-label="Anchor">#</a></span></h2><p>Caddy will keep watch over the certificates and begin the renewal process as the expiration approaches. It has a very careful retry mechanism that ensures your certificates are updated without tripping any rate limits at the certificate provider.</p> <h2 id="further-reading" class="relative group">Further reading <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#further-reading" aria-label="Anchor">#</a></span></h2><p>Caddy&rsquo;s detailed documentation about <a href="https://caddyserver.com/docs/automatic-https" target="_blank" rel="noreferrer">Automatic HTTPS</a> and the <a href="https://caddyserver.com/docs/caddyfile/directives/tls" target="_blank" rel="noreferrer">tls configuration directive</a> should answer most questions about how the process works.</p>Linux on the AMD ThinkPad Z13 G2https://major.io/p/linux-thinkpad-z13-amd/Sun, 14 Jan 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/linux-thinkpad-z13-amd/<p>AMD&rsquo;s new <a href="https://en.wikipedia.org/wiki/Zen_4" target="_blank" rel="noreferrer">Zen 4 processors</a> started rolling out in 2022 and I&rsquo;ve been watching for the mobile CPUs to reach laptops. I like where AMD is going with these chips and how they provide lots of CPU power without eating up the battery.</p> <p>I recently ordered a <a href="https://www.lenovo.com/us/en/p/laptops/thinkpad/thinkpadz/thinkpad-z13-gen-2-%2813-inch-amd%29/len101t0073" target="_blank" rel="noreferrer">ThinkPad Z13 Gen 2</a> with an AMD Ryzen 7. As you might expect, I loaded it up with Fedora Linux and set out to ensure that everything works.</p> <p>This post includes all of the configurations and changes I added along the way.</p> <h1 id="power-management" class="relative group">Power management <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#power-management" aria-label="Anchor">#</a></span></h1><p>I removed the power profiles daemon that comes with Fedora by default. and replaced it with <a href="https://linrunner.de/tlp/index.html" target="_blank" rel="noreferrer">tlp</a>. This is a great package for ThinkPad laptops as it takes care of most of the power management configuration for you with sane defaults. It also offers an easy to read configuration file where you can make adjustments.</p> <p>The defaults seem to be working well so far, but my only complaint is that the power management for <code>amdgpu</code> seems to be <em>really aggressive</em>. Graphics performance on battery power is <em>okay</em>, but I&rsquo;m told this improves in kernel 6.7. I&rsquo;m on 6.6.11 in Fedora 39 right now.</p> <p>I&rsquo;ll wait to see if this new kernel makes any improvements.</p> <h1 id="touchpad" class="relative group">Touchpad <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#touchpad" aria-label="Anchor">#</a></span></h1><p>The ELAN touchpad in the Z13 is a bit different. It&rsquo;s a <em>haptic</em> touchpad. It doesn&rsquo;t push down with a click like the other thinkpads. It provides haptic feedback, much like a mobile phone does when you tap on the screen. (I usually turn this off on my phone, but it feels good on the laptop.)</p> <p>The touchpad works right out of the box without any additional configuration. I made a basic Sway configuration stanza to get it configured with my preferences:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="c1"># ThinkPad Z13 Gen 2 AMD Touchpad</span> </span></span><span class="line"><span class="cl"><span class="na">input &#34;11311:40:SNSL0028:00_2C2F:0028_Touchpad&#34; {</span> </span></span><span class="line"><span class="cl"> <span class="na">drag disabled</span> </span></span><span class="line"><span class="cl"> <span class="na">tap enabled</span> </span></span><span class="line"><span class="cl"> <span class="na">dwt enabled</span> </span></span><span class="line"><span class="cl"> <span class="na">natural_scroll disabled</span> </span></span><span class="line"><span class="cl"><span class="na">}</span> </span></span></code></pre></div><p>The configuration above enables tap to click and dragging with taps. I like the old school scrolling style and I&rsquo;ve disabled the natural scroll.</p> <p>You can always get a list of your input devices in Sway with <code>swaymsg</code>:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="go">swaymsg -t get_inputs </span></span></span></code></pre></div><h1 id="display" class="relative group">Display <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#display" aria-label="Anchor">#</a></span></h1><p>The display worked right out of the box but the UI elements were scaled up far too large for me. I typically value screen real estate over all other aspects, but my usual default of scaling to 1.0 made the UI far too small.</p> <p>I set my output scaling to 1.2:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="c1"># Disable HiDPI</span> </span></span><span class="line"><span class="cl"><span class="na">output * scale 1.2</span> </span></span></code></pre></div><p>I also enabled the <a href="https://rpmfusion.org/Howto/Multimedia" target="_blank" rel="noreferrer">RPM Fusion repos</a> to get the freeworld AMD Mesa drivers:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="go">sudo dnf install \ </span></span></span><span class="line"><span class="cl"><span class="go"> https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm \ </span></span></span><span class="line"><span class="cl"><span class="go"> https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm </span></span></span><span class="line"><span class="cl"><span class="go">sudo dnf swap mesa-va-drivers mesa-va-drivers-freeworld </span></span></span><span class="line"><span class="cl"><span class="go">sudo dnf swap mesa-vdpau-drivers mesa-vdpau-drivers-freeworld </span></span></span></code></pre></div><h1 id="audio" class="relative group">Audio <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#audio" aria-label="Anchor">#</a></span></h1><p>Sound worked right out of the box, but I found that the loudness preset from <a href="https://github.com/wwmm/easyeffects" target="_blank" rel="noreferrer">easyeffects</a> made the speakers sound a little bit better:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="go">sudo dnf install easyeffects </span></span></span></code></pre></div><h1 id="everything-else" class="relative group">Everything else <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#everything-else" aria-label="Anchor">#</a></span></h1><p>Everything else just worked!</p> <p>I&rsquo;m really pleased with the performance and the battery life so far. My only complaint is that the OLED screen can be a battery hog at times.</p> <p>For more details, check out the <a href="https://wiki.archlinux.org/title/Lenovo_ThinkPad_Z13" target="_blank" rel="noreferrer">Arch Linux wiki page for the Z13</a>. They documented lots of the function keys if you want to create keyboard shortcuts and they link to some downloadable monitor profiles.</p>Dark mode in Swayhttps://major.io/p/sway-dark-mode/Tue, 09 Jan 2024 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/sway-dark-mode/<p>Ah, dark mode! I savor my dark terminals, window decorations, and desktop wallpapers. It&rsquo;s so much easier on my eyes on those long work days. 😎</p> <p>However, I think the author <a href="https://en.wikipedia.org/wiki/Mary_Oliver" target="_blank" rel="noreferrer">Mary Oliver</a> said it best:</p> <blockquote> <p>Someone I loved once gave me a box full of darkness. It took me years to understand that this too, was a gift.</p> </blockquote> <p>In most window managers, such as GNOME or KDE, switching to dark mode involves a simple trip to the settings panels and clicking different themes. Sway doesn&rsquo;t offer us those types of comforts, but we can get dark mode there, too!</p> <p> <figure> <picture class="mx-auto my-0 rounded-md" > <img width="295" height="273" class="mx-auto my-0 rounded-md" alt="sunglasses-wiggle.gif" loading="lazy" decoding="async" src="https://major.io/p/sway-dark-mode/sunglasses-wiggle.gif" /> </picture> </figure> </p> <h1 id="gtk-applications" class="relative group">GTK applications <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#gtk-applications" aria-label="Anchor">#</a></span></h1><p>If you happen to have GNOME on your system alongside sway, go into <strong>Settings</strong>, then <strong>Appearance</strong> and select <em>Dark</em>. You can also get dark mode by applying a setting in <code>~/.config/gtk-3.0/settings.ini</code>:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="k">[Settings]</span> </span></span><span class="line"><span class="cl"><span class="na">gtk-application-prefer-dark-theme</span><span class="o">=</span><span class="s">1</span> </span></span></code></pre></div><p>Restart whichever application you were using and it should pick up the new configuration.</p> <p>Firefox, for example, ships with an automatic appearance setting that follows the OS. That should be reflected immediately upon restart. If not, go into Firefox&rsquo;s settings, and look for dark mode under the <strong>Language and Appearance</strong> section of the general settings.</p> <h1 id="qt-applications" class="relative group">QT applications <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#qt-applications" aria-label="Anchor">#</a></span></h1><p>Most of my applications are GTK-based, but I have one or two which use QT. Again, just like the GTK example, if you have KDE installed along side Sway, you can configure dark mode there easily. Just open the system settings and look for <em>Breeze Dark</em> in the <strong>Plasma Style</strong> section.</p> <p>You don&rsquo;t have KDE? Don&rsquo;t worry! There are a couple of commands which should work:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl"><span class="c1"># This should work for all QT/KDE apps</span> </span></span><span class="line"><span class="cl"><span class="c1"># if you have the Breeze Dark theme installed.</span> </span></span><span class="line"><span class="cl">lookandfeeltool -platform offscreen <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> --apply <span class="s2">&#34;org.kde.breezedark.desktop&#34;</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="c1"># You can set the theme for GTK apps here as well</span> </span></span><span class="line"><span class="cl"><span class="c1"># if you run into problems.</span> </span></span><span class="line"><span class="cl">dbus-send --session --dest<span class="o">=</span>org.kde.GtkConfig <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> --type<span class="o">=</span>method_call /GtkConfig org.kde.GtkConfig.setGtkTheme <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> <span class="s2">&#34;string:Breeze-dark-gtk&#34;</span> </span></span></code></pre></div><h1 id="alternate-dark-mode-based-on-time" class="relative group">Alternate dark mode based on time <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#alternate-dark-mode-based-on-time" aria-label="Anchor">#</a></span></h1><p>Many window managers offer a method for adjusting dark and light modes based on the time of day. For example, some people love brighter interfaces during the day and darker ones at night. There&rsquo;s a great tool called <a href="https://gitlab.com/WhyNotHugo/darkman" target="_blank" rel="noreferrer">darkman</a> that makes this easier. πŸ€“</p> <p>The darkman service runs in the background and runs various commands to change dark mode settings for all kinds of window managers. It also speaks to dbus directly to set the configurations if needed.</p> <p>It also has a <a href="https://gitlab.com/WhyNotHugo/darkman/-/tree/main/examples/dark-mode.d?ref_type=heads" target="_blank" rel="noreferrer">directory full of user contributed scripts</a> to change dark and light modes for various environments. You might be able to pull some commands from these files to test which configurations might work best on your system.</p>On diversityhttps://major.io/p/on-diversity/Sat, 16 Dec 2023 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/on-diversity/<p>️ πŸ‘‹ <em>This post represents my own views on the topic of diversity and it doesn&rsquo;t represent the views of my employer or any professional group I belong to.</em></p> <hr> <p>I&rsquo;ve written a post on diversity and deleted it several times. It remains a sensitive topic for different people for different reasons. My gut feeling is that no matter how you frame a post on diversity, some group of people will be upset about it<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>.</p> <p>There was a great speaker who came and spoke to us at my last job and she made an excellent point that I remember today:</p> <blockquote> <p>Your experiences are yours. Nobody can take them away from you.</p> <p>Nobody can say that your experiences do not matter.</p> <p>Nobody can tell you that you didn&rsquo;t experience what you experienced.</p> <p>Sharing these experiences with others allows us to grow and understand more about the world around us.</p> </blockquote> <p>That speech entirely changed my way of thinking about interactions with other people at work and at home. There are two main benefits here:</p> <ul> <li>It&rsquo;s incredibly freeing for someone who has experienced something to be able to share it with others and not be told that their experience was wrong or misguided.</li> <li>It&rsquo;s also freeing for the listener to take in someone else&rsquo;s experience and be able to ask clarifying questions so they get a better understanding of how something felt for someone else.</li> </ul> <p>With that in mind, here&rsquo;s we go with the rest of the post. I&rsquo;m not deleting it this time.</p> <p>I promise. πŸ˜‰</p> <h1 id="my-first-experience" class="relative group">My first experience <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#my-first-experience" aria-label="Anchor">#</a></span></h1><p>I&rsquo;ve written in the past about my unexpected leap to lead an information security architecture team in a previous role. Being a director was a new world unto itself, but then I found that my team wasn&rsquo;t performing well. To make matters worse, our success was critical to the ongoing work of the security department as a whole.</p> <p>We had three members of the team that all brought something unique to the team&rsquo;s perspective. All three were men of different races, but each had a different approach to security based on their backgrounds and experiences. One left the team due to some interpersonal issues that eventually boiled over.</p> <p>I was suddenly down to two people and our team needed to hire two as soon as possible.</p> <p>Recruiting teams started putting feelers out into the market to find talented people and I was poking several friends for referrals. A colleague in another department reached out and really wanted to join the team. I knew about her experience from several previous interactions and she was highly recommended from her peers. She joined the team and hit the ground running.</p> <p>As applicants began trickling in through the recruiting team, I started with screening calls for each. We really needed someone with skills in a few key areas:</p> <ul> <li>Great communicator with empathy</li> <li>Knowledge of secure development and operations practices</li> <li>Someone who could be trusted to operate independently and work on team projects</li> </ul> <p>Most of the applicants I screened were male and that wasn&rsquo;t a surprise at the time. We brought five through the screening into interviews and it was down to four males and one female. Three of them turned out to be great and they all had deep knowledge of security architecture. After another round of interviews, we began to realize that the female matched the other applicants, but her communication skills were stronger, especially under pressure.</p> <p>Needless to say, we sent her the offer and she accepted! We were thrilled! Our team was full!</p> <h1 id="getting-underway" class="relative group">Getting underway <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#getting-underway" aria-label="Anchor">#</a></span></h1><p>We began chipping away at the mountain of projects set aside for our team and started making progress. Our new team member was struggling to move from the rigidity of her previous employer to our new way of working, but she adjusted well over time.</p> <p>I sat down in one of our weekly leadership meetings some time later. These meetings usually involved a round-the-horn of what&rsquo;s working well for each team, the threats on the board for the next few months, and our plans.</p> <p>We usually had an attendee from HR in the meetings for various reasons and she asked me how our new team member was doing. I said:</p> <blockquote> <p>Oh, she&rsquo;s doing a good job. Her last company was pretty rigid and things are different here, but she&rsquo;s figuring it out. She knows her stuff and she&rsquo;s a team player. We&rsquo;re working through some small things here and there.</p> </blockquote> <p>Then the HR representative said:</p> <blockquote> <p>Well, I have to commend you for building out a such a diverse team. It&rsquo;s much more so than the other teams. That&rsquo;s really great work and I want to make sure you&rsquo;re recognized for it.</p> </blockquote> <p>I smiled and thanked her (because that&rsquo;s my usual response), but then I almost felt sick.</p> <h1 id="different-view" class="relative group">Different view <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#different-view" aria-label="Anchor">#</a></span></h1><p>I left that meeting and went back to my desk to think.</p> <p>Had I assembled a diverse team intentionally? No, I didn&rsquo;t. I looked for people who had the qualities we desperately needed, gave them guardrails, and got out of the way. That&rsquo;s what you do with smart people, right?</p> <p>Then I wondered, <em>&ldquo;Is my team really diverse?&rdquo;</em></p> <ul> <li>There were two men and two women.</li> <li>Two were on the younger end of a generation and two were on the older end.</li> <li>One was of mixed Asian descent, one was Hispanic, and two were what most people would likely refer to as &ldquo;white.&rdquo;</li> </ul> <p>So maybe my team <em>is diverse</em>.</p> <p>Then I realized that most of the people on the team had the same certifications, all had at least an undergraduate degree, and all were married. All of them were in heterosexual relationships and all dressed in a way that aligned with their gender.</p> <p>Does that mean they&rsquo;re not diverse?</p> <p>Is my team more or less diverse than other teams?</p> <p>Does any of this even matter?</p> <p>Did I do a good or a bad thing?</p> <h1 id="diversity-challenges" class="relative group">Diversity challenges <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#diversity-challenges" aria-label="Anchor">#</a></span></h1><p>This brings me to the two problems I struggle with most around diversity, especially when people talk about <em>increasing</em> or <em>improving</em> diversity on their team or within their company:</p> <ol> <li>Quantifying diversity is highly subjective and in the eye of the beholder.</li> <li>Challenges arise when you apply diversity requirements to real world situations.</li> </ol> <p>I&rsquo;ll break down both of these now.</p> <h2 id="in-the-eye-of-the-beholder" class="relative group">In the eye of the beholder <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#in-the-eye-of-the-beholder" aria-label="Anchor">#</a></span></h2><p>You can choose how you want to measure diversity on all kinds of factors. Depending on the factors, a team can look more or less diverse. Also, your experiences often define how you judge the diversity of people and teams.</p> <p>One could argue that a team made up entirely of white males is likely not very diverse. The majority of people would likely agree with that statement.</p> <p>However, what if those males vary in their sexual orientations, educational backgrounds, and socioeconomic status. Is that diverse?</p> <p>If you have a team of people made up of various genders with various sexual orientations from all contents on the planet, but they all went to Ive League schools and they&rsquo;re all wealthy &ndash; is that diverse?</p> <p>Are any of these examples diverse <em>enough?</em> Does the answer to that question <em>even matter?</em></p> <p>In my experience, assembling a team of people with different backgrounds and approaches to problems is incredibly valuable. That type of diversity led to some incredible innovation in the past.</p> <p>However, these diverse backgrounds and approaches don&rsquo;t always line up with differences in gender identity, socioeconomic status, sexual orientation, or other factors. This is why I find it really challenging to quantify the level of diversity within a company or in individual teams.</p> <h2 id="rubber-meets-the-road" class="relative group">Rubber meets the road <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#rubber-meets-the-road" aria-label="Anchor">#</a></span></h2><p>There&rsquo;s a common phrase in English: <em>&ldquo;when the rubber meets the road.&rdquo;</em></p> <p>In a literal sense, it&rsquo;s referring to when car tires move on pavement during a race. What it really means is, when it comes time to do something for real and the stakes are high, what happens?</p> <p>Here&rsquo;s another example. Let&rsquo;s say you lead an engineering team that is all males and your company says that diversity must be a priority in hiring decisions.</p> <p>So you take your job requisition and send it through the recruiting team. They work hard to remove any gender-specific language or anything else that might turn an applicant away. You put the job on the internet, talk to your friends about people they know, and then wait for the responses.</p> <p>Let&rsquo;s assume you get ten male applicants.</p> <p>Do you proceed with screening and interviewing them while you try harder to drum up more female applicants? If a female applicant never appears, do you pause the hiring process while you try to find one? What if your existing applicants find other roles in the meantime and suddenly your applicant pipeline is empty?</p> <p>Some might say <em>&ldquo;Yes, of course you wait until you can find a female applicant!&rdquo;</em> In that case, your team is still short-staffed and likely not performing as well as it could. Would that be good for your customers? How about your shareholders?</p> <p>Others might say <em>&ldquo;No, go ahead and complete the hiring process but you should search harder for women for future roles.&rdquo;</em> In this case, you&rsquo;ll have a fully staffed team and hopefully be delivering more value quickly. However, you haven&rsquo;t improved the diversity on your team and that could come back to be a problem if you&rsquo;re asked about it later.</p> <hr> <p>Go backwards a bit with the same example and assume you get a split of ten applicants: half male and half female. That&rsquo;s awesome because now you have a diverse talent pool, right?</p> <p>Here&rsquo;s where it gets challenging.</p> <p>If you interview them all and make an offer to the female applicant because she has the skills and qualifications needed, you now have a more diverse team (on one measure) and you&rsquo;re fully staffed! Great!</p> <p>If you interview them all and it turns out one of the men has the best skills and qualifications, what do you do? Your company made diversity a priority, but you&rsquo;re also trying to assemble a strong team.</p> <p>Do you take a less qualified applicant that improves the team&rsquo;s diversity?</p> <p>Or, do you take a more qualified applicant that leaves the team&rsquo;s diversity unchanged?</p> <p>This is where diversity breaks down: when you have to really sit down and compare outcomes, there&rsquo;s not a right answer.</p> <h1 id="another-viewpoint" class="relative group">Another viewpoint <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#another-viewpoint" aria-label="Anchor">#</a></span></h1><p>My wife constantly points out things to me that I completely missed and we&rsquo;ve talked about this topic many times. She has asked me the same thing in the past:</p> <blockquote> <p>Why do people in your field care so much about getting women into technology? I hate technology. Maybe other women hate technology, too. If I knew I was hired someplace because they wanted a woman for the role and they weren&rsquo;t looking at how well someone could do the job, I&rsquo;d be pretty upset.</p> </blockquote> <p>She&rsquo;s a medical professional and she&rsquo;s happy to remind me about this:</p> <blockquote> <p>I went to PA (physician assistant) school and most people there were women. All the nurses at my office are women. All the front office staff are women. We&rsquo;re not out there trying to get male nurses or male front office staff in here all the time. We just find people who do their job well and hire them.</p> </blockquote> <p>Our conversations really make me stop and think.</p> <h1 id="my-goals" class="relative group">My goals <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#my-goals" aria-label="Anchor">#</a></span></h1><p>I&rsquo;d also like to see more people from underrepresented communities across the globe break into the world of technology and really change things. This means empowering a wider array of people with varying gender, education, nationality, wealth, and opportunities to join a field of work which they thought might be inaccessible to them.</p> <p>This is why I try to volunteer as much as possible to inspire young people of all backgrounds to set goals for themselves and look at the world as if nothing is out of reach.</p> <p>It&rsquo;s one of the reasons I write this blog and put everything out there for free. Democratizing access to learning (and my mediocre blog posts) is key to leveling the playing field.</p> <p>These are some of those pieces of work that are never finished.</p> <p>However, I really worry that quantifying diversity or forcing one&rsquo;s definition of diversity onto someone else could lead us to a bad place where no result is satisfactory. It&rsquo;s much more subjective than some would like to admit and that becomes a problem when you directly apply it to specific situations.</p> <p>In the meantime, I&rsquo;ll keep writing these posts, mentoring others, and lifting people up to do things they never imagined they could do. ️β™₯️</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>Then again, we live in a world where someone can say <em>&ldquo;Puppies are cute&rdquo;</em> and the first reply would be <em>&ldquo;Why do you hate cats so much?&rdquo;</em> πŸ˜„&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>Horror book reviews from October 2023https://major.io/p/horror-book-reviews/Sun, 19 Nov 2023 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/horror-book-reviews/<p>Reading allows me to travel to other places and times while also reducing my stress and helping me to think more creatively. Sometimes this leads me to wild fictional stories or takes me on a learning journey into history.</p> <p><em>(I <a href="https://www.goodreads.com/user/show/49133137-major-hayden" target="_blank" rel="noreferrer">track my reading lists on Goodreads</a> if you want to see what I&rsquo;m reading.)</em></p> <p>In this post, I&rsquo;ll list the spooky books I read this October and hopefully you&rsquo;ll find at least one them interesting!</p> <h1 id="the-cabin-at-the-end-of-the-world" class="relative group">The Cabin at the End of the World <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#the-cabin-at-the-end-of-the-world" aria-label="Anchor">#</a></span></h1><p>My first book of the month was Paul Tremblay&rsquo;s <a href="https://www.goodreads.com/book/show/36381091-the-cabin-at-the-end-of-the-world" target="_blank" rel="noreferrer">The Cabin at the End of the World</a>. It&rsquo;s centered around a family with a small child that goes on a relaxing vacation in a lakefront cabin.</p> <p>All is well until a friendly stranger named Leonard befriends the child, Wen, and his three fellow travelers appear. The travelers hold the family hostage and tell them that they have the key to save the world, but it&rsquo;s not as easy as it seems. There seems to be an unseen force that has some sort of control over the travelers. πŸ€”</p> <h2 id="my-thoughts" class="relative group">My thoughts <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#my-thoughts" aria-label="Anchor">#</a></span></h2><p>I thought I had this story figured out several times only to be proven wrong just as many times. There are so many themes in this book that tug at your emotions, including family, racism, homophobia, and socioeconomic differences. This book packs plenty of suspense, but the most frightening parts aren&rsquo;t supernatural or ghostly. The scariest parts feel centered around the essence of human nature.</p> <p>Although this felt like a quick read, it was intense in many places. I definitely enjoyed it and I was looking forward to seeing the movie adaptation, <a href="https://en.wikipedia.org/wiki/Knock_at_the_Cabin" target="_blank" rel="noreferrer">Knock at the Cabin</a>. The movie was done by <a href="https://en.wikipedia.org/wiki/Knock_at_the_Cabin" target="_blank" rel="noreferrer">M. Night Shyamalan</a> (of <a href="https://en.wikipedia.org/wiki/The_Sixth_Sense" target="_blank" rel="noreferrer">The Sixth Sense</a> fame) and I read that he changed the entire ending of the movie. 😞</p> <h1 id="the-ruins" class="relative group">The Ruins <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#the-ruins" aria-label="Anchor">#</a></span></h1><p>I moved onto something <em>completely different</em> next with Scott Smith&rsquo;s <a href="https://www.goodreads.com/book/show/21726.The_Ruins" target="_blank" rel="noreferrer">The Ruins</a>. The story takes place in Mexico with several people on a beach vacation. One of the tourists notes that his brother went to check out an archeological dig further into the countryside but never returned. The group eventually decides to go search for the missing brother as some sort of vacation adventure.</p> <p>The adventure turns ugly as they make their way to the country&rsquo;s interior. 😬</p> <h2 id="my-thoughts-1" class="relative group">My thoughts <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#my-thoughts-1" aria-label="Anchor">#</a></span></h2><p>You&rsquo;ll have a tough time finding the antagonist in this story until it&rsquo;s too late, but that&rsquo;s part of the fun. Every character in the group brings personality quirks and old baggage with them that impairs their judgement in different ways. This works well for some but not for others.</p> <p>This book had several scary moments, but most of the horror came again from how humans interact with one another. As soon as someone (or something) else figures out how to exploit those against them, bad things happen.</p> <p>This book was difficult to read because much of it was quite gruesome and brutal. It&rsquo;s definitely a book for adults only and you should be prepared to work your way through it.</p> <h1 id="the-troop" class="relative group">The Troop <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#the-troop" aria-label="Anchor">#</a></span></h1><p>Everything is fine with a scout trip to an island in Canada in Nick Cutter&rsquo;s <a href="https://www.goodreads.com/book/show/17571466-the-troop" target="_blank" rel="noreferrer">The Troop</a>. Well, it&rsquo;s fine until a mysterious and incredibly hungry man suddenly shows up on the island. He looks a lot like he&rsquo;s dead already and the the leader of the scout troop, a medical doctor, is completely mystified by the man&rsquo;s condition.</p> <p>It goes downhill from there in a story told mostly through diary entries, newspaper clippings, and court testimony.</p> <h2 id="my-thoughts-2" class="relative group">My thoughts <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#my-thoughts-2" aria-label="Anchor">#</a></span></h2><p>Of all the books I read in October, this one scared me the most because it felt like it was entirely possible. There wasn&rsquo;t any part of the book that I looked at and said: &ldquo;Oh, that could never happen.&rdquo; That&rsquo;s what makes this one so good.</p> <p>It&rsquo;s suspenseful enough to keep you turning the pages but it&rsquo;s also plausible enough that you might find yourself wanting to wash your hands a little longer the next time you eat a meal. It feels a bit like <a href="https://www.goodreads.com/book/show/7624.Lord_of_the_Flies" target="_blank" rel="noreferrer">Lord of the Flies</a> mixed with a pandemic novel like <a href="https://www.goodreads.com/book/show/20170404-station-eleven" target="_blank" rel="noreferrer">Station Eleven</a> or <a href="https://www.goodreads.com/book/show/87591651-the-stand" target="_blank" rel="noreferrer">The Stand</a>.</p> <p>This one is also <em>very gruesome</em> in parts with some very difficult scenes to read.</p> <p>This one was my favorite of the group by far.</p> <h1 id="devolution-a-firsthand-account-of-the-rainier-sasquatch-massacre" class="relative group">Devolution: A Firsthand Account of the Rainier Sasquatch Massacre <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#devolution-a-firsthand-account-of-the-rainier-sasquatch-massacre" aria-label="Anchor">#</a></span></h1><p>Sasquatch is back in Max Brooks&rsquo; <a href="https://www.goodreads.com/book/show/52454426-devolution" target="_blank" rel="noreferrer">Devolution</a>! Told through the journals of a woman living in a remote town in Washington, this book covers a modern time where Mount Ranier erupted and caused lots of species to get on the move from their habitats around the mountain. Some of those creatures are the ones you&rsquo;d expect, but some are ones you won&rsquo;t expect.</p> <p>The small community is an experiment in green, off the grid living, and they are quickly tested by just about everything mother nature can throw at them. This includes some rather tall, furry, human-like creatures.</p> <h2 id="my-thoughts-3" class="relative group">My thoughts <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#my-thoughts-3" aria-label="Anchor">#</a></span></h2><p>This one felt scary due to the remoteness of the village and the unprepared people involved. Also, whether you believe in Sasquatch or not, this felt a bit more plausible than I expected.</p> <p>There were plenty of difficult to read scenes in here, but the gore was reduced compared to other books listed here. Much of the suspense came from how humans interact with one another especially when faced with an adversary that presents a unique set of challenges.</p> <p>This book was difficult to get into (it starts slow), but stick with it. It&rsquo;s a wild ride.</p> <h1 id="tender-is-the-flesh" class="relative group">Tender is the Flesh <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#tender-is-the-flesh" aria-label="Anchor">#</a></span></h1><p>Be prepared when you crack open Agustina Bazterrica&rsquo;s <a href="https://www.goodreads.com/book/show/49090884-tender-is-the-flesh" target="_blank" rel="noreferrer">Tender is the Flesh</a>. Imagine a world where animals somehow contract a virus that they quickly spread to each other and to humans as well. Pets are banned and animals near cities are killed. The pandemic puts a significant dent in the human population.</p> <p>However, with all of the animals either infected or gone, where do people get meat to eat? πŸ€”</p> <h2 id="my-thoughts-4" class="relative group">My thoughts <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#my-thoughts-4" aria-label="Anchor">#</a></span></h2><p>This was by far the most challenging book to read out of the group because I honestly felt like I was going to be sick in places. The author doesn&rsquo;t set out for cheap scares or basic unsettling events &ndash; there&rsquo;s something much deeper. It really makes you question how humanity operates and how quickly boundaries can shift when hunger becomes a problem.</p> <p>I had to take a lot of breaks with this book. It has some suspenseful parts but this book has a slow-burn horror feel that gives you hope, crushes that hope, and then starts the cycle once more.</p> <p>I strongly recommend this book <em>for adults only</em> but I feel terrible recommending it at the same time. πŸ˜„</p> <h1 id="whats-next" class="relative group">What&rsquo;s next? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#whats-next" aria-label="Anchor">#</a></span></h1><p>After all of that horror and unsettling fiction, I&rsquo;m shaking things up for November. My current book is Larry McMurtry&rsquo;s <a href="https://www.goodreads.com/book/show/256008.Lonesome_Dove" target="_blank" rel="noreferrer">Lonesome Dove</a>. So many people have recommended it to me as a great book about the American west and I&rsquo;m enjoying it so far.</p>Moving to cloud is more than just a purchasing exercisehttps://major.io/p/cloud-more-than-purchasing-exercise/Fri, 27 Oct 2023 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/cloud-more-than-purchasing-exercise/<p>Much of my work at Red Hat revolves around the RHEL experience in public clouds. I thrive on input from customers, partners, and coworkers about how they consume public clouds and why they made decisions to deploy there.</p> <p>Throughout this process, I run into some wild misconceptions about public clouds and what makes them useful. One that I hear most often is:</p> <blockquote> <p>Businesses are moving to the cloud to reduce cost and improve efficiency. It&rsquo;s mainly just a purchasing exercise.</p> </blockquote> <p>This couldn&rsquo;t be further from the truth.</p> <h1 id="cloud-offers-a-chance-to-start-over" class="relative group">Cloud offers a chance to start over <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#cloud-offers-a-chance-to-start-over" aria-label="Anchor">#</a></span></h1><p>Sometimes businesses find themselves in an IT quagmire<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. No matter what they do to improve their situation, it just gets worse. Capital expenditures grow and grow, datacenter space gets more expensive, and companies spend more time focusing on IT rather than their core business.</p> <p>Deploying in clouds offers that chance to break the capital expense cycle and gradually improve infrastructure. The key word here is <em>gradual</em>.</p> <p>Businesses can choose how much they want to deploy and when without worrying about expensive servers in the datacenter waiting to be used. Some deployments are greenfield, or entirely net new applications. Some are basic migrations of applications from servers or virtual machines directly to the cloud.</p> <p>Either way, businesses have the freedom to deploy as little or as much as they want on their own schedule.</p> <h1 id="cloud-offers-a-chance-to-software-define-nearly-anything" class="relative group">Cloud offers a chance to software-define (nearly) anything <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#cloud-offers-a-chance-to-software-define-nearly-anything" aria-label="Anchor">#</a></span></h1><p>Anyone who has worked in a large organization before knows the pain of change management. Sure, it ticks a box on that yearly compliance program, but it also ensures that everyone is aligned on the plan.</p> <p>One of the greatest aspects of cloud is that you can define almost everything in software. This makes changes easier to apply, easier to roll back, and easier to track.</p> <p>Tools like <a href="https://www.terraform.io/" target="_blank" rel="noreferrer">Terraform</a> or <a href="https://www.ansible.com/" target="_blank" rel="noreferrer">Ansible</a> allow developers and operations team to work from the same playbook. My team enjoys using <a href="https://www.infracost.io/" target="_blank" rel="noreferrer">Infracost</a> to track how much a particular Terraform change might cost us under different scenarios.</p> <p>Once teams set a policy of &ldquo;we define our changes in git, and that&rsquo;s it&rdquo;, you can rely on a git history for change management. This avoids drift in production environments and it also ensures that changes made in development environments make it into staging and then into production. The days of <em>&ldquo;it worked on my system, what&rsquo;s wrong with production?&rdquo;</em> slowly fade away.</p> <p>Less than ideal architectural decisions can also be adjusted over time to fit the applications being deployed. Did you set up a network incorrectly? Did you choose an instance type without enough RAM?</p> <p>That&rsquo;s okay!</p> <p>Just adjust the deployment in git, test it in staging, and push it to production.</p> <h1 id="cloud-offers-managed-services" class="relative group">Cloud offers managed services <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#cloud-offers-managed-services" aria-label="Anchor">#</a></span></h1><p>One thing I tell people constantly is that if you bend the cloud to fit your application, <strong>you will almost always pay more.</strong> You get cost and performance efficiencies if you bend your application to fit the cloud. Confused? I&rsquo;ll explain.</p> <p>When I talk to people doing their first cloud deployments, they deploy everything into VMs, much as they would in a local virtualized environment.</p> <ul> <li><em>You need a database server?</em> Make a couple of VMs and set up replication.</li> <li><em>You need to run a batch job via cron?</em> Deploy a VM and add it to the crontab.</li> <li><em>You need a server to export an NFS share?</em> Deploy a VM with lots of storage and export it to other instances.</li> </ul> <p>Do you see a pattern here?</p> <p> <figure> <picture class="mx-auto my-0 rounded-md" > <img width="480" height="270" class="mx-auto my-0 rounded-md" alt="too-much.gif" loading="lazy" decoding="async" src="https://major.io/p/cloud-more-than-purchasing-exercise/too-much.gif" /> </picture> </figure> </p> <p>Most public clouds offer tons of services that lift the management burden from engineering teams and offload into a managed service. For example, that cron job might be able to move into a &ldquo;serverless&rdquo;<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup> service, such as <a href="https://aws.amazon.com/lambda/" target="_blank" rel="noreferrer">AWS Lambda</a>. It&rsquo;s critical to check the pricing here to ensure you&rsquo;re not headed down a bad path, but you have one less VM to maintain, one less IPv4 address to pay for, and a greatly reduced risk of configuration drift. That reduction in stress and risk might be worth any additional costs.</p> <p>Deployment decisions become much easier and lower stress when you consume services offered by the provider. There are those situations where deploying a whole VM is needed, but I&rsquo;ve managed to avoid that for some of my team&rsquo;s recent deployments.</p> <p>Our last deployment uses GitHub Actions, S3, and CloudFront and costs us about $6.50 per month to run. There are no virtual machines. There&rsquo;s nothing to patch.</p> <p>This blog <a href="https://major.io/p/cloudfront-migration/">runs on a similar stack</a> and costs me about $0.25 per month to run.</p> <h1 id="cloud-offers-geographic-distribution" class="relative group">Cloud offers geographic distribution <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#cloud-offers-geographic-distribution" aria-label="Anchor">#</a></span></h1><p>Nearly every public cloud, even the smallest ones, offer you the same or similar services in a wide variety of geographic regions. Disaster recovery feels more attainable when you can easily deploy to multiple regions with the same software-defined infrastructure.</p> <p>Data sovereignty continues to grow in importance around the world as more countries demand that their data remains within their borders. As long as your cloud offers a region in that country, you can deploy there. There&rsquo;s no challenging legal issues with finding datacenter space or getting hardware delivered. You just change your region and deploy.</p> <p>Cloud regions also allow you to bring your applications much closer to the people who use them. Reduced latency delivers content faster to customers and provides a responsive experience.</p> <h1 id="clouds-offer-purchasing-efficiency" class="relative group">Clouds offer purchasing efficiency <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#clouds-offer-purchasing-efficiency" aria-label="Anchor">#</a></span></h1><p><strong>Wait a minute!</strong> Didn&rsquo;t I say that moving to cloud isn&rsquo;t just a purchasing exercise? πŸ€”</p> <p> <figure> <picture class="mx-auto my-0 rounded-md" > <img width="480" height="281" class="mx-auto my-0 rounded-md" alt="surprise.gif" loading="lazy" decoding="async" src="https://major.io/p/cloud-more-than-purchasing-exercise/surprise.gif" /> </picture> </figure> </p> <p>Your move to cloud should not be solely based on cutting costs or making purchasing IT more efficient. Most teams find that moving to cloud is more expensive than they anticipated because they&rsquo;re finally able to get access to the right amount of resources that they need. <em>(Also, they usually go with some more expensive options up front until they figure out how to optimize for cost.)</em></p> <p>First off, it&rsquo;s much easier to budget and pay one vendor for multiple services than deal with multiple independent vendors. Instead of paying for datacenter space, then paying for servers, then paying for network equipment, then paying for people to set it up, and so on, you pay the cloud provider for all of it.</p> <p>This also extends to other purchases on the cloud, such as products from certain vendors. For example, you can buy Red Hat products directly from some cloud providers and that gets added onto your cloud invoice. You can even deploy your own <a href="https://aws.amazon.com/marketplace/pp/prodview-sltshxd3bzqbg" target="_blank" rel="noreferrer">Cisco ASA in the cloud</a> if you feel so inclined.</p> <p>With all of these purchases going through one vendor, you can also negotiate discounts if you set a spending commitment. Discounts depend on your committed spend, of course, and the term that you agree to spend it. There&rsquo;s a whole industry around financial operations in the cloud, called <a href="https://www.finops.org/introduction/what-is-finops/" target="_blank" rel="noreferrer">FinOps</a>, and this is one of many things that factors into it.</p> <h1 id="wrapping-up" class="relative group">Wrapping up <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#wrapping-up" aria-label="Anchor">#</a></span></h1><p>Public clouds offer an incredible amount of opportunity to get your IT deployments into better shape with better change control and a solid software-defined workflow. They also offer the ability to &ldquo;write one check&rdquo; to consume infrastructure via utility billing.</p> <p><strong>However, public clouds are not ideal for every application or situation.</strong></p> <p>Do I think that <strong>every company</strong> in the world could benefit from getting some part of their IT deployments into a public cloud platform? <strong>Yes, I do.</strong></p> <p>Would <strong>every company</strong> benefit from putting most of their infrastructure into public clouds? <strong>Very unlikely.</strong></p> <p>Some applications still benefit from being on purpose-built hardware or in certain locations where a cloud might not exist today. Clouds can also be extremely expensive if you run large workloads around the clock. They can also be painful for applications with very strict or special requirements that don&rsquo;t fit a cloud deployment model well.</p> <p>The vendors that will succeed the most in the cloud space are the ones that look beyond purchasing efficiencies and IT acquisition concerns. Simply dragging the old world of physical servers or virtual machines into cloud won&rsquo;t lead anywhere.</p> <p>Those companies that help their customers <strong>benefit from the best of what public clouds have to offer</strong> in the most secure, reliable, and simple ways will be in the driver&rsquo;s seat.</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>A quagmire is something that gets worse no matter how you try to improve it. The only way to win is to avoid it entirely.&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p>Boy, I still dislike that <em>serverless</em> term so much. πŸ€¦β€β™‚οΈ&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>How I learned to stop worrying and love the CoreOShttps://major.io/p/why-coreos/Fri, 13 Oct 2023 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/why-coreos/<hr> <p>It&rsquo;s quite clear that I&rsquo;ve been on a <a href="https://major.io/tags/coreos/">CoreOS</a> blogging streak lately. I keep getting asked by people inside and outside my company about what makes CoreOS special and why I&rsquo;ve switched over so many workloads to it.</p> <p>The answer is pretty basic. <strong>It makes my life easier.</strong></p> <p>I&rsquo;m a Dad. I&rsquo;m on the PTC (Parent Teacher Club) at one of my children&rsquo;s schools. I volunteer as an IT person for a non-profit. I write software. I have other time consuming hobbies, such as ham radio, reading, and becoming a longer distance runner<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>.</p> <p>My available time for my own IT projects is <strong>extremely limited</strong> and CoreOS plays a part in keeping that part of my life as efficient as possible.</p> <p>That&rsquo;s what this blog post is about!</p> <h1 id="updates" class="relative group">Updates <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#updates" aria-label="Anchor">#</a></span></h1><p>First and foremost, I love how CoreOS does updates. I encourage you to <a href="https://docs.fedoraproject.org/en-US/fedora-coreos/auto-updates/" target="_blank" rel="noreferrer">read the docs</a> on this topic, but here&rsquo;s a short explanation:</p> <ol> <li>Updates are automatically retrieved and they&rsquo;re loaded into a slot.</li> <li>Your system reboots into the new update but your original OS tree remains in place.</li> <li>Did the update boot? Awesome. You&rsquo;re good to go.</li> <li>Did something break? The system reverts back to the known good tree.</li> </ol> <p>In this way, it&rsquo;s a lot like your smartphone.</p> <p>You have full control over when a node looks for an update and how often it checks for them. Check out the <a href="https://coreos.github.io/zincati/usage/updates-strategy/" target="_blank" rel="noreferrer">Zincati docs</a> for tons of controls over updates and reboots.</p> <p>Some of mine are timed so well that I set maintenance windows with my monitoring provider when I know an update might take place. The updates come through, monitoring shuts off, the node reboots, and monitoring comes back. The nodes almost always come back before the monitoring even alerts me.</p> <p>It also removes the reminders I would set for myself to update packages and run reboots. I know that my CoreOS nodes will do this automatically, so I don&rsquo;t need to think about it.</p> <p>Also, updates are rarely ever impactful to my workloads since all of them are running inside containers. My containers come right back up as soon as the node finishes its reboot.</p> <h1 id="toolbox" class="relative group">Toolbox <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#toolbox" aria-label="Anchor">#</a></span></h1><p>To be fair, you can get toolbox running on lots of different Linux distributions outside of CoreOS, but that&rsquo;s the first place I ever used it. Toolbox, also called <a href="https://containertoolbx.org/" target="_blank" rel="noreferrer">toolbx</a>, gives you a utility container on your CoreOS node for all kinds of adminsitrative and diagnostic capabilities.</p> <p>You might need a certain package for diagnosting a hardware issue or you might want to install some helpful utilities for the command line. Do that in a toolbox container. Just run <code>toolbox enter</code> and if you&rsquo;ve never created a toolbox container before, you&rsquo;ll get a Fedora container that matches your CoreOS release.</p> <p>But it gets better.</p> <p>Toolbox automatically saves your container when you&rsquo;re done with it so all of your installed packages stay there for next time. Also, these containers have seamless access to anything you have in your home directory, including sockets. You&rsquo;re running inside a container, but it&rsquo;s almost like you&rsquo;re running on the host itself inside your home directory. You get the best of both worlds.</p> <p>Don&rsquo;t want Fedora? You have lots of distribution options through toolbox. Read the details on <a href="https://containertoolbx.org/install/" target="_blank" rel="noreferrer">custom images</a> to create your own!</p> <h1 id="layering" class="relative group">Layering <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#layering" aria-label="Anchor">#</a></span></h1><p>Okay, there are those situations where you really want a package on CoreOS and toolbox might not be sufficient. My muscle memory for <code>vim</code> is so strong and CoreOS only comes with <code>vi</code>.</p> <p>You have a couple of options here:</p> <ul> <li>Run <code>sudo rpm-ostree install vim</code>, reboot, and you have <code>vim</code></li> <li>Run <code>sudo rpm-ostree install --apply-live vim</code> and you have <code>vim</code> right now! <em>(And it&rsquo;s there after a reboot as well.)</em></li> </ul> <p>When a new update comes down for the base OS from CoreOS, any packages you&rsquo;ve added will be layered on the base image and available after a reboot. Layering is generally chosen as a last resort option for adding packages to the system but you shouldn&rsquo;t run into issues if you&rsquo;re installing small utilities or command line tools.</p> <h1 id="declarative-provisioning" class="relative group">Declarative provisioning <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#declarative-provisioning" aria-label="Anchor">#</a></span></h1><p>If you&rsquo;ve provisioned Linux distributions on cloud instances in the past, you&rsquo;ve likely provided metadata that cloud-init uses to provision your system. CoreOS has something that acts a lot earlier in the boot process and has more power to get things done: <a href="https://coreos.github.io/ignition/" target="_blank" rel="noreferrer">ignition</a>.</p> <p>There&rsquo;s a handy <a href="https://coreos.github.io/butane/" target="_blank" rel="noreferrer">butane</a> file forma that you use for writing your configuration. You use the <code>butane</code> utility to get it into ignition format. The ignition format is highly compressed to ensure you can fit your configuration into most cloud providers&rsquo; metadata fields.</p> <p>For a real example of what you can do with ignition, check out my <a href="https://major.io/p/quadlets-replace-docker-compose/">quadlets post</a> where I provisioned an entire Wordpress container stack using a single ignition file.</p> <p>There&rsquo;s lots of documentation for <a href="https://docs.fedoraproject.org/en-US/fedora-coreos/producing-ign/" target="_blank" rel="noreferrer">writing butane configuration</a> files for common situations. It&rsquo;s easy to add files, configure Wireguard, set up users, and launch containers immediately on the first boot.</p> <h1 id="pets-and-cattle" class="relative group">Pets and cattle <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#pets-and-cattle" aria-label="Anchor">#</a></span></h1><p>CoreOS works well for systems that I only need online for a short time. These might be situations where I need to test a few containers and throw it away. There&rsquo;s no OS to mess with and no updates to worry about.</p> <p>It also works well for systems that I keep online for a long time. I have a few physical systems at home that run CoreOS and they&rsquo;ve been extremely stable. I also have cloud instances on Hetzner, VULTR, and Digital Ocean that have run CoreOS for months without issues.</p> <h1 id="more-questions" class="relative group">More questions? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#more-questions" aria-label="Anchor">#</a></span></h1><p>Feel free to <a href="mailto:major&#43;coreos@mhtx.net">send me an email</a> or <a href="https://social.lol/@major" target="_blank" rel="noreferrer">drop me a toot on Mastodon</a>. I&rsquo;ll update this post if I get some good ones!</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>Completing a half marathon without keeling over is the current goal! πŸ‘Ÿ&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>Quadlets might make me finally stop using docker-composehttps://major.io/p/quadlets-replace-docker-compose/Mon, 25 Sep 2023 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/quadlets-replace-docker-compose/<hr> <p>I&rsquo;ve <a href="https://major.io/tags/containers/">written a lot about containers</a> on this blog. Why do I love containers so much?</p> <ul> <li>They start quickly</li> <li>They make your workloads portable</li> <li>They disconnect your application stack from the OS that runs underneath</li> <li>You can send your application through CI as a single container image</li> <li>You can isolate workloads on the network and limit their resource usage much like a VM</li> </ul> <p>However, I&rsquo;m still addicted to <a href="https://docs.docker.com/compose/" target="_blank" rel="noreferrer">docker-compose</a>. Can podman&rsquo;s <a href="https://www.redhat.com/sysadmin/quadlet-podman" target="_blank" rel="noreferrer">quadlets</a> change that?</p> <p><strong>Yes, I think they can.</strong></p> <h1 id="whats-a-quadlet" class="relative group">What&rsquo;s a quadlet? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#whats-a-quadlet" aria-label="Anchor">#</a></span></h1><p>Podman introduced support for quadlets in version 4.4 and it&rsquo;s a simpler way of letting systemd manage your containers. There was an option in the past to have podman generate systemd unit files, but those were unwieldy and full of podman command line options inside a unit file. These unit files weren&rsquo;t easy to edit or even parse with eyeballs.</p> <p>Quadlets make this easier by giving you a simple ini-style file that you can easily read and edit. This blog post will include some quadlets later, but here&rsquo;s an example one for Wordpress:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="k">[Unit]</span> </span></span><span class="line"><span class="cl"><span class="na">Description</span><span class="o">=</span><span class="s">Wordpress Quadlet</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">[Container]</span> </span></span><span class="line"><span class="cl"><span class="na">Image</span><span class="o">=</span><span class="s">docker.io/library/wordpress:fpm</span> </span></span><span class="line"><span class="cl"><span class="na">ContainerName</span><span class="o">=</span><span class="s">wordpress</span> </span></span><span class="line"><span class="cl"><span class="na">AutoUpdate</span><span class="o">=</span><span class="s">registry</span> </span></span><span class="line"><span class="cl"><span class="na">EnvironmentFile</span><span class="o">=</span><span class="s">/home/core/.config/containers/containers-environment</span> </span></span><span class="line"><span class="cl"><span class="na">Volume</span><span class="o">=</span><span class="s">wordpress.volume:/var/www/html</span> </span></span><span class="line"><span class="cl"><span class="na">Network</span><span class="o">=</span><span class="s">wordpress.network</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">[Service]</span> </span></span><span class="line"><span class="cl"><span class="na">Restart</span><span class="o">=</span><span class="s">always</span> </span></span><span class="line"><span class="cl"><span class="na">TimeoutStartSec</span><span class="o">=</span><span class="s">900</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">[Install]</span> </span></span><span class="line"><span class="cl"><span class="na">WantedBy</span><span class="o">=</span><span class="s">caddy.service multi-user.target default.target</span> </span></span></code></pre></div><p>Lots of the lines under <code>[Container]</code> should look familiar to most readers who have worked with containers before. However, there&rsquo;s something new here.</p> <p>Check out the <code>AutoUpdate=registry</code> line. This tells podman to keep your container updated on a regular basis with the upstream container registry. I&rsquo;ve used <a href="https://major.io/p/podman-quadlet-watchtower/">watchtower</a> in the past for this, but it requires a privileged container and it&rsquo;s yet another external dependency.</p> <p>Also, at the very end, you&rsquo;ll see a <code>WantedBy</code> line. This is a great place to set up container dependencies. In this example, the container that runs <code>caddy</code> (a web server) can&rsquo;t start until Wordpress is up and running.</p> <h1 id="so-why-not-stick-with-docker-compose" class="relative group">So why not stick with docker-compose? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#so-why-not-stick-with-docker-compose" aria-label="Anchor">#</a></span></h1><p>There&rsquo;s no denying that docker-compose is an awesome tool. You specify the desired outcome, tell it to bring up containers, and it gets containers into the state you specified. It handles volumes, networks, and complicated configuration without a lot of legwork. The YAML files are pretty easy to read, too.</p> <p>However, as with watchtower, that&rsquo;s another external dependency.</p> <p>My container deployments are often done at instance boot time and I don&rsquo;t make too many changes afterwards. I found myself using docker-compose for the initial deployment and then I didn&rsquo;t really use it again.</p> <p>Why not remove it entirely and use what&rsquo;s built into CoreOS already?</p> <h1 id="quaint-quadlets-quickly" class="relative group">Quaint quadlets quickly! <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#quaint-quadlets-quickly" aria-label="Anchor">#</a></span></h1><p>Before we start, we&rsquo;re going to need a few things:</p> <ul> <li>An easy to read <a href="https://coreos.github.io/butane/" target="_blank" rel="noreferrer">butane</a> configuration which gets transformed into a tiny <a href="https://coreos.github.io/ignition/" target="_blank" rel="noreferrer">ignition</a> configuration for CoreOS</li> <li>Some quadlets</li> <li>Extra system configuration</li> <li>A cloud provider with CoreOS images <em>(using <a href="https://www.vultr.com/?ref=9544589-8H" target="_blank" rel="noreferrer">VULTR</a> for this)</em></li> </ul> <p>I&rsquo;ve packed all of these items into my <a href="https://github.com/major/quadlets-wordpress" target="_blank" rel="noreferrer">quadlets-wordpress</a> repository to make it easy. Start by looking at the <a href="https://github.com/major/quadlets-wordpress/blob/main/config.butane" target="_blank" rel="noreferrer">config.butane</a> file.</p> <p>Let&rsquo;s break it down here. First up, we add an ssh key for the default <code>core</code> user.</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">variant</span><span class="p">:</span><span class="w"> </span><span class="l">fcos</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">version</span><span class="p">:</span><span class="w"> </span><span class="m">1.5.0</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">passwd</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">users</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">ssh_authorized_keys</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="l">ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDyoH6gU4lgEiSiwihyD0Rxk/o5xYIfA3stVDgOGM9N0</span><span class="w"> </span></span></span></code></pre></div><p>Next up, we enable the <code>podman-auto-update.timer</code> so we get container updates automatically:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">storage</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">links</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/home/core/.config/systemd/user/timers.target.wants/podman-auto-update.timer</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">target</span><span class="p">:</span><span class="w"> </span><span class="l">/usr/lib/systemd/user/podman-auto-update.timer</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">user</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">group</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span></code></pre></div><p>Next is the long <code>files</code> section:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="w"> </span><span class="nt">files</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># Ensure the `core` user can keep processes running after they&#39;re logged out.</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/var/lib/systemd/linger/core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">mode</span><span class="p">:</span><span class="w"> </span><span class="m">0644</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># Allow caddy to listen on 80 and 443.</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># Allow it to ask for bigger network buffers, too.</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/etc/sysctl.d/90-caddy.conf</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">contents</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">inline</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd"> </span></span></span><span class="line"><span class="cl"><span class="sd"> net.ipv4.ip_unprivileged_port_start = 80 </span></span></span><span class="line"><span class="cl"><span class="sd"> net.core.rmem_max=2500000 </span></span></span><span class="line"><span class="cl"><span class="sd"> net.core.wmem_max=2500000</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># Set up an an environment file that containers can read to configure themselves.</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/home/core/.config/containers/containers-environment</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">contents</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">inline</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd"> </span></span></span><span class="line"><span class="cl"><span class="sd"> MYSQL_DATABASE=wordpress </span></span></span><span class="line"><span class="cl"><span class="sd"> MYSQL_USER=wordpress </span></span></span><span class="line"><span class="cl"><span class="sd"> MYSQL_ROOT_PASSWORD=mariadb-needs-a-secure-password </span></span></span><span class="line"><span class="cl"><span class="sd"> MYSQL_PASSWORD=wordpress-needs-a-secure-password </span></span></span><span class="line"><span class="cl"><span class="sd"> WORDPRESS_DB_HOST=mariadb </span></span></span><span class="line"><span class="cl"><span class="sd"> WORDPRESS_DB_USER=wordpress </span></span></span><span class="line"><span class="cl"><span class="sd"> WORDPRESS_DB_PASSWORD=wordpress-needs-a-secure-password </span></span></span><span class="line"><span class="cl"><span class="sd"> WORDPRESS_DB_NAME=wordpress</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">mode</span><span class="p">:</span><span class="w"> </span><span class="m">0644</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># Deploy the caddy configuration file from the repository.</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/home/core/.config/caddy/Caddyfile</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">contents</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">local</span><span class="p">:</span><span class="w"> </span><span class="l">caddy/Caddyfile</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">mode</span><span class="p">:</span><span class="w"> </span><span class="m">0644</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">user</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">group</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># Add some named volumes for caddy and wordpress.</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/home/core/.config/containers/systemd/caddy-config.volume</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">contents</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">inline</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd"> </span></span></span><span class="line"><span class="cl"><span class="sd"> [Volume]</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">user</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">group</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/home/core/.config/containers/systemd/caddy-data.volume</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">contents</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">inline</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd"> </span></span></span><span class="line"><span class="cl"><span class="sd"> [Volume]</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">user</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">group</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/home/core/.config/containers/systemd/wordpress.volume</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">contents</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">inline</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd"> </span></span></span><span class="line"><span class="cl"><span class="sd"> [Volume]</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">user</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">group</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># Create a network for all the containers to use and enable the</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># DNS plugin. This allows containers to find each other using</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># the container names.</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/home/core/.config/containers/systemd/wordpress.network</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">contents</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">inline</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd"> </span></span></span><span class="line"><span class="cl"><span class="sd"> [Network] </span></span></span><span class="line"><span class="cl"><span class="sd"> DisableDNS=false </span></span></span><span class="line"><span class="cl"><span class="sd"> Internal=false</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">user</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">group</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># Add the wordpress container.</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/home/core/.config/containers/systemd/wordpress.container</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">contents</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">local</span><span class="p">:</span><span class="w"> </span><span class="l">quadlets/wordpress.container</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">mode</span><span class="p">:</span><span class="w"> </span><span class="m">0644</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">user</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">group</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># Add the MariaDB container.</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/home/core/.config/containers/systemd/mariadb.container</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">contents</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">local</span><span class="p">:</span><span class="w"> </span><span class="l">quadlets/mariadb.container</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">mode</span><span class="p">:</span><span class="w"> </span><span class="m">0644</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">user</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">group</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="c"># Add the caddy container.</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span>- <span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/home/core/.config/containers/systemd/caddy.container</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">contents</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">local</span><span class="p">:</span><span class="w"> </span><span class="l">quadlets/caddy.container</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">mode</span><span class="p">:</span><span class="w"> </span><span class="m">0644</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">user</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">group</span><span class="p">:</span><span class="w"> </span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">core</span><span class="w"> </span></span></span></code></pre></div><p>The <a href="https://github.com/major/quadlets-wordpress/blob/main/caddy/Caddyfile" target="_blank" rel="noreferrer">Caddyfile</a> is also in the repository and will be deployed by the butane configuration shown above.</p> <p>We can go through each quadlet in detail. First up is MariaDB. We tell systemd that the wordpress container will want to have this one started first.</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="k">[Unit]</span> </span></span><span class="line"><span class="cl"><span class="na">Description</span><span class="o">=</span><span class="s">MariaDB Quadlet</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">[Container]</span> </span></span><span class="line"><span class="cl"><span class="na">Image</span><span class="o">=</span><span class="s">docker.io/library/mariadb:11</span> </span></span><span class="line"><span class="cl"><span class="na">ContainerName</span><span class="o">=</span><span class="s">mariadb</span> </span></span><span class="line"><span class="cl"><span class="na">AutoUpdate</span><span class="o">=</span><span class="s">registry</span> </span></span><span class="line"><span class="cl"><span class="na">EnvironmentFile</span><span class="o">=</span><span class="s">/home/core/.config/containers/containers-environment</span> </span></span><span class="line"><span class="cl"><span class="na">Volume</span><span class="o">=</span><span class="s">mariadb.volume:/var/lib/mysql</span> </span></span><span class="line"><span class="cl"><span class="na">Network</span><span class="o">=</span><span class="s">wordpress.network</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">[Service]</span> </span></span><span class="line"><span class="cl"><span class="na">Restart</span><span class="o">=</span><span class="s">always</span> </span></span><span class="line"><span class="cl"><span class="na">TimeoutStartSec</span><span class="o">=</span><span class="s">900</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">[Install]</span> </span></span><span class="line"><span class="cl"><span class="na">WantedBy</span><span class="o">=</span><span class="s">wordpress.service multi-user.target default.target</span> </span></span></code></pre></div><p>The wordpress quadlet is much the same as the MariaDB one, but we tell systemd that caddy will want wordpress started first.</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="k">[Unit]</span> </span></span><span class="line"><span class="cl"><span class="na">Description</span><span class="o">=</span><span class="s">Wordpress Quadlet</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">[Container]</span> </span></span><span class="line"><span class="cl"><span class="na">Image</span><span class="o">=</span><span class="s">docker.io/library/wordpress:fpm</span> </span></span><span class="line"><span class="cl"><span class="na">ContainerName</span><span class="o">=</span><span class="s">wordpress</span> </span></span><span class="line"><span class="cl"><span class="na">AutoUpdate</span><span class="o">=</span><span class="s">registry</span> </span></span><span class="line"><span class="cl"><span class="na">EnvironmentFile</span><span class="o">=</span><span class="s">/home/core/.config/containers/containers-environment</span> </span></span><span class="line"><span class="cl"><span class="na">Volume</span><span class="o">=</span><span class="s">wordpress.volume:/var/www/html</span> </span></span><span class="line"><span class="cl"><span class="na">Network</span><span class="o">=</span><span class="s">wordpress.network</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">[Service]</span> </span></span><span class="line"><span class="cl"><span class="na">Restart</span><span class="o">=</span><span class="s">always</span> </span></span><span class="line"><span class="cl"><span class="na">TimeoutStartSec</span><span class="o">=</span><span class="s">900</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">[Install]</span> </span></span><span class="line"><span class="cl"><span class="na">WantedBy</span><span class="o">=</span><span class="s">caddy.service multi-user.target default.target</span> </span></span></code></pre></div><p>Finally, the caddy quadlet contains four volumes and some published ports. These ports will be published to the container host. Also, you&rsquo;ll note that the wordpress volume is mounted here, too. This is because caddy can serve static files <em>much faster</em> than wordpress can.</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="k">[Unit]</span> </span></span><span class="line"><span class="cl"><span class="na">Description</span><span class="o">=</span><span class="s">Caddy Quadlet</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">[Container]</span> </span></span><span class="line"><span class="cl"><span class="na">Image</span><span class="o">=</span><span class="s">docker.io/library/caddy:latest</span> </span></span><span class="line"><span class="cl"><span class="na">ContainerName</span><span class="o">=</span><span class="s">caddy</span> </span></span><span class="line"><span class="cl"><span class="na">AutoUpdate</span><span class="o">=</span><span class="s">registry</span> </span></span><span class="line"><span class="cl"><span class="na">EnvironmentFile</span><span class="o">=</span><span class="s">/home/core/.config/containers/containers-environment</span> </span></span><span class="line"><span class="cl"><span class="na">Volume</span><span class="o">=</span><span class="s">caddy-data.volume:/data</span> </span></span><span class="line"><span class="cl"><span class="na">Volume</span><span class="o">=</span><span class="s">caddy-config.volume:/config</span> </span></span><span class="line"><span class="cl"><span class="na">Volume</span><span class="o">=</span><span class="s">/home/core/.config/caddy/Caddyfile:/etc/caddy/Caddyfile:Z</span> </span></span><span class="line"><span class="cl"><span class="na">Volume</span><span class="o">=</span><span class="s">wordpress.volume:/var/www/html</span> </span></span><span class="line"><span class="cl"><span class="na">PublishPort</span><span class="o">=</span><span class="s">80:80</span> </span></span><span class="line"><span class="cl"><span class="na">PublishPort</span><span class="o">=</span><span class="s">443:443</span> </span></span><span class="line"><span class="cl"><span class="na">Network</span><span class="o">=</span><span class="s">wordpress.network</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">[Service]</span> </span></span><span class="line"><span class="cl"><span class="na">Restart</span><span class="o">=</span><span class="s">always</span> </span></span><span class="line"><span class="cl"><span class="na">TimeoutStartSec</span><span class="o">=</span><span class="s">900</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="k">[Install]</span> </span></span><span class="line"><span class="cl"><span class="na">WantedBy</span><span class="o">=</span><span class="s">multi-user.target default.target</span> </span></span></code></pre></div><h1 id="launch-the-quadlets" class="relative group">Launch the quadlets <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#launch-the-quadlets" aria-label="Anchor">#</a></span></h1><p>There&rsquo;s a <a href="https://github.com/major/quadlets-wordpress/blob/main/launch-instance" target="_blank" rel="noreferrer">launch script</a> that ships this configuration to VULTR and launches a CoreOS instance:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl"><span class="cp">#!/bin/bash </span></span></span><span class="line"><span class="cl"><span class="cp"></span><span class="c1"># This command starts up a CoreOS instance on Vultr using the vultr-cli</span> </span></span><span class="line"><span class="cl">vultr-cli instance create <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> --os <span class="m">391</span> <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> --plan vhp-1c-1gb-amd <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> --region dfw <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> --notify <span class="nb">true</span> <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> --ipv6 <span class="nb">true</span> <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> -u <span class="s2">&#34;</span><span class="k">$(</span>butane --files-dir . config.butane<span class="k">)</span><span class="s2">&#34;</span> <span class="se">\ </span></span></span><span class="line"><span class="cl"><span class="se"></span> -l <span class="s2">&#34;coreos-</span><span class="k">$(</span>date <span class="s2">&#34;+%s&#34;</span><span class="k">)</span><span class="s2">&#34;</span> </span></span></code></pre></div><p>To launch an instance, get your <a href="https://my.vultr.com/settings/#settingsapi" target="_blank" rel="noreferrer">VULTR API key</a> first. Then install vultr-cli and butane:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo dnf -y install butane vultr-cli </span></span></code></pre></div><p>After launch, check to see what your containers are doing:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">[core@vultr ~]$</span> podman ps </span></span><span class="line"><span class="cl"><span class="go">CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES </span></span></span><span class="line"><span class="cl"><span class="go">afa2d6501593 docker.io/library/caddy:latest caddy run --confi... 54 seconds ago Up 53 seconds 0.0.0.0:80-&gt;80/tcp, 0.0.0.0:443-&gt;443/tcp caddy </span></span></span><span class="line"><span class="cl"><span class="go">460426f39e6c docker.io/library/mariadb:11 mariadbd 35 seconds ago Up 35 seconds mariadb </span></span></span><span class="line"><span class="cl"><span class="go">92ece6538d5a docker.io/library/wordpress:fpm php-fpm 28 seconds ago Up 29 seconds wordpress </span></span></span></code></pre></div><p>We should be able to talk to wordpress through caddy on port 80:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">[core@vultr ~]$</span> curl -si http://localhost/wp-admin/install.php <span class="p">|</span> head -n <span class="m">25</span> </span></span><span class="line"><span class="cl"><span class="go">HTTP/1.1 200 OK </span></span></span><span class="line"><span class="cl"><span class="go">Cache-Control: no-cache, must-revalidate, max-age=0 </span></span></span><span class="line"><span class="cl"><span class="go">Content-Type: text/html; charset=utf-8 </span></span></span><span class="line"><span class="cl"><span class="go">Expires: Wed, 11 Jan 1984 05:00:00 GMT </span></span></span><span class="line"><span class="cl"><span class="go">Server: Caddy </span></span></span><span class="line"><span class="cl"><span class="go">X-Powered-By: PHP/8.0.30 </span></span></span><span class="line"><span class="cl"><span class="go">Date: Mon, 25 Sep 2023 21:43:40 GMT </span></span></span><span class="line"><span class="cl"><span class="go">Transfer-Encoding: chunked </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go">&lt;!DOCTYPE html&gt; </span></span></span><span class="line"><span class="cl"><span class="go">&lt;html lang=&#34;en-US&#34; xml:lang=&#34;en-US&#34;&gt; </span></span></span><span class="line"><span class="cl"><span class="go">&lt;head&gt; </span></span></span><span class="line"><span class="cl"><span class="go"> &lt;meta name=&#34;viewport&#34; content=&#34;width=device-width&#34; /&gt; </span></span></span><span class="line"><span class="cl"><span class="go"> &lt;meta http-equiv=&#34;Content-Type&#34; content=&#34;text/html; charset=utf-8&#34; /&gt; </span></span></span><span class="line"><span class="cl"><span class="go"> &lt;meta name=&#34;robots&#34; content=&#34;noindex,nofollow&#34; /&gt; </span></span></span><span class="line"><span class="cl"><span class="go"> &lt;title&gt;WordPress &amp;rsaquo; Installation&lt;/title&gt; </span></span></span><span class="line"><span class="cl"><span class="go"> &lt;link rel=&#39;stylesheet&#39; id=&#39;dashicons-css&#39; href=&#39;http://localhost/wp-includes/css/dashicons.min.css?ver=6.3.1&#39; type=&#39;text/css&#39; media=&#39;all&#39; /&gt; </span></span></span><span class="line"><span class="cl"><span class="go">&lt;link rel=&#39;stylesheet&#39; id=&#39;buttons-css&#39; href=&#39;http://localhost/wp-includes/css/buttons.min.css?ver=6.3.1&#39; type=&#39;text/css&#39; media=&#39;all&#39; /&gt; </span></span></span><span class="line"><span class="cl"><span class="go">&lt;link rel=&#39;stylesheet&#39; id=&#39;forms-css&#39; href=&#39;http://localhost/wp-admin/css/forms.min.css?ver=6.3.1&#39; type=&#39;text/css&#39; media=&#39;all&#39; /&gt; </span></span></span><span class="line"><span class="cl"><span class="go">&lt;link rel=&#39;stylesheet&#39; id=&#39;l10n-css&#39; href=&#39;http://localhost/wp-admin/css/l10n.min.css?ver=6.3.1&#39; type=&#39;text/css&#39; media=&#39;all&#39; /&gt; </span></span></span><span class="line"><span class="cl"><span class="go">&lt;link rel=&#39;stylesheet&#39; id=&#39;install-css&#39; href=&#39;http://localhost/wp-admin/css/install.min.css?ver=6.3.1&#39; type=&#39;text/css&#39; media=&#39;all&#39; /&gt; </span></span></span><span class="line"><span class="cl"><span class="go">&lt;/head&gt; </span></span></span><span class="line"><span class="cl"><span class="go">&lt;body class=&#34;wp-core-ui language-chooser&#34;&gt; </span></span></span><span class="line"><span class="cl"><span class="go">&lt;p id=&#34;logo&#34;&gt;WordPress&lt;/p&gt; </span></span></span></code></pre></div><p>Awesome! πŸŽ‰</p> <h1 id="managing-containers" class="relative group">Managing containers <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#managing-containers" aria-label="Anchor">#</a></span></h1><p>Containers will automatically update on a schedule and you can check the timer:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">[core@vultr ~]$</span> systemctl status --user podman-auto-update.timer </span></span><span class="line"><span class="cl"><span class="go">● podman-auto-update.timer - Podman auto-update timer </span></span></span><span class="line"><span class="cl"><span class="go"> Loaded: loaded (/usr/lib/systemd/user/podman-auto-update.timer; enabled; preset: disabled) </span></span></span><span class="line"><span class="cl"><span class="go"> Active: active (waiting) since Mon 2023-09-25 21:41:31 UTC; 3min 14s ago </span></span></span><span class="line"><span class="cl"><span class="go"> Trigger: Tue 2023-09-26 00:04:46 UTC; 2h 20min left </span></span></span><span class="line"><span class="cl"><span class="go"> Triggers: ● podman-auto-update.service </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go">Sep 25 21:41:31 vultr.guest systemd[1786]: Started podman-auto-update.timer - Podman auto-update timer. </span></span></span></code></pre></div><p>Quadlets are just regular systemd units:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">[core@vultr ~]$</span> systemctl list-units --user <span class="p">|</span> grep -i Quadlet </span></span><span class="line"><span class="cl"><span class="go"> caddy.service loaded active running Caddy Quadlet </span></span></span><span class="line"><span class="cl"><span class="go"> mariadb.service loaded active running MariaDB Quadlet </span></span></span><span class="line"><span class="cl"><span class="go"> wordpress.service loaded active running Wordpress Quadlet </span></span></span></code></pre></div><p>As an example, you can make changes to caddy&rsquo;s config file and restart it easily:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">[core@vultr ~]$</span> systemctl restart --user caddy </span></span><span class="line"><span class="cl"><span class="gp">[core@vultr ~]$</span> systemctl status --user caddy </span></span><span class="line"><span class="cl"><span class="go">● caddy.service - Caddy Quadlet </span></span></span><span class="line"><span class="cl"><span class="go"> Loaded: loaded (/var/home/core/.config/containers/systemd/caddy.container; generated) </span></span></span><span class="line"><span class="cl"><span class="go"> Drop-In: /usr/lib/systemd/user/service.d </span></span></span><span class="line"><span class="cl"><span class="go"> └─10-timeout-abort.conf </span></span></span><span class="line"><span class="cl"><span class="go"> Active: active (running) since Mon 2023-09-25 21:46:28 UTC; 5s ago </span></span></span><span class="line"><span class="cl"><span class="go"> Main PID: 2652 (conmon) </span></span></span><span class="line"><span class="cl"><span class="go"> Tasks: 18 (limit: 1023) </span></span></span><span class="line"><span class="cl"><span class="go"> Memory: 15.1M </span></span></span><span class="line"><span class="cl"><span class="go"> CPU: 207ms </span></span></span></code></pre></div><p>If you need to change a quadlet&rsquo;s configuration, just open up the configuration file in your favorite editor under <code>~/.config/containers/systemd</code>, reload systemd, and restart the container:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> vi ~/.config/containers/systemd/caddy.container </span></span><span class="line"><span class="cl"><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go">--- make your edits and save the quadlet configuration --- </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> systemctl daemon-reload --user </span></span><span class="line"><span class="cl"><span class="gp">$</span> systemctl restart --user caddy </span></span></code></pre></div><p>Enjoy!</p>Mounting the AWS Elastic File Store on Fedorahttps://major.io/p/aws-elastic-file-system-fedora/Wed, 13 Sep 2023 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/aws-elastic-file-system-fedora/<hr> <p>I package a few things here and there in <a href="https://fedoraproject.org/" target="_blank" rel="noreferrer">Fedora</a> and one of my latest packages is <a href="https://src.fedoraproject.org/rpms/efs-utils" target="_blank" rel="noreferrer">efs-utils</a>. AWS offers a mount helper for their <a href="https://aws.amazon.com/efs/" target="_blank" rel="noreferrer">Elastic File System (EFS)</a> product on <a href="https://github.com/aws/efs-utils" target="_blank" rel="noreferrer">GitHub</a>.</p> <p>In this post, I&rsquo;ll explain how to:</p> <ol> <li>Launch a Fedora instance on AWS EC2</li> <li>Install <em>efs-utils</em> and launch the watchdog service</li> <li>Create an EFS volume in the AWS console</li> <li>Mount the EFS volume inside the Fedora instance</li> </ol> <div class="flex rounded-md bg-primary-100 px-4 py-3 dark:bg-primary-900"> <span class="pe-3 text-primary-400"> <span class="icon relative inline-block px-1 align-text-bottom"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><path fill="currentColor" d="M368 128C368 172.4 342.6 211.5 304 234.4V256C304 273.7 289.7 288 272 288H176C158.3 288 144 273.7 144 256V234.4C105.4 211.5 80 172.4 80 128C80 57.31 144.5 0 224 0C303.5 0 368 57.31 368 128V128zM168 176C185.7 176 200 161.7 200 144C200 126.3 185.7 112 168 112C150.3 112 136 126.3 136 144C136 161.7 150.3 176 168 176zM280 112C262.3 112 248 126.3 248 144C248 161.7 262.3 176 280 176C297.7 176 312 161.7 312 144C312 126.3 297.7 112 280 112zM3.379 273.7C11.28 257.9 30.5 251.5 46.31 259.4L224 348.2L401.7 259.4C417.5 251.5 436.7 257.9 444.6 273.7C452.5 289.5 446.1 308.7 430.3 316.6L295.6 384L430.3 451.4C446.1 459.3 452.5 478.5 444.6 494.3C436.7 510.1 417.5 516.5 401.7 508.6L224 419.8L46.31 508.6C30.5 516.5 11.28 510.1 3.379 494.3C-4.525 478.5 1.882 459.3 17.69 451.4L152.4 384L17.69 316.6C1.882 308.7-4.525 289.5 3.379 273.7V273.7z"/></svg> </span> </span> <span class="dark:text-neutral-300"><strong>Always check the pricing for any cloud service before you use it!</strong> <a href="https://aws.amazon.com/efs/pricing/" target="_blank" rel="noreferrer">EFS pricing</a> is based on how much you store and how often you access it. Backups are also enabled by default and they add to the monthly charges.</span> </div> <p>Let&rsquo;s go! πŸš€</p> <h1 id="wait-what-is-efs" class="relative group">Wait, what is EFS? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#wait-what-is-efs" aria-label="Anchor">#</a></span></h1><p>When you launch a cloud instance (virtual machine) on most clouds, you have different storage options available to you:</p> <ul> <li> <p><strong>Block storage:</strong> You can add partitions to this storage, create filesystems, or even use LVM. It looks like someone plugged in a disk to your instance. You get full control over every single storage block on the volume. An example of this is <a href="https://aws.amazon.com/ebs/" target="_blank" rel="noreferrer">Elastic Block Storage (EBS)</a> on AWS.</p> </li> <li> <p><strong>Object storage:</strong> Although you can&rsquo;t mount object storage (typically) within your instance, you can read/write objects to this storage via an API. You can upload nearly any type of file you can imagine as an object and then download it later. Objects can also have little bits of metadata attached to them and some of the metadata include <em>prefixes</em> which give a folder-like experience. AWS <a href="https://aws.amazon.com/s3/" target="_blank" rel="noreferrer">S3</a> is a good example of this.</p> </li> <li> <p><strong>Shared filesystems:</strong> This storage shows up in the instance exactly as it sounds: you get a shared filesystem. If you&rsquo;re familiar with NFS or Samba (SMB), then you&rsquo;ve used shared filesystems already. They give you much better performance than object storage but offer less freedom than block storage. They&rsquo;re also great for sharing the same data between multiple instances.</p> </li> </ul> <p>Using EFS is almost like having someone else host a network accessible storage (NAS) device within your cloud deployment.</p> <h1 id="launching-fedora" class="relative group">Launching Fedora <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#launching-fedora" aria-label="Anchor">#</a></span></h1><p>Every image in AWS has an AMI ID attached to it and you need to know the ID for the image you want in your region. You can find these quickly for Fedora by visiting the <a href="https://fedoraproject.org/cloud/download/" target="_blank" rel="noreferrer">Fedora Cloud download page</a>. Look for <em>AWS</em> in the list, click the button on that row, and you&rsquo;ll see a list of Fedora AMI IDs. Click the rocket (πŸš€) for your preferred region and you&rsquo;re linked directly to launch that instance in AWS!</p> <p>I&rsquo;m clicking the <a href="https://console.aws.amazon.com/ec2/home?region=us-east-2#launchAmi=ami-00ef4597fd9806efc" target="_blank" rel="noreferrer">launch link for us-east-2 (Ohio)</a><sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. To finish quickly, I&rsquo;m choosing all of the default options and using a spot instance (look inside <em>Advanced details</em> at the bottom of the page).</p> <p>Wait for the instance to finish intializing and access it via ssh:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> ssh fedora@EXTERNAL_IP </span></span><span class="line"><span class="cl"><span class="gp">[fedora@ip-172-31-2-38 ~]$</span> cat /etc/fedora-release </span></span><span class="line"><span class="cl"><span class="go">Fedora release 38 (Thirty Eight) </span></span></span></code></pre></div><p>Success! πŸŽ‰</p> <h1 id="prepare-your-security-group" class="relative group">Prepare your security group <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#prepare-your-security-group" aria-label="Anchor">#</a></span></h1><p>Before leaving the EC2 console, you need to make a note of the security group that you used for this instance. That&rsquo;s because EFS uses security groups to guard access to volumes. Follow these steps to find it:</p> <ol> <li>Click <em>Instances</em> on the left side of the EC2 console.</li> <li>Click on the row showing the instance we just created.</li> <li>In the bottom half of the screen, click the <em>Security</em> tab.</li> <li>Look for <em>Security groups</em> in the security details and copy the security group ID for later.</li> </ol> <p>It should be in the format <code>sg-[a-f0-9]*</code>.</p> <p>If you click the security group name (after saving it), you&rsquo;ll see the inbound rules associated with that security group. By default, items in the same security group can&rsquo;t talk to each other. We need to allow that so our EFS mount will work later.</p> <p>Click <em>Edit inbound rules</em> and do the following:</p> <ol> <li>Click <em>Add rule</em>.</li> <li>Choose <em>All traffic</em> in the <em>Type</em> column. <em>(You can narrow this down further later.)</em></li> <li>In the source box, look for the security group you just created along with your EC2 instance. If you took the default during the EC2 launch process, it might be named <code>launch-wizard-[0-9]+</code>.</li> <li>Click <em>Save rules</em>.</li> </ol> <h1 id="installing-efs-utils" class="relative group">Installing efs-utils <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#installing-efs-utils" aria-label="Anchor">#</a></span></h1><p>Let&rsquo;s start by getting the <em>efs-utils</em> package onto our new Fedora system:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo dnf -qy install efs-utils </span></span><span class="line"><span class="cl"><span class="go">Installed: </span></span></span><span class="line"><span class="cl"><span class="go"> efs-utils-1.35.0-2.fc38.noarch </span></span></span></code></pre></div><p>The package includes some configuration, a watchdog, and a mount helper:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> rpm -ql efs-utils </span></span><span class="line"><span class="cl"><span class="go">/etc/amazon </span></span></span><span class="line"><span class="cl"><span class="go">/etc/amazon/efs </span></span></span><span class="line"><span class="cl"><span class="go">/etc/amazon/efs/efs-utils.conf </span></span></span><span class="line"><span class="cl"><span class="go">/etc/amazon/efs/efs-utils.crt </span></span></span><span class="line"><span class="cl"><span class="go">/usr/bin/amazon-efs-mount-watchdog </span></span></span><span class="line"><span class="cl"><span class="go">/usr/lib/systemd/system/amazon-efs-mount-watchdog.service </span></span></span><span class="line"><span class="cl"><span class="go">/usr/sbin/mount.efs </span></span></span><span class="line"><span class="cl"><span class="go">/usr/share/doc/efs-utils </span></span></span><span class="line"><span class="cl"><span class="go">/usr/share/doc/efs-utils/CONTRIBUTING.md </span></span></span><span class="line"><span class="cl"><span class="go">/usr/share/doc/efs-utils/README.md </span></span></span><span class="line"><span class="cl"><span class="go">/usr/share/licenses/efs-utils </span></span></span><span class="line"><span class="cl"><span class="go">/usr/share/licenses/efs-utils/LICENSE </span></span></span><span class="line"><span class="cl"><span class="go">/usr/share/man/man8/mount.efs.8.gz </span></span></span><span class="line"><span class="cl"><span class="go">/var/log/amazon/efs </span></span></span></code></pre></div><p>Let&rsquo;s get the watchdog running so we have that ready later. The watchdog helps to build and tear down the encrypted connection when you mount and unmount an EFS volume:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo systemctl <span class="nb">enable</span> --now amazon-efs-mount-watchdog.service </span></span><span class="line"><span class="cl"><span class="go">Created symlink /etc/systemd/system/multi-user.target.wants/amazon-efs-mount-watchdog.service β†’ /usr/lib/systemd/system/amazon-efs-mount-watchdog.service. </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="gp">$</span> systemctl status amazon-efs-mount-watchdog.service </span></span><span class="line"><span class="cl"><span class="go">● amazon-efs-mount-watchdog.service - amazon-efs-mount-watchdog </span></span></span><span class="line"><span class="cl"><span class="go"> Loaded: loaded (/usr/lib/systemd/system/amazon-efs-mount-watchdog.service; enabled; preset: disabled) </span></span></span><span class="line"><span class="cl"><span class="go"> Drop-In: /usr/lib/systemd/system/service.d </span></span></span><span class="line"><span class="cl"><span class="go"> └─10-timeout-abort.conf </span></span></span><span class="line"><span class="cl"><span class="go"> Active: active (running) since Wed 2023-09-13 18:43:46 UTC; 5s ago </span></span></span><span class="line"><span class="cl"><span class="go"> Main PID: 1258 (amazon-efs-moun) </span></span></span><span class="line"><span class="cl"><span class="go"> Tasks: 1 (limit: 4385) </span></span></span><span class="line"><span class="cl"><span class="go"> Memory: 13.3M </span></span></span><span class="line"><span class="cl"><span class="go"> CPU: 76ms </span></span></span><span class="line"><span class="cl"><span class="go"> CGroup: /system.slice/amazon-efs-mount-watchdog.service </span></span></span><span class="line"><span class="cl"><span class="go"> └─1258 /usr/bin/python3 /usr/bin/amazon-efs-mount-watchdog </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err"> </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go">Sep 13 18:43:46 ip-172-31-2-38.us-east-2.compute.internal systemd[1]: Started amazon-efs-mount-watchdog.service - amazon-efs-mount-watchdog. </span></span></span></code></pre></div><h1 id="setting-up-an-efs-volume" class="relative group">Setting up an EFS volume <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#setting-up-an-efs-volume" aria-label="Anchor">#</a></span></h1><p>Start by going over to the <a href="https://us-east-2.console.aws.amazon.com/efs/home?region=us-east-2#" target="_blank" rel="noreferrer">EFS console</a> and do the following:</p> <ol> <li> <p>Click <em>File systems</em> in the left navigation bar</p> </li> <li> <p>Click the orange <em>Create file system</em> button at the top right</p> <p>A modal appears with a box for the volume name and a VPC selection. Select an easy to remember name (I&rsquo;m using <em>testing-efs-for-blog-post</em>) and select a VPC. If you&rsquo;re not sure what a VPC is or which one to use, use the default VPC since that&rsquo;s likely where your instance landed as well.</p> </li> <li> <p>Click <em>Create</em>.</p> </li> </ol> <p>There&rsquo;s a delay while the filesystem initializes and you should see the filesystem show <em>Available</em> with a green check mark after about 30 seconds. Click on the filesystem you just created from the list and you&rsquo;ll see the details page for the filesystem.</p> <h1 id="security-setup" class="relative group">Security setup <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#security-setup" aria-label="Anchor">#</a></span></h1><p>EFS volumes come online with the default security group attached and that&rsquo;s not helpful. From the EFS filesystem details page, click the <em>Network</em> tab and then click <em>Manage</em>.</p> <p>For each availability zone, go to the <em>Security groups</em> column and add the security group that your instance came up with in the first step. In my case, I accepted the defaults from EC2 and ended up with a <em>launch-wizard-1</em> security group. Remove the <em>default</em> security group from each. Click <em>Save</em>.</p> <h1 id="mounting-time" class="relative group">Mounting time <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#mounting-time" aria-label="Anchor">#</a></span></h1><p>You should still be on the filesystem details page from the previous step. Click <em>Attach</em> at the top right and a modal will appear with mount instructions. The first option should use the EFS mount helper!</p> <p>For me, it looks like <code>sudo mount -t efs -o tls fs-0baabc62763375bb1:/ efs</code></p> <p>Go back to your Fedora instance, create a mount point, and create the volume:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo mkdir /mnt/efs </span></span><span class="line"><span class="cl"><span class="gp">$</span> sudo mount -t efs -o tls fs-0baabc62763375bb1:/ /mnt/efs </span></span><span class="line"><span class="cl"><span class="gp">$</span> df -hT <span class="p">|</span> grep efs </span></span><span class="line"><span class="cl"><span class="go">127.0.0.1:/ nfs4 8.0E 0 8.0E 0% /mnt/efs </span></span></span></code></pre></div><p>We did it! πŸŽ‰</p> <p>We see <code>127.0.0.1</code> here because efs-utils uses stunnel to handle the encryption between your instance and the EFS storage system.</p> <p>The disk was mounted by root, so we can add a <code>-o user=fedora</code> to give our Fedora user permissions to write files:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> umount /mnt/efs </span></span><span class="line"><span class="cl"><span class="gp">$</span> sudo mount -t efs -o <span class="nv">user</span><span class="o">=</span>fedora,tls fs-0baabc62763375bb1:/ /mnt/efs </span></span><span class="line"><span class="cl"><span class="gp">$</span> touch /mnt/efs/test2.txt </span></span><span class="line"><span class="cl"><span class="gp">$</span> stat /mnt/efs/test2.txt </span></span><span class="line"><span class="cl"><span class="go"> File: /mnt/efs/test2.txt </span></span></span><span class="line"><span class="cl"><span class="go"> Size: 0 Blocks: 8 IO Block: 1048576 regular empty file </span></span></span><span class="line"><span class="cl"><span class="go">Device: 0,54 Inode: 17657675890899444015 Links: 1 </span></span></span><span class="line"><span class="cl"><span class="go">Access: (0644/-rw-r--r--) Uid: ( 1000/ fedora) Gid: ( 1000/ fedora) </span></span></span><span class="line"><span class="cl"><span class="go">Context: system_u:object_r:nfs_t:s0 </span></span></span><span class="line"><span class="cl"><span class="go">Access: 2023-09-13 19:14:23.308000000 +0000 </span></span></span><span class="line"><span class="cl"><span class="go">Modify: 2023-09-13 19:14:23.308000000 +0000 </span></span></span><span class="line"><span class="cl"><span class="go">Change: 2023-09-13 19:14:23.308000000 +0000 </span></span></span><span class="line"><span class="cl"><span class="go"> Birth: - </span></span></span></code></pre></div><p>Also, <em>efs-utils</em> uses encrypted communication by default, which is great. There may be some situations where you don&rsquo;t need encrypted communications or you don&rsquo;t want the overhead. In that case, drop the <code>-o tls</code> option from the mount command and you&rsquo;ll mount the volume unencrypted.</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="gp">$</span> sudo umount /mnt/efs </span></span><span class="line"><span class="cl"><span class="gp">$</span> sudo mount -t efs -o <span class="nv">user</span><span class="o">=</span>fedora fs-0baabc62763375bb1:/ /mnt/efs </span></span><span class="line"><span class="cl"><span class="gp">$</span> df -hT <span class="p">|</span> grep efs </span></span><span class="line"><span class="cl"><span class="go">fs-0baabc62763375bb1.efs.us-east-2.amazonaws.com:/ nfs4 8.0E 0 8.0E 0% /mnt/efs </span></span></span></code></pre></div><h1 id="extra-credit" class="relative group">Extra credit <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#extra-credit" aria-label="Anchor">#</a></span></h1><p>You can get fancy with <a href="https://docs.aws.amazon.com/efs/latest/ug/create-access-point.html" target="_blank" rel="noreferrer">access points</a> that allow you to carve up your EFS storage and only let certain instances mount certain parts of the filesystem. So instance A might only be able to mount <code>/files/hr</code> while instance B can only mount <code>/documents</code>.</p> <p>It would also be a good idea to take an inventory of your security groups and ensure the least amount of instances can reach your EFS volume as possible. Much of the work I did in this post was just for testing. A good plan might be to make a security group for your EFS volume and only allow inbound traffic from security groups which should access it. That would allow you to gather up all of your instances into different security groups and limit access.</p> <p>Also, be aware of the <a href="https://aws.amazon.com/efs/pricing/" target="_blank" rel="noreferrer">EFS pricing</a>! πŸ’Έ</p> <p>You are billed not only for how much storage you use, but also on requests. Different requests are priced differently depending on access frequency. Backups are also <strong>enabled by default</strong> at $0.05/GB-month!</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>Why Ohio? I&rsquo;m mainly doing it to irritate <a href="https://www.lastweekinaws.com/" target="_blank" rel="noreferrer">Corey Quinn</a>. 🀭 Any region you prefer should be fine.&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>Car buying guidehttps://major.io/p/car-buying-guide/Mon, 04 Sep 2023 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/car-buying-guide/<hr> <p>I love optimizing nearly everything in my life. Sometimes it means saving money. Other times it means squeezing every bit of performance out of a server.</p> <p>But let&rsquo;s try optimizing something I&rsquo;ve never done on this blog before: <strong>Buying a car.</strong></p> <p>Although most of this information applies best to people in the USA, there are several things I&rsquo;ve learned over the years that might benefit people in other places. Car purchases are often the second largest purchase for most Americans after buying a house. Why not optimize it as much as possible?</p> <p>My family is in the market for new vehicle and I&rsquo;ve immersed myself in learning more about the whole process. There are plenty of legal issues in play that many buyers don&rsquo;t know about and there are tons of ways to walk into a dealership fully prepared.</p> <p>Without further ado, I&rsquo;ll share what I&rsquo;ve learned with you!</p> <h1 id="shopping" class="relative group">Shopping <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#shopping" aria-label="Anchor">#</a></span></h1><p>Shopping and buying are two different things. <strong>You must keep them separate.</strong> If you&rsquo;re not sure on the exact model of car you want, go shopping and don&rsquo;t commit to buy anything.</p> <p>If you have an idea of a particular car you want, go to Google and search for 5-10 competitors to that car. It&rsquo;s as easy as searching &ldquo;top competitors Camry&rdquo; to find other cars that compete with a Toyota Camry. Go to those dealers and take a good look at the cars to see if they have the features you want. Test drive each and see if you still like them.</p> <p>The really important thing here is to separate <strong>shopping</strong> from <strong>buying</strong>.</p> <h1 id="search-for-prices" class="relative group">Search for prices <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#search-for-prices" aria-label="Anchor">#</a></span></h1><p>Once you narrow your search down a bit, start to compare prices between dealers for different trim levels. I love using <a href="https://caredge.com/" target="_blank" rel="noreferrer">CarEdge</a> for this since you can examine dealer inventory across the country and see how dealer prices stack up against each other.</p> <p>Also check how long the vehicle has been sitting on the lot. CarEdge offers this data and it can give you an idea of how desperate a dealer is to get rid of a particular vehicle. If the total supply of a model is really high and the vehicle has been sitting on the lot longer than 90 days, you have the ability to negotiate for that car. Cars with a very low supply or cars that just arrived to the lot will likely be priced higher and dealers are less likely to budge on price.</p> <p>For used cars, this is even more important since you don&rsquo;t have an MSRP to work with as you do for new cars. Comparing prices for used cars is critical.</p> <p>Keep in mind that some cars are in higher demand in some areas than others, too. You might be able to drive a few hours and save quite a bit. A friend of mine drove from Texas to Colorado to buy a car and saved $3,500.</p> <p><strong>Watch out for arbitrary markups!</strong> All dealers in the US are required to display a Monroney label and this shows the manufacturer&rsquo;s suggested retail price. Dealers might add an addendum sticker somewhere else that show accessories they added. Sometimes these stickers show arbitrary markups from the dealer.</p> <p>Since COVID, many dealers have been stacking massive fees on top of car purchases. Some of these fees exceed $10,000, $25,000 or more! <strong>This should be a massive red flag from the start of a negotiation with any dealer.</strong> If they aren&rsquo;t willing to budge from their arbitrary markups, look for another dealer. 🚩</p> <h1 id="you-found-the-car-you-want" class="relative group">You found the car you want <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#you-found-the-car-you-want" aria-label="Anchor">#</a></span></h1><p>Awesome! πŸ₯³</p> <p>Whether it&rsquo;s new or used, take the VIN number and dig up information about the car. You can learn a lot from just a quick Google search from the VIN!</p> <p>Other than that, CarEdge has some good tools for digging up data on individual cars without too much expense. CarFax is the gold standard used and it offers some fairly inexpensive options.</p> <p>Some might be saying: <em>&ldquo;Isn&rsquo;t this a waste for a brand new car?&rdquo;</em> <strong>No, it&rsquo;s not.</strong></p> <p>I was once buying a pickup truck that was advertised as one model year, but the truck was actually a year older. The dealer even switched the paper tags attached to the keys so I wouldn&rsquo;t notice. I didn&rsquo;t notice this until months later when I found it buried in my paperwork. 😑</p> <p>There have also been situations where <strong>stolen cars</strong> are sitting on dealer lots. 😱</p> <p>Remember, this is a big investment. Spending $20 on a CarFax report as you purchase a $30,000 car should be worth it.</p> <h1 id="financing" class="relative group">Financing <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#financing" aria-label="Anchor">#</a></span></h1><p>Take care of your financing <strong>before visiting a dealer.</strong> Local credit unions are often great for this but they&rsquo;ve been under a squeeze lately with an increase in reposessions. Work with the credit union to get a good rate for the type of car you want.</p> <p>Some people have had good luck financing through a dealership. However, getting financing set up ahead of time gives you the upper hand with financing negotiations. For example, when you sit down with in the finance office, you could say: <em>&ldquo;I have 5.9% through my local credit union. If you can beat that, I&rsquo;ll finance with you.&rdquo;</em></p> <p>If you choose to go through financing with the dealer, here&rsquo;s what I recommend:</p> <ol> <li><strong>Do not allow the dealer to run your credit report until the last minute.</strong> If you let them run it, they know that you&rsquo;ll get a hard hit on your credit report and you&rsquo;re less likely to look at other dealers. You want to leave your options open. Don&rsquo;t let them run your credit until you are <strong>100% sure you&rsquo;re ready to buy from them</strong>.</li> <li>Ask to see the &ldquo;call sheet&rdquo; and see what the bank offered the dealer for financing (the &ldquo;buy rate&rdquo;). It&rsquo;s very likely that the dealer gets one rate from the bank and then marks it up for you. For example, the dealer might say <em>&ldquo;Oh, your rate is 8%.&rdquo;</em> Then you ask for the call sheet or the buy rate and see that they got 5.9% from the bank. <strong>You&rsquo;re getting a huge markup.</strong> That&rsquo;s another negotiation point.</li> <li>If the dealer says they can lower your interest rate if you buy an add-on from them, such as extended warranty, <strong>STOP IMMEDIATELY.</strong> 🚨 This is called &ldquo;tied selling&rdquo; and is almost always <a href="https://www.ftc.gov/advice-guidance/competition-guidance/guide-antitrust-laws/single-firm-conduct/tying-sale-two-products" target="_blank" rel="noreferrer">illegal in the USA</a>. This is a good moment to stop and re-evaluate the whole transaction.</li> <li>You are not required to buy any of the add-ons that the finance manager offers you. Extended warranties, dent and ding protection, and tire/wheel protection are common items. There&rsquo;s no requirement to purchase these. If you do decide to purchase one, be sure they can show you <strong>the actual amount that it will cost you.</strong> Don&rsquo;t let them tell you how much it adds to your monthly payment. That allows them to hide the cost of certain add-ons.</li> <li>You are required to pay reasonable fees for tax, title, and license. Sometimes documentation fees are rolled up into this, too. Every state is a little different on how much these cost, but you can Google &ldquo;tax title and license&rdquo; for your state and get a good estimation.</li> <li><strong>Demand to see the &ldquo;out the door price.&rdquo;</strong> If a dealer asks how much you can afford per month, <strong>don&rsquo;t answer.</strong> You are interested in the full price of the car plus fees. Most states require that this comes out on a single sheet of paper with each expense clearly labeled with what you will owe for the car. <strong>If you tell a dealer &ldquo;I can&rsquo;t affort more than $500 per month&rdquo;, then they will tinker with various parts of the deal to ensure you pay more in the long term without exceeding $500 per month.</strong> If the dealer won&rsquo;t budge, keep asking them &ldquo;If I was getting you a cashier&rsquo;s check today, how much needs to be on the check?&rdquo; They will eventually get the idea.</li> </ol> <p>Dealers make the most money by far not on the lot itself, but in the finance office. Don&rsquo;t get swindled there.</p> <h1 id="get-copies-and-read-them" class="relative group">Get copies and read them <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#get-copies-and-read-them" aria-label="Anchor">#</a></span></h1><p>You are entitled to a copy of everything that shows up in the finance office. <strong>Don&rsquo;t get up from the chair until you have a copy of everything and you&rsquo;ve examined each page.</strong></p> <p>Dealers commonly adjust numbers or conveniently leave out &ldquo;We Owe&rdquo; sheets (see the next section) in the finance office. Sometimes it&rsquo;s an honest mistake. Sometimes it&rsquo;s not.</p> <h1 id="the-dreaded-we-owe" class="relative group">The dreaded &ldquo;We Owe&rdquo; <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#the-dreaded-we-owe" aria-label="Anchor">#</a></span></h1><p>If a dealer doesn&rsquo;t have something in stock that they promised you, such as an accessory or add-on, ensure it lands on the &ldquo;We Owe&rdquo; sheet in your paperwork.</p> <p>There should be page somewhere in your sale paperwork that shows anything that the dealer owes you. In many states, this sheet <em>must</em> be in your paperwork even if it&rsquo;s blank or zeroed out!</p> <p>If a dealer promised you something and it didn&rsquo;t land on the &ldquo;We Owe&rdquo; sheet, stop immediately and ask for that to be corrected right then.</p> <h1 id="trade-ins" class="relative group">Trade-ins <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#trade-ins" aria-label="Anchor">#</a></span></h1><p>Any time you trade in a vehicle, <strong>make it a different transaction.</strong> Allowing a dealer to add your trade-in with the current deal for purchase allows them to hide money for themselves in the deal.</p> <p>First, get multiple offers from various sites that buy cars all day long. I recommend Carvana, Driveway, and Vroom. They will give you an immediate estimate online with a little bit of information. The CarEdge site I mentioned earlier also has some tools that allow you to look up your car in the Black Book, which is what dealers use to appraise cars.</p> <p>Next, when you go to the dealer and they ask if you&rsquo;re trading in a car, tell them you haven&rsquo;t decided yet. Your best bet is to act as if you want them to talk you into it. Either way, <strong>keep the trade-in as a separate transaction.</strong></p> <p>When it comes time to talk about your trade-in, you should already have an out the door price on your car purchase. <em>Scroll up if you&rsquo;re not sure about this.</em> πŸ˜‰</p> <p>Let the dealer know you have some other offers on your trade and let them know you&rsquo;ll do the trade there if they can beat the offers. If they offer to beat the other deals, that&rsquo;s awesome! That&rsquo;s less work for you!</p> <p>If they can&rsquo;t, don&rsquo;t worry. You can handle the trade-in separately with those other companies.</p> <h1 id="buying-online" class="relative group">Buying online <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#buying-online" aria-label="Anchor">#</a></span></h1><p>Finally, you can buy a car almost entirely online these days. Carvana, Driveway, and Vroom offer a fully online experience, but traditional dealers often have salespeople focused on internet deals.</p> <p>This is a good way to sort out dealers who want to work with you and those that don&rsquo;t. Let dealers know that:</p> <ol> <li>You know what vehicle you want.</li> <li>You&rsquo;re comparing the offers from multiple dealers.</li> <li>You&rsquo;re looking to purchase within the next few weeks.</li> <li>You want an out the door price on the vehicle so you know how much to get on the cashier&rsquo;s check.</li> </ol> <p>If they&rsquo;re willing to deal with you via email or phone, that&rsquo;s great! If not, there&rsquo;s plenty of other dealers out there.</p> <h1 id="further-learning" class="relative group">Further learning <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#further-learning" aria-label="Anchor">#</a></span></h1><p>I&rsquo;ve learned a lot from several YouTube channels over the years. Here are my favorites:</p> <ul> <li><a href="https://www.youtube.com/@carquestionsanswered" target="_blank" rel="noreferrer">Car Questions Answered</a>: Brandon runs a small used car dealership in North Carolina and gives lots of insights on what is happening the used and new car markets. He has lots of good advice on when to buy a car and which cars are the ones to avoid at certain times. If you ever wanted to go behind the scenes to see how a small used car dealership works, his channel is great.</li> <li><a href="https://www.youtube.com/@DeshoneTheAutoAdvisor" target="_blank" rel="noreferrer">Deshone The Auto Advisor</a>: Deshone has lots of helpful short videos. He does offer a membership program that comes with a fee, but his short videos cover a ton of car buying and leasing recommendations. If you&rsquo;re interested in leasing a car, be sure to watch some of his leasing videos.</li> <li><a href="https://www.youtube.com/@CarEdge" target="_blank" rel="noreferrer">CarEdge</a>: CarEdge offers tons of services to help with car buying including 1:1 coaching once you have a deal sheet from a dealer. However, they also have tons of videos that explain how to buy a car effectively. They do role playing for finance managers and salespeople that highlight certain areas where buyers usually lose. Their role play videos are highly recommended.</li> <li><a href="https://www.youtube.com/@LuckyLopez777" target="_blank" rel="noreferrer">Lucky Lopez</a>: Lucky has tons of insight from a dealer perspective and he follows lots of trends around pricing, supply, and reposessions. Most of his content might be too much for car buyers, but it&rsquo;s good information to know.</li> <li><a href="https://www.youtube.com/channel/UCziwLfSSPLWc8bp02Pc5azg" target="_blank" rel="noreferrer">Chevy Dude</a>: The Chevy Dude used to work for multiple dealers but now it running his own. He shares lots of sales tricks and secrets that help you prepare for making your next deal on a car &ndash; new or used.</li> </ul> <h1 id="did-i-miss-something" class="relative group">Did I miss something? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#did-i-miss-something" aria-label="Anchor">#</a></span></h1><p>Let me know if I missed something and I&rsquo;ll come back and edit this post! Just contact me via any of the methods below in the author block. ️⬇️</p>Fixing a ghost database migration failurehttps://major.io/p/ghost-db-migration-failure/Thu, 31 Aug 2023 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/ghost-db-migration-failure/<p>I love learning about the <em>behind the scenes</em> aspects of just about everything. I do <a href="https://w5wut.com" target="_blank" rel="noreferrer">ham radio</a>, I self-host lots of my personal infrastructure, and I&rsquo;ve been learning more about the math behind the stock market for the last year or two.</p> <p>That led me to start a blog on <a href="https://unsplash.com/photos/l6mLi-iKUW0" target="_blank" rel="noreferrer">Ghost</a> to share my findings with others. I started <a href="https://thetanerd.com" target="_blank" rel="noreferrer">Theta Nerd</a><sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> earlier this summer.</p> <p>My deployment looked great when I started! Everything was automatically updated with <a href="https://major.io/p/watchtower/">watchtower</a> and running with <a href="https://major.io/p/docker-compose-on-coreos/" target="_blank" rel="noreferrer">docker-compose on Fedora CoreOS</a>. <em>(Click these links to read the posts on both topics!)</em></p> <p>However, I woke up one morning to my monitoring going off and my site was down. 😱</p> <h1 id="why-is-the-site-down" class="relative group">Why is the site down? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#why-is-the-site-down" aria-label="Anchor">#</a></span></h1><p>Anyone who has worked in IT knows this sinking feeling. Something is down, you don&rsquo;t know why, and you suspect the worst possible scenarios.</p> <p>The instance hosting the blog was online and responsive, so I started digging into the logs with <code>docker-compose logs</code>. I suddenly found a wall of text in the logs for the Ghost container:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">[2023-08-03 11:10:16] INFO Adding members.email_disabled column </span></span><span class="line"><span class="cl">[2023-08-03 11:10:16] INFO Setting email_disabled to true for all members that have their email on the suppression list </span></span><span class="line"><span class="cl">[2023-08-03 11:10:16] INFO Setting nullable: stripe_products.product_id </span></span><span class="line"><span class="cl">[2023-08-03 11:10:16] INFO Adding table: donation_payment_events </span></span><span class="line"><span class="cl">[2023-08-03 11:10:16] INFO Rolling back: alter table `donation_payment_events` add constraint `donation_payment_events_member_id_foreign` foreign key (`member_id`) references `members` (`id`) on delete SET NULL - Referencing column &#39;member_id&#39; and referenced column &#39;id&#39; in foreign key constraint &#39;donation_payment_events_member_id_foreign&#39; are incompatible.. </span></span><span class="line"><span class="cl">[2023-08-03 11:10:16] INFO Dropping table: donation_payment_events </span></span><span class="line"><span class="cl">[2023-08-03 11:10:16] INFO Dropping nullable: stripe_products.product_id with foreign keys disabled </span></span><span class="line"><span class="cl">[2023-08-03 11:10:16] INFO Setting email_disabled to false for all members </span></span><span class="line"><span class="cl">[2023-08-03 11:10:16] INFO Removing members.email_disabled column </span></span><span class="line"><span class="cl">[2023-08-03 11:10:16] INFO Rollback was successful. </span></span><span class="line"><span class="cl">[2023-08-03 11:10:16] ERROR alter table `donation_payment_events` add constraint `donation_payment_events_member_id_foreign` foreign key (`member_id`) references `members` (`id`) on delete SET NULL - Referencing column &#39;member_id&#39; and referenced column &#39;id&#39; in foreign key constraint &#39;donation_payment_events_member_id_foreign&#39; are incompatible. </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl">alter table `donation_payment_events` add constraint `donation_payment_events_member_id_foreign` foreign key (`member_id`) references `members` (`id`) on delete SET NULL - Referencing column &#39;member_id&#39; and referenced column &#39;id&#39; in foreign key constraint &#39;donation_payment_events_member_id_foreign&#39; are incompatible. </span></span><span class="line"><span class="cl">{&#34;config&#34;:{&#34;transaction&#34;:false},&#34;name&#34;:&#34;2023-07-27-11-47-49-create-donation-events.js&#34;} </span></span><span class="line"><span class="cl">&#34;Error occurred while executing the following migration: 2023-07-27-11-47-49-create-donation-events.js&#34; </span></span><span class="line"><span class="cl">Error ID: </span></span><span class="line"><span class="cl"> 300 </span></span><span class="line"><span class="cl">Error Code: </span></span><span class="line"><span class="cl"> ER_FK_INCOMPATIBLE_COLUMNS </span></span><span class="line"><span class="cl">---------------------------------------- </span></span><span class="line"><span class="cl">Error: alter table `donation_payment_events` add constraint `donation_payment_events_member_id_foreign` foreign key (`member_id`) references `members` (`id`) on delete SET NULL - Referencing column &#39;member_id&#39; and referenced column &#39;id&#39; in foreign key constraint &#39;donation_payment_events_member_id_foreign&#39; are incompatible. </span></span><span class="line"><span class="cl"> at /var/lib/ghost/versions/5.57.2/node_modules/knex-migrator/lib/index.js:1032:19 </span></span><span class="line"><span class="cl"> at Packet.asError (/var/lib/ghost/versions/5.57.2/node_modules/mysql2/lib/packets/packet.js:728:17) </span></span><span class="line"><span class="cl"> at Query.execute (/var/lib/ghost/versions/5.57.2/node_modules/mysql2/lib/commands/command.js:29:26) </span></span><span class="line"><span class="cl"> at Connection.handlePacket (/var/lib/ghost/versions/5.57.2/node_modules/mysql2/lib/connection.js:478:34) </span></span><span class="line"><span class="cl"> at PacketParser.onPacket (/var/lib/ghost/versions/5.57.2/node_modules/mysql2/lib/connection.js:97:12) </span></span><span class="line"><span class="cl"> at PacketParser.executeStart (/var/lib/ghost/versions/5.57.2/node_modules/mysql2/lib/packet_parser.js:75:16) </span></span><span class="line"><span class="cl"> at Socket.&lt;anonymous&gt; (/var/lib/ghost/versions/5.57.2/node_modules/mysql2/lib/connection.js:104:25) </span></span><span class="line"><span class="cl"> at Socket.emit (node:events:513:28) </span></span><span class="line"><span class="cl"> at addChunk (node:internal/streams/readable:315:12) </span></span><span class="line"><span class="cl"> at readableAddChunk (node:internal/streams/readable:289:9) </span></span><span class="line"><span class="cl"> at Socket.Readable.push (node:internal/streams/readable:228:10) </span></span><span class="line"><span class="cl"> at TCP.onStreamRead (node:internal/stream_base_commons:190:23) </span></span></code></pre></div><p>Ah, so a failed database migration in the upgrade to 5.57.2 is the culprit! πŸ‘</p> <p>I brought the site back online quickly by changing the container version for Ghost back to the previous version (5.55.2).</p> <h1 id="why-did-the-database-migration-fail" class="relative group">Why did the database migration fail? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#why-did-the-database-migration-fail" aria-label="Anchor">#</a></span></h1><p>The error message from above boils down to this:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">Error: alter table `donation_payment_events` add constraint </span></span><span class="line"><span class="cl">`donation_payment_events_member_id_foreign` foreign key (`member_id`) </span></span><span class="line"><span class="cl">references `members` (`id`) on delete SET NULL - Referencing column </span></span><span class="line"><span class="cl">&#39;member_id&#39; and referenced column &#39;id&#39; in foreign key constraint </span></span><span class="line"><span class="cl">&#39;donation_payment_events_member_id_foreign&#39; are incompatible. </span></span></code></pre></div><p>Adjusting the <code>donation_payment_events.member_id</code> column to be a foreign key of <code>members.id</code> is failing because they are incompatible types. However, as I examined both tables, both were regular <code>varchar(24)</code> columns without anything special attached to them:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">mysql&gt; describe members; </span></span><span class="line"><span class="cl">+------------------------------+---------------+------+-----+---------+-------+ </span></span><span class="line"><span class="cl">| Field | Type | Null | Key | Default | Extra | </span></span><span class="line"><span class="cl">+------------------------------+---------------+------+-----+---------+-------+ </span></span><span class="line"><span class="cl">| id | varchar(24) | NO | PRI | NULL | | </span></span><span class="line"><span class="cl">| uuid | varchar(36) | YES | UNI | NULL | | </span></span><span class="line"><span class="cl">| email | varchar(191) | NO | UNI | NULL | | </span></span><span class="line"><span class="cl">| status | varchar(50) | NO | | free | | </span></span><span class="line"><span class="cl">| name | varchar(191) | YES | | NULL | | </span></span><span class="line"><span class="cl">| expertise | varchar(191) | YES | | NULL | | </span></span><span class="line"><span class="cl">| note | varchar(2000) | YES | | NULL | | </span></span><span class="line"><span class="cl">| geolocation | varchar(2000) | YES | | NULL | | </span></span><span class="line"><span class="cl">| enable_comment_notifications | tinyint(1) | NO | | 1 | | </span></span><span class="line"><span class="cl">| email_count | int unsigned | NO | | 0 | | </span></span><span class="line"><span class="cl">| email_opened_count | int unsigned | NO | | 0 | | </span></span><span class="line"><span class="cl">| email_open_rate | int unsigned | YES | MUL | NULL | | </span></span><span class="line"><span class="cl">| last_seen_at | datetime | YES | | NULL | | </span></span><span class="line"><span class="cl">| last_commented_at | datetime | YES | | NULL | | </span></span><span class="line"><span class="cl">| created_at | datetime | NO | | NULL | | </span></span><span class="line"><span class="cl">| created_by | varchar(24) | NO | | NULL | | </span></span><span class="line"><span class="cl">| updated_at | datetime | YES | | NULL | | </span></span><span class="line"><span class="cl">| updated_by | varchar(24) | YES | | NULL | | </span></span><span class="line"><span class="cl">+------------------------------+---------------+------+-----+---------+-------+ </span></span><span class="line"><span class="cl">18 rows in set (0.00 sec) </span></span></code></pre></div><h1 id="going-upstream" class="relative group">Going upstream <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#going-upstream" aria-label="Anchor">#</a></span></h1><p>I went to Ghost&rsquo;s GitHub repository and <a href="https://github.com/TryGhost/Ghost/issues/17584" target="_blank" rel="noreferrer">opened an issue</a> with as much data as I can find.</p> <p>One of the <a href="https://github.com/TryGhost/Ghost/issues/17584#issuecomment-1671134556" target="_blank" rel="noreferrer">first replies</a> mentioned something about database collations. Long story short, collations describe how databases handle sorting and comparing data for different languages. Comparing some languages to other languages can be particularly challenging and this can lead to problems.</p> <p>I made a switch from MariaDB to MySQL recently for the blog. Could that be related?</p> <h1 id="more-searching" class="relative group">More searching <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#more-searching" aria-label="Anchor">#</a></span></h1><p>I figured that I wasn&rsquo;t the first one to stumble into this problem, and sure enough &ndash; I wasn&rsquo;t! There&rsquo;s a <a href="https://dnsmichi.at/2022/06/01/ghost-v5-upgrade-with-mysql-8-collation-migration-in-docker-compose/" target="_blank" rel="noreferrer">great blog post</a> about a broken migration from MySQL 5 to 8 with Ghost.</p> <p>In short, it required several steps to fix it:</p> <ol> <li>Stop the Ghost container</li> <li>Back up the database first (always a good idea)</li> <li>Do a quick find/replace on the dumped database to change the collations</li> <li>Drop the <code>ghost</code> database from the database 😱</li> <li>Import the database back into MySQL</li> <li>Start Ghost again</li> </ol> <p>Dropping databases always makes me pause, but that&rsquo;s what backups are for! πŸ˜‰</p> <h1 id="how-i-fixed-it" class="relative group">How I fixed it <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#how-i-fixed-it" aria-label="Anchor">#</a></span></h1><p>In my case, my MySQL container is called <code>ghostmysql</code> and my Ghost database is <code>ghostdb</code>. Then I made a backup of the database using <code>mysqldump</code>:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="go">sudo docker-compose exec ghostmysql mysqldump \ </span></span></span><span class="line"><span class="cl"><span class="go"> -u root -psuper-secret-password ghostdb &gt; backup-ghost-db.sql </span></span></span></code></pre></div><p>Next, I copied the SQL file to another directory <em>just in case</em> I accidentally deleted this backup with an errant command.</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="go">cp backup-ghost-db.sql ../ </span></span></span></code></pre></div><p>Then I made a copy of the SQL file in the current directory and ran the find and replace on that copy. This changes the collations from the wrong one, <code>utf8mb4_general_ci</code>, to the right one, <code>utf8mb4_0900_ai_ci</code><sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="go">cp backup-ghost-db.sql backup-ghost-db-new.sql </span></span></span><span class="line"><span class="cl"><span class="go">sed -i &#39;s/utf8mb4_general_ci/utf8mb4_0900_ai_ci/g&#39; \ </span></span></span><span class="line"><span class="cl"><span class="go"> backup-ghost-db-new.sql </span></span></span></code></pre></div><p>Now I have the collations right for importing the database back into MySQL. But first, I have to drop the existing database. <strong>This is a good time to double check your backups!</strong></p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="go">sudo docker-compose exec ghostmysql mysql -u root \ </span></span></span><span class="line"><span class="cl"><span class="go"> -psuper-secret-password </span></span></span><span class="line"><span class="cl"><span class="go">mysql&gt; DROP DATABASE ghostdb; </span></span></span></code></pre></div><p>Now we can import the modified backup:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="go">cat backup-ghost-db-new.sql | sudo docker-compose exec -T \ </span></span></span><span class="line"><span class="cl"><span class="go"> ghostmysql mysql -u root -psuper-secret-password ghostdb </span></span></span></code></pre></div><p>Start all the containers:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-console" data-lang="console"><span class="line"><span class="cl"><span class="go">sudo docker-compose up -d </span></span></span></code></pre></div><p>Ghost was back online with the older version and everything looked good! I updated my <code>docker-compose.yaml</code> back to use <code>latest</code> for the Ghost version and ran <code>sudo docker-compose up -d</code> once more.</p> <p>Within seconds, the new container image was in place and the container was running! Both migrations completed in seconds and the blog was back online with the newest version. πŸŽ‰</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>Theta is one of many <a href="https://en.wikipedia.org/wiki/Greeks_%28finance%29" target="_blank" rel="noreferrer">financial Greeks</a> that measure certain aspects of options contracts in the market. It&rsquo;s also a <a href="https://en.wikipedia.org/wiki/Theta" target="_blank" rel="noreferrer">letter in the Greek alphabet</a>.&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p>The default collation in MySQL 8 is <code>utf8mb4_0900_ai_ci</code>.&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>Open source contributions: Just do ithttps://major.io/p/open-source-like-nike/Wed, 16 Aug 2023 00:00:00 +0000major@mhtx.net (Major Hayden)https://major.io/p/open-source-like-nike/<p>I had a great time on the <a href="https://www.youtube.com/watch?v=aA-pBYxUPgU" target="_blank" rel="noreferrer">Fedora Podcast</a> yesterday to talk about Fedora cloud! We talked about all kinds of Fedora-related topics, but a couple of questions came up around how to contribute, especially when there&rsquo;s not a lot of structure in place for a particular type of contributions. Here&rsquo;s the full video if you&rsquo;re interested:</p> <div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;"> <iframe allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen="allowfullscreen" loading="eager" referrerpolicy="strict-origin-when-cross-origin" src="https://www.youtube.com/embed/aA-pBYxUPgU?autoplay=1&controls=1&end=0&loop=0&mute=1&start=0" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%; border:0;" title="YouTube video" ></iframe> </div> <p>That made me think about a post that deserves to be written: <strong>How do you get started with open source contributions in a new project?</strong> πŸ€”</p> <p>My answer is pretty simple: <strong>Just do it.</strong> πŸš€</p> <h1 id="just-do-what" class="relative group">Just do what? <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#just-do-what" aria-label="Anchor">#</a></span></h1><p>In the late 1980&rsquo;s, one of Nike&rsquo;s ad agencies <a href="https://en.wikipedia.org/wiki/Just_Do_It" target="_blank" rel="noreferrer">came up with the phrase</a> as a way to push through uncertainty. I was pretty young when this campaign started, but the general idea was this:</p> <ul> <li>Anyone can achieve what they want</li> <li>Stop worrying about whether you can actually do something</li> <li>Try something new</li> <li>Just do it</li> </ul> <p>Simple, right?</p> <p>This works for open source contributions, too. I often have conversations with people inside and outside of work where they identify a problem or an improvement in an open source project. My customary response is <em>&ldquo;Let&rsquo;s go upstream and make this better!&rdquo;</em></p> <p>However, what I hear back most often is <em>&ldquo;I don&rsquo;t know how.&rdquo;</em> This is where the whole <strong>just do it</strong> part comes in.</p> <h1 id="i-found-a-bug" class="relative group">I found a bug <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#i-found-a-bug" aria-label="Anchor">#</a></span></h1><p>Nearly every open source project wants to know about bugs that users experience. Start by finding out the best way to communicate with the people working on a particular project.</p> <p>For projects on GitHub or GitLab, you can open up an issue and describe your problem. Some repositories have a template generated for bugs that ask you several important questions, so be sure to follow those templates. If there isn&rsquo;t a template, I usually follow this format:</p> <ul> <li>What happened that was unexpected?</li> <li>What were you doing right before that unexpected event happened?</li> <li>What did you expect to happen instead?</li> <li>What else is nearby in the environment that might have an impact? <ul> <li>For example, the versions of Python might be important for Python-based projects.</li> </ul> </li> <li>What log files or other diagnostic materials exist?</li> </ul> <p>What&rsquo;s the goal? We want to give maintainers enough information for a quick diagnosis in the best case. If it&rsquo;s not obvious, then they need enough information to try to reproduce it on their own machine for debugging.</p> <p>Maintainers might come back with additional questions about your environment or the events just before the bug occurred. Be sure to respond in a timely way while the information is top of mind for them.</p> <p><strong>Always remember that these maintainers are real people who are likely not being paid for their work.</strong> Assume the best of intentions (unless proven otherwise) and stay focused on the solution. There might always be the chance that the maintainers are not interested in your use case and might not be interested in solving it.</p> <p>That leads me to the next step.</p> <h1 id="i-found-a-bug-and-i-want-to-fix-it" class="relative group">I found a bug and I want to fix it <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#i-found-a-bug-and-i-want-to-fix-it" aria-label="Anchor">#</a></span></h1><p>Start by opening an issue or a bug report first (see the previous section).</p> <p>This ensures that maintainers get a full picture of the problem you&rsquo;re trying to solve. Also, I&rsquo;ve had maintainers immediately reply and tell me that it&rsquo;s a known issue already being solved in another issue. That could save you some work.</p> <p>If you have a patch that fixes the issue, go through the following steps before submitting the fix upstream:</p> <ul> <li>Ensure your fix references the issue or bug report that you opened</li> <li>Use a very clear first line in your commit message, such as <code>parser: Fix emoji handling in YAML</code> rather than <code>Fix YAML bug</code></li> <li>Include a very brief explanation of the bug you&rsquo;re fixing in the commit message</li> <li><strong>Extra credit:</strong> Add or update existing tests so they catch the bug you just found</li> <li><strong>Extra credit:</strong> Add or update the project documentation for your change if necessary</li> </ul> <p>These extra credit items often make it easier to review your patch. Maintainers love extra test coverage, too.</p> <p>Submit your change in a pull request or merge request and watch for updates. Be patient with replies from the maintainers, but be timely in your replies. Remember that your use case might be an edge case for the upstream project and you might need to explain your fix (or the original bug) in more detail.</p> <h1 id="i-want-to-improve-something" class="relative group">I want to improve something <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#i-want-to-improve-something" aria-label="Anchor">#</a></span></h1><p>Improving an open source project could involve several things, such as:</p> <ul> <li>Enhancing by adding a new feature</li> <li>Optimizing an existing feature</li> <li>Creating documentation</li> <li>Building integrations</li> </ul> <p>I strongly recommend opening an issue first with the project maintainers to explain your enhancement. These <em>Requests for Enhancements</em>, or RFEs, should include several things:</p> <ul> <li>Your use case that made you think of the enhancement in the first place</li> <li>What you plan to add, substract, or change</li> <li>How the changes might affect different users, especially as they upgrade from older versions</li> <li>How the changes might affect testing or release processes</li> <li>Any changes in dependencies required</li> </ul> <p>Before going down the road of enhancements, always bring up these ideas with the maintainers first. You want to ensure that your ideal changes are aligned with the future goals of the project. In addition, maintainers will want to better understand your use case.</p> <p>Remember that an enhancement almost always requires additional work from maintainers. Every new use case means more work to ensure the project still functions. That&rsquo;s why it&rsquo;s critical to share your use case and have a good plan for testing and documentation.</p> <h1 id="getting-involved" class="relative group">Getting involved <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100"><a class="group-hover:text-primary-300 dark:group-hover:text-neutral-700" style="text-decoration-line: none !important;" href="#getting-involved" aria-label="Anchor">#</a></span></h1><p>Whenever I find an open source project that I&rsquo;d like to get involved with, I start looking around for several things:</p> <ul> <li><strong>What do they use for informal asynchronous chat?</strong> IRC? Matrix? Slack? Something else? I join the chat, introduce myself, and get an idea for how they interact. Some groups are very chatty and informal while others are much more formal and regimented.</li> <li><strong>Where do they have detailed discussions?</strong> Many projects have detailed discussions in their issues/bugs or in places like GitHub&rsquo;s discussions. Others use <a href="https://major.io/p/mailing-list-beef/">old school mailing lists</a>. Some groups have regular meetings where anyone can add agenda items for discussions. If I need to talk about something a bit more long form and I expect some back and forth on it, I look for this avenue.</li> <li><strong>What requirements exist for contributors?</strong> Some projects require that contributors sign a <a href="https://en.wikipedia.org/wiki/Contributor_License_Agreement" target="_blank" rel="noreferrer">CLA</a> or some other sort of agreement. Make sure that any CLAs you sign are approved by your employer (if applicable). You might need an account on a system that you don&rsquo;t have, so check for that as well.</li> </ul> <p>From there, I take the <strong>just do it</strong> mentality and go for it. The worst thing you&rsquo;ll be told is <em>&ldquo;No&rdquo;</em>. If that happens, take a step back, see if there&rsquo;s another way to approach it, and try again.</p> <p>Remember one thing most of all: <strong>avoid taking anything personally.</strong> All of us have our bad days and some people have personalities that might be totally incompatible with yours (and most people in general). 🀭</p>