In late 2009 I migrated all of my websites to my personal Mac Mini server, which is hosted in MacMiniColo’s data center (now part of MacStadium). You can read about my reasons for moving from hosted services to my own server here.
I’ve never looked back, and have mostly enjoyed having my own server because of the freedom it gives me to experiment and customize my environment.
Mostly.
When I first got the server, I was new to Linux and was really happy Apple provided Server.app, which is a GUI for the standard fare of services, including Apache, mail, FTP, VPN, and certificate management. I had previously dabbled in Linux server administration via hosted services and Microsoft IIS at my workplace, but it’s safe to say I was still a n00b. Server.app handled the heavy lifting and made it easy for a lightweight like me to get a simple site up and running.
Almost exactly eight years later, I’ve replaced the hardware once (a newer, faster Mini), have updated macOS seven times, and replaced Server.app six or seven times. Through it all, the Mini (and MacMiniColo’s hosting) has been solid. The software? No so much.
Apple’s Server.app is a compilation of open-source software, which sounds great — plenty of people use the same software and there are literally thousands of how-to guides on the interwebs. Except… Apple in their wisdom decided to customize pretty much everything, which meant the aforementioned guides were often useless, causing endless headaches. (On the bright side, my Google-Fu has grown immensely.)
Over the past few years, HTTPS has become an increasingly important part of web hosting. Before the advent of Let’s Encrypt, I had purchased a couple of commercial SSL certificates (WOW they’re expensive) and installed them via Server.app. This was not very difficult. But as I started adding more and more sites and SSL certs to my server, I started running into really weird Apache errors, which often caused ALL of my sites to become unavailable. Remember, Server.app was doing the Apache config, not me, so it should have been as easy as drag-and-drop. Finding solutions to these errors proved to be incredibly painful, as there are very few resources for Server.app, and even fewer that are up-to-date. Every Apache troubleshooting guide I’d find referred to the standard Apache installation, not the Apple-flavored installation, which stored files in completely different locations and included many modifications.
But I soldiered on, eventually sorting out each issue and hoping it would be fixed in the next version of Server.app.
Last month I finally reached a tipping point. I purchased a domain name for my wife and created a placeholder site on my server. When I added an SSL cert for the new domain, all of my sites went down (again), and I kept getting cryptic Apache errors (again).
I seriously considered switching to a hosted service and giving up the Mini, but my prior experience with hosted services was horrible, and it would likely cost even more than what I pay for the Mini.
I decided to focus on getting out from under Server.app’s grip. Two of the most appealing paths:
- Go the Homebrew route and install all the key software (Apache, SQL, PHP, etc.) via Homebrew.
- Run a Linux server in a VM.
I love Homebrew and use it frequently on my MacBook. I knew it would work well on a server. However, when I gave it a try, I had the darndest time getting Server.app to let go of resources. I was running into conflicts left and right, even after uninstalling Server.app and running cleanup scripts. I put Homebrew on hold, thinking maybe I’d need a clean install of macOS to build on, but I wasn’t ready to nuke my server just yet.
I started looking into virtualization. Having worked with server virtualization (Proxmox) at my day job, I was excited to give a virtualized environment a try on my Mini. The Mini is not a powerhouse, and only has one network card, but I figured I run VMs on my MacBook Pro all the time, the Mini should be able to handle it as well. Worst case scenario, it would be a learning experience and I would go back to macOS or maybe a commercial hosting service. Best case scenario, I’d have new life for my server.
I downloaded VirtualBox and used my MacBook as a testing ground to see if I could get a proof of concept up and running. I managed to get a simple but powerful LAN going in just a few hours — pfSense handled all NAT and port fowarding, and an Ubuntu server VM provided the LAMP stack. It was working very well for a proof of concept, but I still had reservations about macOS running underneath, and those pesky conflicts caused by Server.app on my Mini.
I decided it was time for a clean install of macOS on my Mini. I got in touch with MacStadium’s (formerly MacMiniColo) very helpful support staff, and they mentioned VMWare’s ESXi was available for their customers, and that they’d handle the ESXi installation, free of charge.
If you’re not familiar with ESXi, it’s VMWare’s free hypervisor offering. Similar in concept to VirtualBox, but designed to be run “bare metal”, as an operating system on the hardware, not on top of macOS. Since ESXi runs as an OS, it’s notoriously tricky to install on a Mac, especially if your server is hundreds of miles away in a data center. I jumped at the chance to get it installed by folks who know what they’re doing.
I spent the last three weeks sorting out the architecture and am pleased to announce it’s all up and running. My sites, including this one, are now being served via an Ubuntu VM on ESXi, running on my Mac Mini in Las Vegas. Finding documentation for Ubuntu has been super easy, and tasks that were previously time consuming and manual, such as obtaining and updating Let’s Encrypt certs, are now completed in a few minutes.
It was a time-consuming transition, which explains why my sites were down for so long (sorry), but I’m really glad I made the switch. A few weeks in, and I don’t miss Server.app or macOS at all. If all goes well this server setup should last for years (with security updates, of course).
I hope to write a more detailed account of the architecture in a future post.