For the past month I've tried to see how much of my digital life I can switch from mainstream providers of cloud services to my own personal self-hosted VPS (Virtual Private Server) appliances without inconveniencing myself . Why? Well, partly out of curiosity and a desire to experiment, but mostly because I've read too many security scare stories recently to fully trust companies not to mess with me by claiming copyright on my own documents, fully safeguard my data and credentials from targeted breeches and resist government attempts to mass survey (what should be) private data.

For those who don't know what a VPS is a virtualised (or 'pretend') server instance that runs on top of a single virtual/physical host. This is a great way of getting more "bang for your buck" when you buy real server hardware as you get several servers out of one piece of server hardware, it makes backup and duplication of those servers very easy and it means for those of us who don't have hundreds of thousands of pounds in our back pocket to buy our own server hardware/firewall boxes that we can rent own 'servers' cheaply from providers like Digital Ocean, Amazon S3 and OVH.

You don't have to live with a tiny amount of space on someone else's server to run your own website/web-based application and you get exactly the same level of control you would with a dedicated server, but it's your responsibility to maintain the system and keep it secure/backed up. In short, it's the "big bazooka" solution to the problem ...and like a big bazooka, if you use it wrong you can blow your own face off!

VPS Diagram

Website/Personal blog

To put it mildly, I've recently been having serious problems with my existing web host. Not only did they seem to have issues keeping their own infrastructure 'up' but they seemed to be allocating a very paltry level of hardware resources for the money I was spending on it (with the net result that if I had more than a few visitors hitting my personal blog at the same time, either PHP would crash complaining about lack of memory or MySQL would crash).

So, not being particularly satisfied with this I switched to another service and configured a VPS. On this occasion I used a pre-packed image with Ubuntu server as a base, imported all content across, re-setup various plugins and hardened the box against attacks. I suddenly found I was paying a fraction of the price but had a robust solution that wasn't falling over every time it got hit by fairly modest demand.

It wasn't perfect though. Ubuntu Server seems to not be the best system for availability as it seems to fall over periodically due to intermittent problems. This makes sense, given Ubuntu normally exists as a desktop GNU/Linux distribution. Debian and CentOS are the GNU/Linux distributions we'd normally choose for server applications as they're focussed on providing stable tools from the ground up instead of providing the 'shiny new features' developers like myself usually want to play with!

I initially put in place two pre-built Ubuntu boxes and relied on DNS to auto-switch between them when they flaked out. I later discovered the reason they were flaking out is I'd only allocated 512MB of RAM and MySQL (the database that drives Wordpress) is very prone to memory leaks. Increasing the RAM to 1GB and periodically restarting the database server completely eliminated the need for failover, and I opted to just keep regular backups & snapshots instead.

ownCloud logo

Cloud file storage

So, by this stage I had my website hosted in a fairly reliable VPS solution. But I wasn't satisfied with this - my files were still scattered across a multitude of free file storage services like Dropbox, OneDrive, Google Drive, iCloud, Amazon Cloud Drive and others. While this wasn't costing me any money (and are all very convenient), for the problems I mentioned at the start of my blog post it makes sense to have a go at hosting this kind of service myself.

After doing some research and experimentation with a Raspberry Pi connected to a 1TB external drive the system I found most closely mirrored the same functionality is an aptly-named tool called OwnCloud. This is a tool you simply download, extract, shift to a web-based directory and configure/harden as you like, after which it exposes both a browser-based interface and interacts with a desktop client in much the same way Dropbox and others do.

By default you can make sure all files stored are encrypted on the server side, and the system also seems to host calendars quite well (providing CalDav & WebDav) access. And recent changes in Version 7 give us some really useful features like the ability to share files and preview/make minor edits to documents in the browser. I seem to be having trouble getting it to import contacts, but apparently it's a known bug that's being worked on.

You'll notice that after extensive testing/hardening the server I'm so confident in this setup you'll spot the files I normally host publicly from Dropbox on my More Articles page is now being served by my OwnCloud VPS. As you'd expect I've taken SSL technology as far as I can to make sure file transfer/downloads are as secure and private as they can reasonably be.

The setup itself is extremely straightforward, and you can find some great tutorials online that take you through the basics step-by-step (here's an example I initially followed on a CentOS 7 VPS setup). The only complexity has been making sure my SSL certificate infrastructure was working properly in all browsers after I moved from my own self-signed setup to one offered by an approved authority! So far the only downtime has been when I've configured with the server wrong while tinkering - as you'd expect from a GNU/Linux distribution based heavily on "Red Hat Enterprise Linux" it's an extremely robust and stable system.

I think in future it would probably be more straightforward to use a pre-built box and customise it. But as a learning experience this was still worthwhile.

iRed Mail logo


As you may (or may not) be able to tell from the logo above this title I used iRed Mail as a base setup for my self-hosted system, using this tutorial as a starting point. It's particularly useful as it helps install & configure various security tools that would otherwise take many hours to setup, has a good control panel and installs Roundcube (an open source webmail interface) by default.

There are a few annoying limitations though, such as the fact it doesn't support CentOS 7 yet and the developers apparently think it's fine to not support SELinux (an important security subsystem used in Fedora/CentOS/RHEL to limit the impact of malware or hacks by only giving apps & processes the bare minimum permissions they need to perform their particular function). In fact, they actually tell people to turn it off in their own tutorial which just adds insult to injury! My current test setup therefore uses CentOS 6.x and has a whole host of other security measures setup to make up for the lack of SELinux support.

It took some time to fully trust it, but I have gone live with my self-hosted email! By and large I've had no issues with it, although some Exchange servers still seem to insist my emails are spam, even with the relevant measures I took (see next paragraph)

This week I also managed to ensure email sent from the email address this serves didn't automatically end up in people's' spam folders! Doing SPF and DKIM checks went a long well to helping reassure other mail systems that my emails came from a legitimate source, as did providing an SSL certificate chain from a reputable source rather than relying on the automatically generated self-certified SSL configurations generated by iRed Mail.

How long I continue self-hosting email rather than falling back to the usual providers remains to be seen, but much like hosting my own cloud file storage this has still be a great learning opportunity.

Aren't you missing something?

Yes, I am! Information I've deliberately missed out here are things like:

  • Who's hosting my VPSs
  • Who I registered domains with/who's managing my DNS records
  • How I secured/hardened the servers and domain names
  • Domains/subdomains I use for my VPSs
  • The IP addresses/URLs of said servers
  • Which online and offline services I may or may not intend to link to my self-hosted email

With some effort you can probably find some of these things out, but for security reasons I've not explicitly disclosed them. Try reading Mat Honan's tale of woe if you're still not convinced how badly things can go wrong with seemingly innocuous information which (when put together) can be used for malicious intent/annoyance.

Next steps

Once I've finished migrating much of my digital life to these setups, there are a few ideas I have bouncing around in my head about where to go next. I still need to assess merits of each, as every system I self-host comes at the cost of my own time and money!

One idea I've had is to set up my own self-hosted VPN. This would be a trusted source I could use to encrypt traffic to and from devices I use on public WiFi or tether to cellular networks to foil hackers/snoopers. If I want to share files between specific people/offer remote assistance between machines/play online between friends I could just ask them to connect to it. However I could just setup Hamachi and Privoxy and/or a SOCKS proxy to accomplish similar results, so I'm still giving this some thought.

Another idea is self-hosting my own Firefox sync account to propagate bookmarks between devices instead of relying on Mozilla doing it for me (or Google/Apple/Microsoft in the case of other browsers). I still need to see what would be technically involved with making this happen, but on the surface at least it doesn't sound like a bad idea.

I'm also tempted to develop my own secure self-hosted alternatives to apps I use regularly like Evernote/OneNote and Things. If that actually happens, I will open source my work.

In short, there's some cool directions I can continue to go with what I've learned through all this that I'll be exploring over the coming year. Or at the very least will form a more constructive use of my time outside work/STEM volunteering/occasional travel than binge-watching Netflix and playing Civilization V, SimCity 4 and Minecraft!

Share on Diaspora*