Quinlan Project

Home Server

I feel like this is an essential thing every person should do to demystify computers and the role they should have in your life.

All you need is an old laptop you have lying around and some patience, and you can save so much money on cloud data storage/photo backups.

This is a guide. Not a tutorial. I am not liable for you taking any advice, especially if you loose data or piss someone off and they DDOS you. Please research, Google questions you have, and persevere.

Benefits

Before even launching into it. Here’s what I don’t have to pay for anymore because I have a ~$200 home server:

And things I’m ‘safe’ from now:

I own (or at least it feels a lot more like I own) my own data. I honestly think it’s not that hard to do so, and anyone who tells you different is trying to sell you something.

“Ok but why should I do this…

…that sounds complicated” I hear you say, and yes. It can be. But this is a blog post—not a Tumblr blog, a real flesh-and-blood blog made in the year of our lord 2024—so there are good odds you’ve already been forced to learn HTML and host a website of your own. If you haven’t good news, it’s “easy”. Web development is easy. So is server hosting once you know a little web dev.

Also, this isn’t an open invitation to test my security, but I am self-hosting this website 🥰. As long as you listen to this beginners’ advice you can safely and securely host all of the services listed before, for free (minus cost of hardware and electricity).

What is a home server

Basically servers are just computers. A revelation I didn’t realize until late into my computer science career. They have all the same components, but the difference is they usually have more hardware resources, which is to say more RAM, more CPU Cores, and more Storage. I might make a ‘what is a computer’ post #TODO later. Cloud Storage Services like iCloud, DropBox, Google 1, etc. are just servers. Cloud Web-Hosting Services like Linode, digital ocean, etc. are also just servers. So what’s stopping you from making your own? Not a lot. You just need a computer of some kind.

Where to get one, and what to consider when browsing

There are plenty of creative ways to get a computer. My first home servers were a pair of Dell All-In-One computers from my old job. If you have a friend that works in IT, ask them to save some computers for you that were meant for the trash. Most of them still work, but they have drive failures.

Also think about going to an E-Waste Recycling plant. It’ll feel like shopping for your next car at a trash heap, but we actually don’t need a beast, just an ok computer with 16 gigs of ram.

Old gaming computers are also good, if not a little power hungry. Rip out the graphics card (GPU) and sell it unless you know there’s something dumb you want to do, like self-hosting your own fox-girl waifu chatbot. In that case, yeah, keep the GPU and make sure it’s being “passed in” correctly to your buff werewolf Korean-mobster husband. Otherwise no GPU, use C.AI like a normal adult.

Architecture

As you start digging into this, you’ll see that there’s 2 options for cheap home servers. ARM servers, like Raspberry Pis and smartphones (and Apple Silicon computers when they get old and someone figures out how to install Linux on them), and low-tier x86 servers, like old laptops/desktops and some cheap “looser-tier” mini PCs. I say “looser-tier” lovingly, as a low-spec x86 is what I’m using as of 2024, because it’s compatible with most things and draws hardly any power. As time goes on and ARM takes off in popularity, the which architecture consideration will likely skew away from x86, but that’s a long way off. Their exact differences are nuanced, but ARM is more power efficient at the cost of using a different instruction set, meaning it’s compatible with less software.

And just to hedge my bets, I should also mention RISC-V is currently getting some hype, but that’s the bleeding-est of bleeding edge and I can’t be bothered no more than you can to look into it.

Resources

There are cheap computers available on Amazon for $100—at time of writing, on sale—that are 4 core “N100” processors and just good enough for most usage. I’ve had good luck with it so far, with only one bug causing the whole thing to crash that I fixed by updating the service that was causing it. (Nextcloud Image preview would overload the RAM and crash the computer.) I’m making do with 8GB of ram, but I would suggest 16GB just in case.

Get a CPU that is relatively new, the higher the number the better. It’s worth knowing that newer generations of CPUs are usually better than older ones, and the I# for intel CPUs just indicate how good it was relative to the other CPUs in that generation (gen), so a 13th gen i5 is going to be better than an 8th gen i9, etc. When in doubt google benchmarks and compare the two.

8GB of RAM can work, believe it or not, but 16GB is preferred.

I only have 64GB storage, and I have an external hard drive (USB 3.0 NVME in my case, so it’s not slow despite being plugged in outside the PC) with 1TB of storage. This is entirely up to you and how much space you think you need. If you don’t know, 1TB storage total should be plenty for all your media and personal files.

Making your own lil data center

So you have a computer. Whether it’s an old laptop or a new amazon junk-tier mini PC, What are we installing on it? Linux? Windows Server 2000? Not so fast.

I want to get into virtualization a little bit. And Packets. What we ultimately need to do is make a bunch of little servers out of our one server, and make sure traffic is only getting from where it is securely coming in to where it needs to go. This would suck. Like the fattest load, but there’s tools out there that make it alarmingly easy.

Services and how to use them

Services are what we’re running on the home server. They’re like HomeAssistant for smart home automation. Pihole for local DNS and adblocking. Nextcloud for self-hosted contacts storage, calendar, files, cookbook recipes, etc. Jellyfin for playing back your video media anywhere. Foundry for a Dungeons and Dragon virtual table top. Gittea for your own private GitHub.

Point is you have options. Anything you use a website to do, you can use your home server for… Except email. Don’t do that, you’ll get blocked by spam filters.

Here’s a server running some services

(I’m not an apple girlie, I’m just at my parent’s and Freeform is the only thing I got right now ;))

Cool, we’re good, right? We can just install an OS (or just use whatever’s installed) download all of our stuff, run them, and be good. Well normally yes, but there’s a problem.

Problem 1: Accessing the service locally

Well there’s a lot but let’s address this one first. A lot of services use a web UI (The User Interface you interact with the service in). Meaning you go to your browser, enter http://[the server's IP address] and the webpage that pop’s up is how you interface with (control/use) the service.

Quick Primer: Domains and DNS for Home Servers

Every HTML URL like google.com or mail.yahoo.com resolves (or secretly becomes in the background thanks to your browser) to an IP address through something called Domain Name Service (DNS). We’ll learn more about that later.

When you’re accessing your service, instead of a domain (the part of the website you type out to visit it; also the part that DNS resolves), you can use an address. Either the IPv4 address (e.g. 100.234.56.7), an alias (e.g. my-server), or the IPv6 address (e.g. [1234:5678::90ab:cdef] in square brackets), or if you’re accessing it from the same computer you’re using it on you can use localhost (localhost). To use any of these, you’d type them in your browser like any other URL: http://{address}/{etc}. The etc. is just the rest of the URL, for example, “login.html” or “blog/posts/home_server”. Like how to log into Pinterest you go to https://www.pinterest.com /login or https://youtube.com /watch?id={id}. BTW, that IPv6 tip ended up saving me once, so it’s worth knowing. If you set up PiHole’s “local DNS” later, that’s the easiest way to do aliases, but IPv4 over HTTP is going to be how you connect 99% of the time at this stage.

Ok so what’s the problem? Well if you have two services both with a login.html, the computer won’t know how to handle it. I mean actually, they’ll try to run on the same port and one of them will complain that another app already claimed or “bound” to the port it wants. So, we run them on different ports. A port is just a number you can assign to a service so web traffic gets routed to it correctly. Some common defaults are here but the big ones are 80 is HTTP, 443 is HTTPS. Gamers who have ever had to port forward their router know Minecraft is famously 25565. Doom is 666. We’ll port forward 443 later, but for now we need to know (1) you can specify a custom port by using this syntax (the correct way to write code/urls/etc.) http://{domain}:{port}/{etc} and (2) if you or the service’s developers don’t specify a custom port, it’ll default to the protocol’s (in this case HTTP’s) default (80) so for example https://www.google.com is secretly https://www.google.com:443 and all your web traffic (at least going to google) is on port 443.

Ports are like highways, so your Minecraft packet isn’t shouting over your web traffic. Packets are the discrete parts of internet being sent off from your router and modem to other servers. All packets are either TCP or UDP. TCP is for file transfer, email, web browsing and UDP is for online gaming, video streaming, DNS lookups, etc. Basically TCP is for when you want to make sure a packet is received because it’s “connection based” protocol. While UDP is for if you don’t care if a packet is lost like when streaming data “connectionless”-ly. TCP and UDP each has their own set of ports 1-65535 (16-bits). Both send information containing other protocols like SMPT, HTTP, HTTPS, Minecraft, etc. If you ever don’t know if an app uses TCP or UDP, it’s a safe bet to do just TCP or… do both.

Lucky for us, a lot of the services we’d want to use have default ports assigned. HomeAssistant uses 8123. ProxMox uses 8006. We’re going to be using a lot of services, so this is a good thing.

So to access a service we know we need 2 thigs: (1) what IP (server or virtual server (more later (I’m sorry it’s a lot of information (take a second to breathe)))) and (2) what Port a service is running on, on that server.

So now we’re ready to go? All my services are running on one computer with different ports?

I mean yeah, technically. If you installed everything according to the tutorials, and put http://{Avocado IP}:8008/ you would get back Avocado’s web UI. But there’s another problem.

Problem 2: Accessing not locally

When we want to start accessing Avocado from our phones, other laptops, etc., we’re not always going to be on our local network. So the IPv4, alias, IPv6, and localhost only work if we’re using the machine on the same network (WiFi or Ethernet).

I’m going to suggest some ways to manage our services later, so don’t install anything just yet. Unless you want to, it might be a good exercise.

Quick Primer: Your home network and your home servers

You’re using a computer. That computer has a local IP. You can google how to find this out, usually by typing ipconfig into your terminal. If you’ve never done this go ahead and do it now. Same goes for your old laptop. ipconfig again. Different IP!

Now if you google “What’s my IP” Google will just tell you what your public IP is. Do it on your old laptop, SAME IP, different from both local IPs!

Your “Internet Box” that you call a “router” is actually a couple of things. I’ll be using a post office analogy, so packets can be thought of as letters, which is fitting because packers are headers (envelopes) marked with IP addresses (home addresses) with data (mail) inside them.

  1. A Modem, which is what connects you to the broader internet. Everything running from your Modem to a wall somewhere in your house (assuming you don’t have a Verizon 5G box or something) is the Internet. Everything inside the “Internet box” and plugged into it/otherwise connecting to it is your local area network or LAN. LAN is sometimes also called WAN for our purposes, but feel free to Google the difference. When you’ve heard of nerds of old coming together to have a LAN party, this is what it meant. They brought all their computers together and connected to the same network. In the post office this can be thought of as an international mail carrier. We’re leaving the country and are using international addresses (adding USA to our letters, etc.)
  2. A router. This is the major post office. It routes the mail. It looks at the address and decides if it needs to go internationally (Internet) or stay local (LAN). It screens for bad mail (that’s your firewall). It also takes traffic from the Modem coming in and blocks all of it… unless you forward the ports. We’re not going to forward ports yet, because we don’t want just any 443 traffic coming in be patient…
  3. An Ethernet Switch and
  4. A WiFi Access Point. Both simply are roads. The mail truck knows where to turn at each intersection (switch/access point) to eventually find it’s way to the house the packet is meant for (computer/phone/etc.). Switches are just smart enough to route traffic, not make decisions like routers.

Quick Tangent: They make all of these as standalone devices, but odds are you were given an “Internet Box” from your Internet service provider (ISP; Comcast, ATT, Spectrum, Verizon, etc.). I recommend having your own Router and switches/access points; it’s your home network, may as well use your own equipment if you can. Also we should be campaigning for the government to give us the freedom to use our own modems, but putting your router in bridge mode does about the same thing as turning it into just a modem.

You can go into your router and manually choose IP addresses. There’s another address called a MAC address that uniquely identifies every computer in the world. It can be spoofed btw, but in general every one is unique. Your router assigns every PC/phone/etc. on the network an IP through something called DHCP. Basically the computer asks nicely for an IP to the router and the router’s DHCP service assigns it an IP. I’d take a moment now to log into your router (google it, it’s going to be shockingly similar to how we access our home server later) and make a note of what your server’s local IP address is! On top of that, make a DHCP Reservation so it doesn’t accidentally change in your router’s Web UI.

Before we solve Problem 2 of how to access our servers from the wider internet, lets isolate them better, so if an internet baddie or a random bug takes a service down, it doesn’t take the rest of the server with it. Worst case, the bug happens on boot and we’d have to wipe the server and start over!!

Problem 3: Isolating the services

Lets say Avocado is using some dependency that Lemon also uses. Dependencies are stuff the software needs to run, like how Minecraft needs Java to run… Sorry for alienating everyone that has never played Minecraft lmao. And let’s say Lemon crashes. Avocado is coming with it. Or say they’re not sharing a dependency, and Avocado does something to crash the whole operating system!? Well what are we going to do about that? Just buy a new server for every service. Seriously, yes. In the past that’s how they actually did it. And it was wasteful, because not every cranny of hardware resource was being used.

But these days we have technology!

Quick Primer: Making one server many with virtualization

So we want to make sure the services aren’t going to take out the whole physical server if it goes down. How? Virtualization.

Option 1: VMs

You’ve probably used VMs before to try out Linux. If not I’d recommend trying it out. It’ll give you a feel for the idea that you don’t just have to run programs designed for your operating system… And now that you have you should also have gathered that it’s SLOW. Running an OS on a different kind of OS means you need to have the software resources for the virtualized OS (the one in the VM) translated to ones your host OS (the one running the VM) can understand. It sucks. What are our other options?

Option 2: Type I Hypervisor

Ok, so what if the host OS is just in charge of running virtualized OSs and it doesn’t do a whole lot else. Like, the host OS is just virtualization software, so it doesn’t need to do that translation step. That’s called a Type I hypervisor.

The Type II hypervisor is the one described in the last section, where the hypervisor is the software (like VMWare Workstation, Virtual Box, Hyper-V) hosts the VM.

Lucky for us, there’s a free and open source operating system called ProxMox Virtual Environment we can install on our computers to do exactly this. It’s based on Debian, really lightweight (not large on the disk, and doesn’t slow down performance really at all for other Linux OSs), and free for personal use!

It lets you take backups, access your computer’s CLI (command line interface, most server computers are headless, which means they don’t have a desktop, mouse, etc.), and a bunch of other stuff. I also hate using a cli, so we shouldn’t have to touch it much beyond initial setup. Also some services like HomeAssistant really don’t like being ran in containers (more info on containers next section; see their site where they explain the benefits of them having their own VMs and/or “Supervised” admin access to their VM).

This is what I’d recommend putting on your home server It can be a little tricky to install. Follow tutorials. Make sure that, but once you have a computer with it up and running you should get this message!

Welcome to the Proxmox Virtual Environment. Please use your web browser to configure this server - connect to:

https://{your server's local IP}:8005/

proxmox login:

Now you can follow some other tutorials to get set up, but the basic process is you can upload images (operating system installers, ISOs, etc.), make a new VM with them, provision resources (e.g. deciding how much RAM/CPU Cores/Storage each VM gets).

For what VMs I recommend, you’ll want:

  1. HomeAssistant OS (remember which architecture you have, don’t follow that tutorial just get the ISO) as it’s own VM.
  2. You’ll also want a VM for services that don’t mind being containerized (most all of them; more info in next section).

Option 3: Containers

Ok but I don’t want to make a separate VM for every service I use. I also want to be able to install/reinstall services easily. Stop them. Start them. Have their own contained logs/CLIs and security configurations. Ok nerd. Happy to hear that because that’s what Docker is for. Their site.

Docker is like the old Type II Hypervisor option, but we don’t have to worry about the overhead (extra computer effort) of running a second OS. Containers are limited in what they are able to access, so they’re unable to affect each other, while also still being able to share resources. It’s magic. It’s like a half-VM. It can crash and leave the rest of the containers totally fine. They’re also designed to be temporary, so every time you spin up a VM it reinstalls the software (making updates really easy) and works off of only the files you specify (via Docker Binds).

Quick note: Docker containers are made by developers into what are called Docker Images. The configurations for these images (variables, file mapping, etc.) are usually made by the community as either install scripts (docker run --env FOO=1 --env BAR=2 --publish 8008:80/tcp --volume /my/data:/media/ user/image:version) or Compose files (Google examples, they use something called “YAML”, sample Plex install below). Docker-only gang with no Casa or Umbel has to install compose separately. CasaOS and UmbelOS should be able to import them, or their respective “App Stores” should have a lot of what you need preconfigured as a one-click install.

services:
  plex:
    image: linuxserver/plex
    container_name: plex
    network_mode: host
    ports:
      - 32400:32400
    environment:
      - VERSION=docker
    restart: always
    volumes:
      - ${PLEX_MEDIA_PATH}:/media/

We got:

Even if you use CasaOS or UmbelOS, these concepts are inescapable.

“Wait that’s it?”

“Also that wasn’t a primer, that was a whole crash course”

Yeah. Sorry, but congrats. You now have a hobby that pays for itself. All the software is free, now it’s just a matter of finding what works for you. You’re most of the way there

“Wait, what about accessing it from my phone and using WebDAV?”

Oh dang that’s right! Back to Problem 2.

Problem 2, contd.: Accessing not locally

You want to be able to turn off your lights from the HomeAssistant app and have your notes available on your phone? You have options!

All options are FREE!!

Bad option: VPN

You can set up your own VPN and use that. As a reminder, a VPN (Virtual Private Network) isn’t a privacy tool, it lets you connect to one or many LANs as if you were on them. Or at least that’s why they were invented. Don’t get me wrong VPNs are useful if you’re traveling internationally and don’t want to see French ads (me #relatable), but it’s also a away for you to access your home server, but the kicker is, you have to be on your VPN.

I don’t like this solution, but it works for some people. It was a nice little add-on though if you travel a lot and want to watch American shows/YouTube.

Ok option: DDNS

You need this if you’re just using NRPM + DNS on top of it. If you go the Cloudflared route I think you can skip it.

You have to set up a Dynamic DNSS! Your ISP can change your public IP whenever it wants. These services monitor for changes to your IP and basically maps it to a static URL. Here are some recommendations:

Real quick. With DuckDNS and a lot of these services, they will either have you make an App Password or use a Token. Both are random passwords that are in theory safer to use in third-party (apps not made by the guys who make the first app, like the DuckDNS Windows Client, etc.)

Configuring DDNS means you can now access everything with http://{your ddns url}:{service port}, as long as {service port} is port forwarded on your router.

But it’s not HTTPS, which is free and easy thanks to NRPM.

Good option: NRPM + DNS

Configure DDNS in above section.

NRPM

So we need to do a couple of things, but a good 1st step is just setting up NGINX Reverse Proxy Manager (NRPM). It’s a container that lets you set up a reverse proxy easily. A forward proxy is used in offices to take in a bunch of traffic from many IPs and put it all behind one IP address. A reverse proxy does the opposite, taking in a bunch of traffic from the internet and sending it to the right server and port! It’s fairly easy to set up, but some tips: (1) some apps need webhook support, but it should be off if it’s not (2) if you use a wildcard certificate (more later) you need to force HTTPS and if you force HTTPS in Cloudflare (more later) you need to turn that option OFF.

I recommend setting up a wildcard certificate. So after you register and pay for a domain (you can get one super cheap; $1/yr at .xyz, google it, though it may not work with Cloudflare later) you can certify that the NRPM application is where the web traffic is supposed to go. You’ll have to do a DNS challenge to prove this, but it’s usually as simple as copying a code from the domain registrar (where you bought the domain) into NRPM. Optionally you can point the domain’s name servers (where the DNS happens, big servers owned by registrars usually) to Cloudflare, and set up the domain DNS in Cloudflare (may not work with .xyz domains). This will be useful later for when we configure Cloudflared (more later).

After being certified, you can port forward 443 traffic to the server with NRPM on it in your router. Configure your DNS to point with a CNAME to your DDNS URL.

DNS

Now you can add a CNAME record to your DNS mapping your domain name (and optionally subdomains) to your home server. If you configure traffic to be sent to a subdomain e.g. CNAME plex.my.com my.duckdns.org you can then go into NRPM and add a Proxy Host for plex.my.com to http://{server IP}:{port}. Add your wildcard certificate.

For stupid reasons I won’t get into, you should add a Proxy Forward my.com to www.my.com if you do self-host your site, and if you’re using Cloudflare you can add a CNAME record pointing my.com to the home server using some proprietary nonsense magic.

Set up all the subdomains you need in NRPM and point them to the services you need (this isn’t hard, but each service is different and may take some tweaking)

The goodest Option: Cloudflared

Now we’ll talk about CloudflareD, with a D. Just HTTPS alone does not a secure site make. Using Cloudfared means you can use their Tunneling service to keep your connections secure and anonymous. You don’t want randoms on the internet finding out your home IP in theory. On top of installing the Cloudflared container, also limit IP traffic on ports 443 (and 80, but may as well leave that closed) to the IP ranges they publish on their website.

It’s really easy to configure, just drop the link they give you into the Cloudflared container’s Web UI.

Now you’re DDOS protected, too, in theory. Also you can close the ports on your router so no more sketchy internet traffic will hit your router and all requests will have to be routed through Cloudflare!

Now we’re done!

You can go to https://{subdomain}.my.com and it’ll hop invisibly from there to Cloudflare DNS to Cloudflare tunnel to your home IP to your router to your webserver to the virtual server to the port the service is running on in the span of a few milliseconds!

Thanks for reading, if you have questions contact me on my socials in the about section. Cunningham’s Law applies ❤️ I’m not an expert.

Other Considerations

Pihole Aliases

Also set up Pihole local DNS aliases, I like to set it to {server}.my.local but you can set it up to be whatever.

WebDAV, etc.

WebDAV, CalDAV and CardDAV are open source ways services can connect to your Nextcloud and get access to your files, calendar, and contacts respectively. Thanks apple for pioneering and open sourcing this (now if only you were so kind about wallet passes). This just means you might be shocked how many random things support your Nextcloud because they use the same tech as iCloud!

Back up your files!

I’m storing everything on cloud storage I have through proton, but another external hard drive is also a very valid, cost-effective backup solution. Ideally you’d have one copy where you live and one somewhere else (relative’s house, secure cloud service, etc.). All stored redundantly, ideally. You should also have one copy in cold storage (not a plugged-in/running drive).

Password management

You can self-host your password manager too! BitWarden is the current open-source leader, but ofc this statement can always age like milk. I just buy their hosted option for $10/yr.

Tags: