deleted by creator
deleted by creator
Mostly just as a wrapper for Docker. The main issue I’ve run into is Docker’s union file system functionality doesn’t work when backed by ZFS, so disk usage can balloon out of control. I wouldn’t use this in production but don’t tell me how to live my life mom.
Beyond various Docker stacks I also have a Certbot container that uses Snap (sigh), and Hashicorp Vault container which runs as a vanilla SystemD service. I run Wireguard as part of my OPNSense VM. That’s something I would run in a VM since it’s exposed to the internet. I have an older MinIO and Concourse CI Docker Compose config that I’d love to run in LXC but I suspect that isn’t realistic.
Note on Vault, I haven’t been able to get mlock to work (used to prevent sensitive memory from being swapped). By all accounts it should just work in LXC, but since it isn’t and there’s no swap on the host I just turned it off. I may migrate Vault to a VM at some point.
I’m personally just interested in lightweight environments with good enough isolation and don’t break all the time over nothing. Docker mostly accomplishes that for me. LXC + Docker also mostly accomplishes that.
(My heart yearns for FreeBSD Jails but with decent tooling)
I originally excited by Podman, but ultimately migrated away from it. Friendship ended with Ubuntu and Docker -> CentOS and Podman -> Proxmox + Debian LXC (which has its own irritations but anyway). Off the top of my head:
I brought all this up in another community and was told the problem was [paraphrased] “people keep trying to use Podman like they use Docker” - whatever that means. I do like a number of design choices in it, like including the command used to create containers in the metadata, and how it’s easy to integrate into SystemD for things like scheduled updates.
Cockpit is pretty slick though, need to install it on my bare metal Debian host.
It’s easy* to setup Hashicorp Vault with your own CA and do automated cert generation and rotation, if you are willing to integrate everything into Vault and install your root CA everywhere. (*not really harder than any other Vault setup, but yaknow). I may go down this route eventually since I don’t think a device I don’t control has ever accessed anything I selfhost, or ever will.
I have a wildcard subdomain pointing to my public IP, and forward port 80 to an LXC container with certbot. Port 80 appears closed outside the brief window when certbot is renewing certs. Inside my network I have my PiHole configured to return the local IP for each service.
Nothing exposed to the internet at all. There is a record of my hostnames on Let’s Encrypt but not concerned if someone will, say, deduce apollo-idrac is the iDRAC service for a Dell rackmount server called apollo and the other Greek/Roman gods are VMs on it. Seemed like a house of cards that would never work reliably, but three odd years later I only have issues if a DNS resolver insists on bypassing my PiHole. And that DNS resolver is SystemD-ResolveD which should crawl back into whatever hellhole it came out of.
They could hijack your site at any time, but with a copy of your live private certs they (or more likely whatever third party that will invariably breach your domain provider) can decrypt your otherwise secure traffic.
I don’t think there’s significant real tangible risk since who cares about your private selfhosted services and I’d be more worried about the domain being hijacked, and really any sort of network breach is probably interested in finding delicious credit card numbers and passwords and crypto private keys to munch on. If someone got into my network, spying on my Jellyfin streaming isn’t what I’m going to be worried about.
But it is why CSRs are used.
Friction between Snap and AppArmor is to be expected. The corporate sponsor of Snap, Canonical, is well known for their icy relationship with the corporate sponsor of AppArmor, Canonical.
I’ve found the idea of LXC containers to be better than they are in practice. I’ve migrated all of my servers to Proxmox and have been trying to move various services from VMs to LXC containers and it’s been such a hassle. You should be able to directly forward disk block devices, but just could not get them to mount for an MinIO array - ended up just setting their entire contents to 100000:100000 and mounting them on the host and forwarding the mount point instead. Never managed to CAP_IPC_LOCK to work correctly for a HashiCorp Vault install. Docker in LXC has some serious pain points and feels very fragile.
It’s damning that every time I have a problem with LXC the first search result will be a Proxmox forum topic with a Proxmox employee replying to the effect of “we recommend VMs over LXC for this use case” - Proxmox doesn’t seem to recommend LXC for anything. Proxmox + LXC is definitely better than CentOS + Podman, but my heart longs for the sheer competence of FreeBSD Jails.
Do you have any trouble with cooling or anything with them? Got like a billion unused PCIe lanes in my Dell R730 and can think of a few things that might benefit from a big NVMe ZFS pool.
Likely an attempt to claim there’s fewer calories per slice, even though people will just cut it in quarters instead of fifths.
@TrenchcoatFullofBats I think this is the winning answer. Looks like it’s about a 1060 6GB, which should be enough horsepower for several desktop VMs, and keeps open my full profile slots should I ever want to install something even more powerful in the future. vGPU support is also nice so I don’t have to juggle which VM gets which GPU.
Only issue I had with a similar setup is turns out the old HP desktop I bought didn’t support VT-d on the chipset, only on the CPU. Had do some crazy hacks to get it to forward a 10gbe NIC plugged into the x16 slot.
Then I discovered the NIC I had was just old enough (ConnectX-3) that getting it to properly forward was finicky, so I had to buy a much more expensive ConnectX-4. My next task is to see if I can give it a virtual NIC, have OPNsense only listen to web requests on that interface, and use the host’s Nginx reverse proxy container for SSL.
If you live by a major university, they likely have property disposition where you can pick up slightly older equipment, sometimes for super cheap.
Absolutely, and you can generalize that to any sort of requirements are worthless if they don’t have as much teeth as the power they are trying to check.
I’m not going to watch the video, but what’s the procedure for switching between Linux and Windows? Usually you dedicate a GPU entirely to VFIO, with a 2nd GPU for the host OS (or run headless).
Anyway, will it work? Yes, minus some anti-cheat software. Will it be a simple solution? Well, once you get things stable, yes. The tech behind this is mature, but it can be a rabbit hole.
I would look into a non-Nvidia GPU for your 2nd PCIe x16 slot (x4, shared with the 2nd M.2 slot FYI). Good idea to check IOMMU groups before buying anything, but modern AMD motherboards are usually fine. Blacklist the Nvidia drivers and dedicate the 3070 to VFIO to make your life easier, and run Linux off the secondary GPU. Intel A380 might be a good choice. Do gaming stuff on Windows and stream via Parsec/Looking Glass/Moonlight+Sunshine; everything else on Linux.