This is a continuation of my other post
I now have homeassistant, immich, and authentik docker containers exposed to the open internet. Homeassistant has built in 2FA and authentik is being used as the authentication for immich which supports 2FA. I went ahead and blocked connections from every country except for my own via cloudlfare (I’m aware this does almost nothing but I feel better about it).
At the moment, if my machine became compromised, I wouldn’t know. How do I monitor these docker containers? What’s a good way to block IPs based on failed login attempts? Is there a tool that could alert me if my machine was compromised? Any recommendations?
EDIT: Oh, and if you have any recommendations for settings I should change in the cloudflare dashboard, that would be great too; there’s a ton of options in there and a lot of them are defaulted to “off”
I’ve tried different approaches with fail2ban, crowdsec, VPNs, etc. What I settled on is to divide the data of my services in two categories: confidential and “I can live with it leaking”.
The ones that host confidential data is behind a VPN and has some basic monitoring on them.
The ones that are out in the public are behind a WAF from cloudflare with pretty restrictive rules.
Yes, cloudflare suck etc., but the value of stopping potential attacks before they reach your services is hard to match.
Just keep in mind: you need layers of different security measures to protect your services (such as backups, control of network traffic, monitoring and detection, and so on).
has some basic monitoring on them.
What monitoring software are you using?
I feel like the other measures you talked about (backups, condom of network traffic, etc) I’m doing ok on. Its really just the monitoring where I’m stuck. There’s so many options
There are so many monitoring tools with various degrees of complicated setup / configuration or the amount of information you get. And honestly, I’ve looked into various tools: checkmk, monit, Prometheus… And realised that I rarely look into that information anyway. Of all “fancy” tools, I liked the ease of Netdata to set up and the amount of information that you get. However, beware that their in the process to make their free / homelad offering worse. I’ve been eyeing beszel and don’t forget CLI based tools that are avaible such as atop, btop, htop or glances.
If you want to delve deeper into the rabbit hole of monitoring, I can recommend you to read this article below: https://matduggan.com/were-all-doing-metrics-wrong/
By not making them publicly accessible. With Wireguard there’s really no reason.
Setup service to be active on a subnet, enable Wireguard to VPN into the subnet and use the services.
Yeah, I’m not gonna tell the 50 users of my plex server to set up wireguard on their devices so they can request movies and TV series on my overseer, when I can instead just use NPM to make it publically accessible with a password prompt
Your use case, and OPs, are completely different scenarios. I can’t tell if you’re being purposefully disingenuous or just flippantly stupid.
With Wireguard there’s really no reason.
Well, that’s kinda of a personal choice. If somebody needs to have services accessible by someone else besides him, that service can’t be behind a VPN (let’s face the truth: we know that we can’t ask all out relatives and friends to use a VPN).
There’s also something to be said about some services being cordoned off in a VPN while leaving some public with security. I don’t necessarily want everyone within my full network if all I want is to share one service with them.
For that, you can restrict access to a single service with iptables.
This is effectively the same damn thing with a single exception. If your VPN is down, there’s no access to your server. If for whatever reason your firewall is down, there’s unrestricted access to your server…
VPN is unquestionably the correct choice 100 times out of 100.
If for whatever reason your firewall is down, there’s unrestricted access to your server…
I don’t know what kind of firewall you use, but if my firewall is down there is NO traffic at all passing through!
And by the way, since I’ve replied to someone that don’t want to use VPN because he doesn’t want to give access to the whole network, I meant that he could use a VPN AND iptables to restrict the guest access to single services instead of the whole network.
I don’t know what kind of firewall you use, but if my firewall is down there is NO traffic at all passing through!
Only a hardware firewall would do this. If it’s software, like implied in your post, no traffic is filtered and all connections are accepted.
VPN is the least amount of work for the most secure setup. There’s nothing to even argue, its superior in every way.
Only a hardware firewall would do this. If it’s software, like implied in your post, no traffic is filtered and all connections are accepted.
Talking abut netfilter, since it manages also the forwardning, it for some strange reason it should crash, NO IP traffic is flowing
VPN is the least amount of work for the most secure setup. There’s nothing to even argue, its superior in every way.
If there’s nothing to even argue, then I say goodby to you since I’m here to discuss. All the best!
If somebody needs to have services accessible by someone else besides him, that service can’t be behind a VPN
Again, this is the reason VPNs exist. If that person needs access, then setup Wireguard…
It’s like saying you don’t need a front gate with an access code because then you would have to give out your own access code. But I mean, the lock has the ability to setup more access codes. And you’re saying the only viable option is the leave the gate open and hire a guard to manage access. It’s just… Weird and wrong.
Again, this is the reason VPS’ exist.
What? What’s the difference between a VPS and your home server? You may say that’s a good practice to separate things, so maybe have a a VM with public facing services and another with more private stuff reachable only with a VPN. But for something like Nextcloud, it needs to be public (if you’re not the only one using it), but it contains personal stuff and then comes the OP request!
You may say that’s a good practice to separate things
You’re missing the point. VPN isn’t about separating anything… I’m not even sure what you mean by that. VPN is the accepted practice here. Unquestionably. You create private services, and for security you only expose them to the least amount of people possible. You authenticate via VPN connections. You only have to maintain a single database of users to access any number of services, even tens of thousands.
OP is specifically talking about hosting local content that they want to protect. VPN is the solution here.
Well…if you edit your post after someone has replied to it at least specify what’s you’ve edited and don’t pretend that the answer that somebody else has already given you wasn’t about your non edited post!
If you (my mistake) wrote VPS instead of VPN, you can’t pretend that I’ve answered about VPN!
If you can convince your family member and your friends to use a VPN to use your service, that’s good for you, and I mean it!
But saying that it’s quite impossible to do that, I think that I’m speaking for 99% of the self hoster (is this correct in English? Bah, you got me!)The entire point of selfhost is to host private services not available to the public. By literal definition, that’s allowing only local traffic to connect to your services. It’s infinitely more secure. A VPN allows you to extend those services over the clearnet to authorized devices via virtualized networks. You don’t have to worry about messing with inbound/outbound ports, or worrying about software failure or misconfigurations accidentally exposing you to the clearnet. You don’t have to worry about DDoS, or abuse. Being attacked? Bring down your VPN and that completely shuts down your issue. Your network is completely unreachable by anyone but a local host.
There’s simply no room for an argument. VPN is objectively better in all possible situations.
The entire point of selfhost is to host private services not available to the public
Probably your entire point, a lot of self hosters self host services that family members and friends can reach most of the time without the need of a VPN. This very community is full of examples.It’s infinitely more secure
I’m with you about that.
There’s simply no room for an argument.
As stated in the other post, I’m sorry about that, I’m here to discuss and learn, if you don’t have room for an argument, our discussion ends here.
VPN is objectively better in all possible situations.
Exactly! in all possible situation!!!
I agree with WG however I need https for a few locally hosted items like actual budget so I have that through nginx proxy manager. I was debating adding Authelia in front with some of my others (audiobook shelf, home assistant and music assistant) as sometimes I disconnect from my home network and forget to reconnect.
Just out of curiosity, why do you disconnect from your home VPN?
Why not swap from nginx-proxy-manager to Caddy2, which can handle everything? SSL and reverse_proxy?
There should be an option in your phone VPN setup to reconnect if app X is being used.
There is. It’s called VPN Split Tunneling.
If you want to proxify your connection between you and a service, you enable the split. If you don’t care, or want to not use the VPN, then disable it for that application. So it’s effectively “proxify all connections to this app,” which is the same as your use case.
Check out crowdsec. Like fail2ban, but with crowdsourced lists on top.
will do, thanks
Wazuh
Active responses are like fail2ban but better
I’ll look into it, thank you
I would put this stuff behind VPN.
So there is https://en.wikipedia.org/wiki/Fail2ban which helps already to some degree.
But what are you trying to prevent? You have your services in a docker container, hopefully not running as root, which already makes it difficult to break out even if through a bug someone would be able to get access to the docker container.
I mean its not like your stuff is very important for someone to break in like the pentagon, you probably just have some photos from your phone on it, some lights can be switched on and off and some temperatures read.
I’m not trying to say that you should not care about it but I’m trying to figure out what your threat model is.
I feel weird about having those apps on the internet and basically being blind to threats. I mean yeah, I’m not a target on anyone’s list and most IPs visiting the site are bots but I would still like to know what’s going on.
I don’t work in tech for a living, this is just a hobby for me so I have limited time to work on this stuff and do research. It’s very possible I fucked something up and don’t know it. I figured if I at least got an alert that said “hey, your immich server db was dumped and sent to <insert IP>”, I could at least turn it off
Yah, it’s just a hobby for you, but it’s also a hobby for script kiddies to use Shodan to find people with out of date web interfaces and pop them. I tell you right now, the Immich team would be the first to say not to put their application publicly accessible.
Just don’t get into this practice, it ends in tears and is way more maintenance to stay protected than just setting up tailscale and using that.
Auth portal for VPN tunnell -> Authelia -> fail2ban -> VLAN with services only.
ELK stack monitors the LAN. (Including VLAN)
Keep that VLAN segmented. You’re good unless you’re a DOGE employee, then I’d recommend quite a bit more security.
We expose about a dozen services to the open web. Haven’t bothered with something like Authentik yet, just strong passwords.
We use a solid OPNSense Firewall config with rather fine-grained permissions to allow/forbid traffic to the respective VMs, between the VMs, between VMs and the NAS, and so on.
We also have a wireguard tunnel to home for all the services that don’t need to be available on the internet publicly. That one also allows access to the management interface of the firewall.
In OPNSense, you get quite good logging capabilities, should you suspect someone is trying to gain access, you’ll be able to read it from there.
I am also considering setting up Prometheus and Grafana for all our services, which could point out some anomalies, though that would not be the main usecase.
Lastly, I also have a server at a hoster for some stuff that is not practical to host at home. The hoster provided a very rudimentary firewall, so I’m using that to only open necessary ports, and then Fail2Ban to insta-ban IPs for a week on the first offense. Have also set it up so they get banned on Cloudflare’s side, so before another malicious request ever reaches me.
Have not had any issues, ever.
Have also set it up so they get banned on Cloudflare’s side, so before another malicious request ever reaches me.
How did you end up setting that up?
Fail2ban allows you set different actions for different infringements, as well as multiple ones. So in addition to being put in a “local” jail, the offending IP also gets added to the cloudflare rules (? Is that what its called?) via their API. It’s a premade action called “cloudflare-token-multi”
I’ve half ass thought about this but never have tried to actually self host. If you have access to all devices, why not just use your own self signed certificates to encrypt everything and require the certificate for all connections? Then there is never a way to log in or connect right? The only reason for any authentication is to make it possible to use any connection to dial into your server. So is that a bug or a feature. Maybe I’m missing something fundamental in this abstract concept that someone will tell me?
If you have access to all devices, why not just use your own self signed certificates to encrypt everything and require the certificate for all connections?
Sounds like you are describing a VPN. I was using that setup before but small stuff like immich album sharing via a link won’t work properly. Also, having to ensure a vpn is on and connected is a little to much to ask of my partner; they would turn it off and forget about it and then ask why their app wasn’t working :/
Some of these you’re already doing, but writing a complete* list. *almost garuanteed not to be complete, suggestions welcome
- Have everything behind the same reverse proxy, so that you have only one endpoint to worry about. Run it through ssllabs or similar to check your config.
- On your reverse proxy, add one or more layers of authentication if possible. Many possibilities here: If one app supports client certificates, while another has limited capabilities, you could probably tie together something where IPs are whitelisted to the ither services based on that certificate auth.
- Geoblock all countries you won’t be accessing from
- crowdsec is pretty nice, this detects/blocks threats. kinda like fail2ban but on steroids.
- if you use one of those 5$/month VPSes, with a VPN tunnel to your backend services, that adds one layer of “if it’s compromised, they’re not in your house”.
lastly consider if these things need to be publically avilable at all. I’m happy with 95% of my services only being available through Tailscale (mesh VPN, paid service with good enough free tier, open source+free alternatives available), and I’ve got tailscale on all my devices
-
check
-
check
-
check
-
I saw someone else recommend crowdsec. I’ll look into it, thanks
if you use one of those 5$/month VPSes, with a VPN tunnel to your backend services, that adds one layer of “if it’s compromised, they’re not in your house”.
I’ve heard this mentioned before but I don’t really understand how this works in practice. If the VPS was compromised, couldn’t they use the VPN to then connect to my home?
I set the VPN tunnel from the VPS to deny everything to the internal network by default, then put the services that need to be accessed on the allow list in the firewall. So the VPN endpoint from the VPS can only hit the very specific IPs/ports/protocols that were explicitly allowed. There is still the possibility of a compromise chain of VPS->service->container/VM->hypervisor->internal network access, but I feel comfortable with those layers.
You could also setup an IDS such as Snort to pick up on that exploit traffic between the services and internal VPN endpoint if extra security is necessary on top of fail2ban and log alerts on the VPS.
-
Why? Not every service is meant to be exposed to the open internet. Immich is the only one of what you listed that makes sense to have out in the open.
I’ve been playing around with the voice assistant stuff in homeassistant and it seemingly needs a public url to get all the features. I could be wrong about that though?
I put authentik in front of immich to handle authentication so that I would need need a 2FA code
Most definitely does not need a public URL for Assist in HA. Not sure where you read that.
It sounds like you need a VPN to your internal services if you’re concerned about security.
What about home assistant? Me and the Family quite often use the HA app on the go, sometimes even from other computers like at my parents or in a hotel to check on the house and the cat. I also gave my dad access to it so he can see if we’re at home and things like that.
Same with my dads HA.
Set up a VPN to get back inside your network if you’re outside it. Then you don’t need HA (or anything else for that matter) to be public.
