Background: 15 years of experience in software and apparently spoiled because it was already set up correctly.
Been practicing doing my own servers, published a test site and 24 hours later, root was compromised.
Rolled back to the backup before I made it public and now I have a security checklist.
Don’t use passwords for ssh. Use keys and disable password authentication.
More importantly, don’t open up SSH to public access. Use a VPN connection to the server. This is really easy to do with Netbird, Tailscale, etc. You should only ever be able to connect to SSH privately, never over the public net.
It’s perfectly safe to run SSH on port 22 towards the open Internet with public key authentication only.
https://nvd.nist.gov/vuln/detail/cve-2024-6409 RCE as root without authentication via Open SSH. If they’ve got a connection, that’s more than nothing and sometimes it’s enough.
That attack vector is exactly the same towards a VPN.
Are you talking a VPN running on the same box as the service? UDP VPN would help as another mentioned, but doesn’t really add isolation.
If your vpn box is standalone, then getting root is bad but just step one. They have to own the VPN to be able to even do more recon then try SSH.
Defense in depth. They didn’t immediately get server root and application access in one step. Now they have to connect to a patched, cert only, etc SSH server. Just looking for it could trip into some honeypot. They had to find the VPN host as well which wasn’t the same as the box they were targeting. That would shut down 99% of the automated/script kiddie shit finding the main service then scanning that IP.
You can’t argue that one step to own the system is more secure than two separate pieces of updated software on separate boxes.
Why is your VPN jump box better than an SSH jump box?
A VPN like Wireguard can run over UDP on a random port which is nearly impossible to discover for an attacker. Unlike sshd, it won’t even show up in a portscan.
This was a specific design goal of Wireguard by the way (see “5.1 Silence is a virtue” here https://www.wireguard.com/papers/wireguard.pdf)
It also acts as a catch-all for all your services, so instead of worrying about the security of all the different sshds or other services you may have exposed, you just have to keep your vpn up to date.
Yeah I don’t do security via obscurity :D I agree you need to keep your Internet facing services up to date.
(No need to educate me on Wireguard, I use it. My day job is slightly relevant to the discussion)
Another one who misunderstands that phrase… Yes, obscurity shouldn’t be your only line of defense, but limiting discoverability of your systems should be an integral part of your security strategy.
There’s no difference to the work I need to do to secure an open SSHd vs an open WireGuard server. None.
Yes I harden, and penetrate, systems for a living. If your systems need remote access there is no standard (neither in fintech or military) that classifies SSHd as being “worse” than a VPN.
Tailscale? Netbird? I have been using hamachi like a fucking neanderthal. I love this posts, I learn so much
what’s netbird
https://netbird.io/. Wireguard based software defined networking, very similar to Tailscale.