• 3 Posts
  • 42 Comments
Joined 1 year ago
cake
Cake day: July 5th, 2023

help-circle
  • That advertisement would be interpreted as Node C’s advertisement.

    The plan is to treat public keys as node’s identity and trust mechanism similar to OpenPGP (e.g. include any node key signed by a master key as a cluster member)

    Right now, none of the encryption part is done and it is not a priority right now. I need to first implement transitive node detection, actually forward packets between nodes, some way to store and manage routes, and then trust and encryption mechanisms before I’d dare to test this stuff on a real network.







  • Just did some basic testing on broadcast addresses using socat, broadcast is not working at all with /32 addresses. With /24 addresses, broadcast only reaches nodes that share a subnet. Nodes that don’t share the subnet aren’t reachable by broadcast even when they’re reachable via unicast.

    Edit1: Did more testing, it seems like broadcast traffic ignores routing tables.

    On 192.168.0.2, I am running socat -u udp-recv:8000,reuseaddr - to print UDP messages.

    Case 1: add 192.168.0.1/24

    # ip addr add 192.168.0.1/24 dev eth0
    # # Testing unicast
    # socat - udp-sendto:192.168.0.2:8000 <<< "Message"
    # # Worked
    # socat - udp-sendto:192.168.0.255:8000,broadcast <<< "Message"
    # # Worked
    

    Case 2: Same as above but delete 192.168.0.0/24 route

    # ip addr add 192.168.0.1/24 dev eth0
    # ip route del 192.168.0.0/24 dev eth0
    # # Testing unicast
    # socat - udp-sendto:192.168.0.2:8000 <<< "Message"
    2024/02/13 22:00:23 socat[90844] E sendto(5, 0x5d3cdaa2b000, 8, 0, AF=2 192.168.0.2:8000, 16): Network is unreachable
    # # Testing broadcast
    # socat - udp-sendto:192.168.0.255:8000,broadcast <<< "Message"
    # # Worked
    






  • Thank you… I had to learn kubernetes for work and it was around 2 weeks of time investment and then I figured out I could use it to fix my docker-compose pains at home.

    If you run a lot of services, I can attest that kubernetes is definitely not overkill, it is a good tool for managing complexity. I have 8 services on a single-node kubernetes and I like how I can manage configuration for each service independent of each other and also the underlying infrastructure.




  • If one service needs to connect to another service then I have to add a shared network between them. In that case, the services essentially shared a common namespace regarding DNS. DNS resolution would routinely leak from one service to another and cause outages, e.g if I connect Gitlab container and Redmine container with OpenLDAP container then sometimes Redmine’s nginx container would access Gitlab container instead of Redmine container and Gitlab container would access Redmine’s DB instead of its own DB.

    I maintained some workarounds, like starting Gitlab after starting Redmine would work fine but starting them other way round would have this issue. But switching to Kubernetes and replacing the cross-service connections with network policies solved the issue for me.


  • As someone who is operating kubernetes for 2 years in my home server, using containers is much more maintainable compared to installing everything directly on the server.

    I tried using docker-compose first to manage my services. It works well for 2-3 services, but as the number of services grew they started to interfere with each other, at that point I switched to kubernetes.