0-OS support for IPV4 only nodes - Yggdrasil integration

[zaibon]

0-OS v0.3.5 has been released and with it a long awaited feature, namely: the support for farms that are not IPv6 enabled yet.

As the grid has been conceived with a mindset to start with IPv6 first, so that it’s not an afterthought that would be needed to be addressed in the near future anyway, we need some form of IPv6 available even if the hosting network doesn’t provide IPv6.

The problem

0-OS v2 has been designed around the fact that a node needs to have an IPv6 public address. This choice was made to solve two problems: the limited availability of public IPv4 and the need to implement some complex IP management between nodes in one and the same farm.

Because each node in the grid must be able to connect to any other node, having public IPv6 address everywhere made it very easy, solving all these pesky NAT and unreachable node problems.

It becomes more of a problem when you start adding nodes that are fully hidden, like nodes running in people’s homes behind a NAT router (virtually all homes have 192.168.1.0/24 that then go to the Internet over IPv4 through a NAT router).

These nodes will never be reachable from outside and then would only be able to provide compute capacity for applications that don’t expose any service. In other words these nodes could only be used for containers and k8s clusters. 0-DB namespaces (our storage building block) wouldn’t even be possible on these nodes, as they would not be reachable.

The solution

To allow all types of nodes to provide both compute and storage capacity, these nodes need to be reachable by any other node in the network regardless of their network setup.

To achieve this, 0-OS has integrated Yggdrasil. Yggdrasil is an implementation of a fully end-to-end encrypted IPv6 network where nodes are part of a globally-agreed spanning tree.

It allows traffic to be forwarded between all nodes of the network.

That solves the “hidden node” problem. By running Yggdrasil on all the 0-OS nodes, they become part of the yggdrasil network too.

This in turn allows any node to expose services on top of an Yggdrasil IP. Since all 0-OS node are part of the yggdrasil network, there is no more “hidden nodes”.

You can find more information about Yggdrasil on their website: https://yggdrasil-network.github.io/

Roadmap

The yggdrasil integration into 0-OS will be executed in multiple phases. By going step by step, it allows us to monitor how the network behaves and scales before pushing for more features.

Here is a brief roadmap of what we have already done and what we want to do next.

label:

• : done

• : planned

Roadmap:

• Allow IPv4 only nodes to boot be part of the grid

• Expose 0-DB server on yggdrasil IP

• Allow containers to have an yggdrasil IP

• Allow IPv4 only nodes to access IPv6 network

[weynandkuijpers]

Clear as I have been bound to a non-ipv6 connection, I am going to install a node.

Although I have always seen a role for the non IPv6 nodes (hidden) as processing nodes with an added benefit of being more secure “by being obscure”

[scott]

Thanks for posting this update! I think I’m finally starting to understand some things about node networking. It hadn’t occurred to me that one outcome of the larger IPv6 address space is that every device has a public address. My next thought was that this seems somehow more risky, but then again NAT isn’t a replacement for firewalls. That led me to reconsider the relationship between my node, my firewall, and the grid.

Even after reading the network configuration documents, it wasn’t clear to me how limited my node would be behind a firewall. Today is actually the first time I tried to access solutions running on my home node from somewhere else, and it turns out they’re unreachable. I took for granted that even though I was addressing my node by its public IPv6 address, the traffic was still being routed locally and that’s the only reason I could actually reach my solutions.

This all got me wondering if there’s any plan to check whether nodes are actually reachable and include this in say the explorer view (this, I guess, is one reason for certification). Corollary to that is wondering if this is a barrier to nodes being designed for plug and play, as some router config will still be needed to allow inbound connections.

Yggdrasil as a way to bridge with IPv4 is brilliant, and very cool tech in it’s own right (maybe someday it won’t be possible to break the internet with bad router configuration ).

Talk to you soon, @zaibon.

[zaibon]

Yep these are all very valid observations.

Let me get back on why even though your home router gives your node a public IPv6 address it might be that this address is still unreachable from the public internet.
Most of these home router actually have a firewall enabled that prevent traffic from going inside your LAN. You might need to tweak the configuration of the router if you want to expose it completely.

Now depending on what service you want to provide on your node, you don’t always have to do this. The private overlay network that interconnects all the nodes used by your workloads actually only needs one publicly reachable node to allow a connection from any of the node in your network. The reason is the networks configuration generated by the SDK tries to create a full mesh. So all nodes connects to all other nodes. But this is not always achievable because of some node behind NAT. In this case, the networks uses a start topology where the public nodes act as a relay for the hidden nodes.

. Today is actually the first time I tried to access solutions running on my home node from somewhere else, and it turns out they’re unreachable

What solution did you try to access ? And using what address ?

Yggdrasil as a way to bridge with IPv4 is brilliant, and very cool tech in it’s own right

Yeah Yggsrasil is a very nice project. There are still plenty of feature we could implement on top.

Also our network documentation still need to be updated to reflect the latest status since Yggdrasil integration. This is in progress (@delandtj wink wink )

[scott]

Ah, so it is more complicated than I thought My first impression was that the overlay network would effectively bypass my router’s firewall, which it sounds like it does in some cases. I was considering moving the node into DMZ but thought that was more of an added bonus. So it surprised me when I couldn’t get through, and maybe the problem is something else.

I’m trying to ssh into an Ubuntu container with reservation ID 10719. Using the same wireguard configuration that worked before with:
Endpoint = [2601:1c0:c803:360:1602:ecff:fe49:1b51]:2858

[zaibon]

My first impression was that the overlay network would effectively bypass my router’s firewall, which it sounds like it does in some cases.

It doesn’t “by-pass” it. There are not NAT traversal or hole punching involved. It’s simply that when using a public node as relay, the connection is initiated from your hidden node the public node, so NAT is not in the way.

Endpoint = [2601:1c0:c803:360:1602:ecff:fe49:1b51]:2858

I guess this is an extract of your wireguard configuration. If this is the IP of your node behind your home router, it is very possible that it is not actually reachable from outside of your LAN. And this when you are on the move you might not be able to connect to it and access your private network.

[scott]

I read up on wireguard a bit more and see now how the “NAT and Firewall Traversal Persistence” works. My meaning by “bypass” was only that incoming connections are enabled from within the overlay, versus being disabled altogether. That this is made possible from an outbound connection opened by the private peer to a public peer makes sense.

What I think I’m understanding now is that I could deploy my networks in the future to include another node that’s publicly reachable and thus be able to access what’s running on my firewalled node. By running through the list of nodes in Grid explorer, I’m able to find at least one that I can ping, which I’m guessing would do the trick.

[zaibon]

What I think I’m understanding now is that I could deploy my networks in the future to include another node that’s publicly reachable and thus be able to access what’s running on my firewalled node. By running through the list of nodes in Grid explorer, I’m able to find at least one that I can ping, which I’m guessing would do the trick.

That’s absolutely right

[karoline]

Thank you for posting that update!!!
SUPER EXCITING!