What is a Threefold Gateway and Why Do We Need Them

[ParkerS]

What is a Threefold Gateway and Why Do We Need Them

      There has been lots of discussion lately about Threefold Gateways, what they are, how they work, and why they are important to the future of the grid. as a general concept any node that is capable of receiving inbound traffic can function as a gateway for the grid in some capacity and the infrastructure provided by this is a major part of what makes the grid capable of creating a “virtual data center”. which is fancy way of saying that we have the capability to make nodes that are geographically separated, and without public Ip addresses, behave as if they are on a local LAN network with each other. this is conceptually similar to a private VPN network. Where the grid differs from traditional technologies is its availability for endpoints within that network to be controlled autonomously.

      A node becomes a “Gateway” when a Farmer adds a public ip address to the node itself on the dashboard, in doing this, that ip is then handed over to the base operating system of the node itself, in order for that I.P. address to be used in the overall functions of the grid,

• This differs from when an I.P that has been added to a farm is deployed with a workload in order for that workload to be accessible on the overall internet.

This IP address is then used in providing Gateways for other nodes on the network to be able to receive inbound traffic in a private and low latency way. for instance a user can deploy a peer tube instance and have access to the website on a node/farm that does not have any public ipv4 addresses. this accomplished by the gateway nodes working in a fashion similar to a load balancer where a web server separates traffic based on the domain and forwards it to and from a node over the internal networks (Z-net and Planetary).

A gateway node is also capable of participating in the overall mesh network, anytime a node comes online it has to join the network so that it is capable of being place into a private network with other nodes. This is accomplished by the node coming online sending an outgoing request to a gateway node to establish a “tunnel” that node then connects to the gateway, and holds open a connection so that at any time the gateway node can forward traffic to it. This relationship is similar to your home router and your computer, the gateway simply receives traffic and forwards it to the destination, then the process is reversed, to the user, it appears they connecting directly to the server with no public address.

The concept of mesh networks has been around a long time, and while their popularity is growing, they have always suffered a shared flaw when expanded to a community level, they rely on each node within the system to be configured correctly, a misconfigured link in the chain can cause traffic to be diverted, intercepted or simply lost resulting in performance problems and security risks. Threefold Gateways provide the opportunity to solve this flaw, each gateway node is controlled by the blockchain allowing for configurations to be synced, and actively changes as performance needs necessitates. More importantly they provide a secure way for anyone to participate in the mesh, without exposing the contents of the traffic to the participants.

Each gateway added to the network allows the grid to route traffic through its network, this allows the grid to utilize the transport infrastructure of that nodes i.s.p, as more and more gateways come online, we are able to choose from a wider selection of “routes” that traffic can be carried on in service of the grid.

[weynandkuijpers]

@ParkerS thank you for this explanation. Love to take this one step further and describe/showcase in a few steps how / where to find gateways and how to make on work in (Europe) and a 3node (private network, behind a NAT) somewhere else?

I have personally never created / deployed a web gateway / nodes, so love to learn that as well.

[ParkerS]

I receieve some questions by PM on this thread and wanted to answer them here as I think this will be of benefit to the discussion

1. I’d like to try and get direct admin to work on at least one node. Also maybe Mastodon on another and Nextcloud somewhere. Do i understand it correctly that i would need ip’s to be assigned to the farm rather than the node itself?

   • Correct, if you want to expose the services of a VM directly to the internet, you would add that IPv4 to the farm section something like directadmin would require this, because your going to be routing multiple domains to a single ip address, though mastadon and peertube could be deployed through a gateway node with a subdomain from a gateway node, because ultimately they would only need to rely on one domain for access

     • When you do this, that I.P is placed on a list of available addresses that can be deployed when a user wants their VM to have its own public addresses. on the playground this would functionally be accomplished by activating the ipv4 slider.

2. can you have both? Both it’s own ip on a node and still also use it for deployments through the farms ip?

   • Yes you can have both, but a single ip address can only provide one of the services. the IP addresses you add to the farm serve a single vm workload, The I.P address you add to the node itself, Serves the grid and other nodes,

   • The way this works is an ip address added to the farm belongs entirely to the vm that deploys it, that means all traffic will be given to that vm.

   • An ip address added to the node, allows the node to become a Gateway, this allows it to provide a path for traffic from the internet to reach nodes inside the grid that dont have their own public address,

     • We will use the functional example of a mastodon server to describe how both types of I.P addresses can be used to provide access to a workload.

     • If a user were to deploy a mastodon server with public IPV4 address from the farm section, all traffic to that ip address belongs to their vm and their server is exposed directly to the internet.

     • A user could deploy the same workload with no Public ipv4 address by routing traffic over one of the gateway nodes, this creates a web server on the gateway node and forwards all traffic related to deployed domain name on to another node on the grid, through the internal networks.

     • both of these deployments mastodon servers would be accessible by use of standard web domain, but the one that is deployed using a gateway node, traffic has to go through the gateway node that has a public ip and be routed to the node hosting the back end.

3. If i would use one or more (would ‘more’ still make sense if they’re at the same location?) nodes as a gateway, how would a new home 3node know where to find it to create a tunnel? Does it look for the nearest one? How would it know?

   • When a workload is deployed utilizing a web gateway it deploys a workload on the node hosting the deployed service and the gateways node, having multiple per farm provides additional capacity, and fail over to the workloads relying on your gateway to access the workloads running on other nodes.

   • Users are able to deploy their workload through any of the gateways when setting up the deployment, On the playground you can see this by choosing an applet like mattermost or peertube, you will select both a node to host the actual workload, and a node to host the gateway that makes it accessible by domain name.

   • as it stands when a new node is brought online it joins the grids networks because zos is preconfigured with a peer list where that new node sends its packets to request the tunnels be established.

Some Reference Pictures

This is the “Farm Section” where you add I.P addresses that will ultimately serve individual VMS in providing a public address used solely by the deployed VM

This is the section where you add I.P addresses that are used to route traffic to other nodes and allow workloads without public addresses to be accessible by domain name from the traditional internet.

This is an example of an applet that users can choose a gateway node to provide domain access to their workload.

[ParkerS]

How-To use a Gateway to Deploy Your Workload On Any Node.

      Many workloads that live on the grid require the ability for them to be accessible to devices outside of the grid, One option you have for accomplishing this is using a Gateway Node to route internet traffic to your workload through use of domain name. This setup allows you to expose workloads running on any node on the grid to the internet, without the additional charge and resource requirement of a Dedicated IPv4 Address.

      For this example, We will deploy a Peertube Instance, We want to host this workload on a Farm that does not have ipv4 addresses available, but we still want it to be accessible through a users standard web browser, without additional software or configuration.

• For this example we will deploy our workload on the playground by selecting the Peertube applet.
  

  
  
  In the applet you will see configuration options for both Gateway Node and Node selection.
  • Gateway Node: This will be the node that works as a gateway to your workload, all traffic from the internet to your workload will be proxied through this node, you will want to consider this nodes location in relation to you, and the node that you are hosting your workload on. currently only the domain of the gateway node is displayed. This needs to be updated to display the node id so it is easy to locate the node, currently you can identify the location of a gateway, by matching the domain found in the playground, with the domain listed under public configuration on a gateway nodes page in explorer, this definitely needs improvement.

  Currently it is advantageous to determine what gateway you want to use by accessing the explorer and locating your closest gateway, noting the domain and then selecting the domain when deploying.

  • Node Selection: This is where you will choose the node that will host the contents of your workload, all of your files and services will live on this node this creates the opportunity to decentralize your workloads storage, from where your services will be exposed to the internet.

Once you have selected your nodes and set the appropriate settings, you will click deploy and the grid will create a virtual network between the Gateway Node and the Workload Node and route all traffic for the subdomain of the workload to the internal ip address of the workload node.

In the above example we would deploy our workload on test net node 198 without a public address, but we would connect to it from the internet, through test net node 12, the gateway hosting Gent02.grid.test.tf

in this example the result is being able to navigate to our PeerTube site using

https://pt667peertube1.gent02.test.grid.tf

despite their being no public ip addresses associated with the VM running the workload.

This usage has a few limitations,

• The domain will be a sub-domain of the Gateway node

• Only traffic using that sub-domain will be routed to the node, the workload itself does not have a public I.P. Address

[teisie]

Why is this not yet possbile on VM’s?

[ParkerS]

It is possible with a standard vm, if you deploy using Terraform.

[FLnelson]

Any limit to the number of nodes that can be assigned to a gateway?

[ParkerS]

eventually i imagine the workloads would fill the node to capacity, but i dont believe there is set limit otherwise.

[FLnelson]

If a gateway goes down, are all those workloads lost or can they be reassigned to a new gateway?

[ParkerS]

If a gateway goes Down that workload would not be accessible by that subdomain but it’s public/ygg address would remain up