Resources For Setting up Redundant High Availability Clusters for the Fediverse

Resources For Setting up Redundant High Availbility Clusters for the Fediverse

      One of the biggest challenges to deploying on a decentralized grid is ensuring you provide the level of service to your users that is expected of any major social media site, simply put, storing other peoples information is a huge responsibility and as a community we need to work to provide solutions that take this into account. The Threefold grid provides a unique opportunity to deployers to place their workloads in thousands of locations, utilizing thousands of different network connections. With the availability of the planetary network and z-net this capacity extends the ability of the host to utilize all of those nodes, whether they have public ip space or not. I do not intend this to be a tutorial, yet, but a opening of discussion on how we can use the tools that are open source and available today to provide an acceptable level of service to users. this will also be a place to keep track of relevant documentation for anyone to be able to learn the skills necessary to participate. This is a highly technical project and the discussion will likely be very confusing for the uninitiated, I welcome PMs here and on telegram for any offline clarifications needed.

Introduction to High Availability

      I wanted to get this post, with this information out and available now, I plan to produce and post a more thorough idea of how all of this can come together to create a user experience that matches that of any social media provider in the coming days.

3 Likes

This is a post by the host of a large mastadon instance talking about how they have setup their infrastructure

https://medium.com/@kris-nova/hachyderm-infrastructure-74f518bc7472

Good starting point, and lots to read. What I would like to add is that I have been in the IT industry for more than 2 decades, from building the first datacenters in Europe ('90’s) when “the internet” existed predominantly in the US, to today. And in all my humble experience, architecting efficient, reliable and scalable solutions has not changed. Things moved from physical to virtual, and a large number of additional CI/CD frameworks have been added but the basic building block for a good architecture have node changed. It’s like Lego, you have access to a lot of pieces of different size, shape and color but it is the architect that creates a beautiful, functional and solid house (bridge, car, plane etc etc.).

The building blocks that ThreeFold present are produced and deployed in a different way, but are also components that need to be architected into a reliable and strong online solution.

@ParkerS thank you for starting this, I’d love to see where this discussion is going (and I will offer the simple learning I moving from physical to virtual over the last 2 decades).

That is exactly the idea behind this thread, how do we build real infrastructure on top of the grid, it started with just a webmail server and is now progressing to having fully grid hosted name servers, and creating solution for those looking to use the grid to provide managed web hosting as a service.