GEP for Super Green Farming with 100% Renewable Energy [Closed]

[jakubprogramming]

I think we should start looking at the technology side of this, as it stands even if someone were to get this certification, we don’t have a realistic solution for using that capacity without major configuration by the user.

You are right a certain amount of configuration will be required by the user. I don’t think this is a deal breaker though. Running workloads on (super)-green infrastructure is something that I believe mostly business and larger scale customers will be interested in initially. After all it’s them that can best leverage this for marketing and their public image. Also they will have the resources and IT-professionals to do their own configuration.

Once it’s possible to deploy something on super green nodes, we could work on making it more accessible for smaller customers. I could imagine actively seeking out gold certified farmers (or setting up a gold certified farm myself) and working on a new version of the current K3s based kubernetes weblet, that would allow to deploy a workload on multiple super green nodes as the worker nodes, and a central gold certified node as the master/main node easily from within the browser. That obviously would take a developing effort. But we are willing to expand and even hire people to get it done, once the basic infrastructure allows for it. If not we will do it anyway but we will have to rely on other another base infrastructure & tech stack (unfortunately).

[jakubprogramming]

before we consider the implications of trying to the implement the technology portion of this we have multiple layers that will Require entire systems to be built/possible new staffed roles.

For simple green farming that relies on existing certification systems such as Grüner Strom this shouldn’t be too hard. Farmers could just use their electricity bill to proof this.

For super green farmers yes. It’s not as easy and will probably have to be decided on a case by case basis. As I’ve discussed in a call with @weynandkuijpers to get started we may actually rely on a bit of trust. Maybe a certain reputation within the community will be needed and after some more super green farmers have joined they could potentially verify each other. I don’t expect the ThreeFold foundation to do all of this work. After all we are a decentralized movement, aren’t we?

[jakubprogramming]

This brings in another problem, deploying like this is going to expensive, if I have two nodes with unpredictable uptime, two isn’t going to be enough I’m going to need 3-4 to ensure the page gets served.

Yes if anything close to 100% uptime is desired at least 3 nodes with 90% uptime need to be included.
So if each node has a downtime of 10% (0.1) the calculation is as followed:

P(all offline) = P(server 1 offline) * P(server 2 offline) * P(server 3 offline)
= 0.1 * 0.1 * 0.1
= 0.001

Therefore, the percentage of time that all three servers are offline at the same time is 0.1%
Could be good enough for a range of workloads. If not a fourth node could be included.

And yes this could be more expensive for small workloads. But if we think about how highly distributed systems can be managed today, this becomes insignificant. What I mean is, if I use 10-20 different servers/nodes for my workload anyway, I don’t care if one, two or three of them go offline for a bit.

[jakubprogramming]

Also, I would propose this “Solution Provider + Commercial Grid Service + Green Farming Alliance” Idea. Note that it’s in point form to show the overall idea. I’ll get into details if people seem interested.

I think this is a great idea. In general, usage and deployments on super green nodes with limited availability should probably be managed in a more professional way. A commercial grid service in combination with the solution provider concept could allow for a solid system with benefits for all participants. Also Green Farming Alliance sounds like something I would be proud to be part of
Obviously I am interested in more details of how this could potentially play out. The current point for, does give a nice general overview of your proposal already though.

I specifically like the different visions you’ve laid out here and I believe there is immense potential.
Especially for QSFS on super green nodes. If enough professional developer support is given, qsfs could be made accessible to everybody in the most environmentally friendly way possible. Extremely safe & reliable storage without fear for data supervision or censorship in combination with the most sustainable and decentralised infrastructure just seems like a perfect match.

I dont think this would be a good use of qsfs, even though it is capable of rebuilding shards, we don’t want to make it do that intentionally.

Qsfs still needs highly available host, data transfer takes time and uses resources. Think of it like having a raid 10 you just plan to pull a disk out of every few days, not something you want to do.

I am no expert on in this field, but I believe comparing raid 10 to qsfs in this way is not too useful.
They do work quite differently in regards to how fault tolerance and performance are achieved. For example raid 10 relies on mirroring while qsfs relies on mathematical algorithms.

During the community call last week @scott has assured me in the group chat, that qsfs is well suited to take advantage of storage capacity that is online intermittently. I hope @scott doesn’t mind me posting a screenshot of his statement here (I often keep those kinds of things, usually for personal reference).

[jakubprogramming]

I’ve repeatedly seen people on the telegram farmers chat and the forum being worried or confused about what is considered green and what is not green. I kindly invite you once again to join us here in to forum to discuss this. After all my recommendations are only recommendations and are very much up for debate. If you think the way you’ve setup your nodes should be eligible for green or super green farming please let us know here and we can talk about it. In this process please also try to elaborate how running nodes the way you do has a planet positive impact. Personally I’ve put my focus on running nodes using sustainable wind & solar power with intermittent availability. Others may have different ideas for green farming and are welcome to share their ideas.

[scott]

During the community call last week @scott has assured me in the group chat, that qsfs is well suited to take advantage of storage capacity that is online intermittently. I hope @scott doesn’t mind me posting a screenshot of his statement here

No worries, and I do stand by the statement, as my untested opinion

I dont think this would be a good use of qsfs, even though it is capable of rebuilding shards, we don’t want to make it do that intentionally.

There’s no need to rebuild shards until some threshold of tolerance has been crossed for data availability. In the case of wind/solar powered nodes, you expect they will have some amount of downtime and thus that threshold is higher. It’s also possible to make the reconstruction algorithm smarter for this use case. For example, check if other nodes at the same site have powered on, then conclude that it’s a node failure rather than power outage and prioritize reconstruction.

It’s worth noting here too that “cold” archival data storage can take 48 hours or longer to retrieve data. So there’s definitely a market for low cost and low availability storage.

Instead I once more suggest to lower the uptime requirements for farmers using physically green energy.

This is the part of the proposal that I’m most comfortable supporting. Personally, I think that any boost to rewards should come through a utilization booster that is available to everyone. If there is indeed demand for renewable powered IT, then it will pay off for the farmers.

Otherwise, we get mired in a lot of very difficult questions about what is good enough to be “green”. Adding a booster just for electricity type opens the door for farmers to go buy some renewable energy credits/offsets and show up with their receipt asking for the extra TFT. This of course isn’t the intention, and I don’t think the team or the DAO should get caught up in judging where different arrangements fall on the spectrum.

On the other hand, a farming class with 90% uptime requirement for farmers who can show that their nodes are connected directly to a renewable energy source is much simpler. There’s still a small incentive to try to abuse this arrangement, but much less than with boosted rewards.

[Mik]

Hey guys.

Sounds like some progress is being made here!

So as a synthesis,

• The idea of building a “Solution Provider + Commercial Grid Service + Green Farming Alliance” looks like it would work, for workloads such as QSFS and https load balancers or serving static files. (big thanks to @scott for confirming this)

• There seem to be a consensus on the suggestion of “lowering the uptime requirements for farmers using physically green energy.” Instead of aiming for “booster rewards” for green farming.

---

One thing, @jakubprogramming. I am not sure about this calculation:

P(all offline) = P(server 1 offline) * P(server 2 offline) * P(server 3 offline)
= 0.1 * 0.1 * 0.1
= 0.001

This doesn’t take into account the distribution of the farms. For a very simple counter-example, if the 3 farms have their downtime at the same time, it will still be a 10% downtime.

Without going into details, the equation you wrote is more in line with Bernoulli trials.

To be sure we get the highest uptime possible, one would need to select farms that have different moments of downtime. A simple distribution could be having solar farms distributed across the globe. This could lead to higher uptime than, say, solar farms all in France.

---

One question about the type of green farming: Do we consider it to be green farming if one’s farm is using 100% hydroelectricity?

I would say yes. I’d be curious to see what others think.

That being said, hydroelectricity farms can easily have 100% uptime (mine does ).
Still farmers in my situation wouldn’t get anything from the program if we don’t have boosters, but only lower uptime requirement for green farming, since uptime isn’t a problem with hydroelectricity (generally speaking, of course). Not that it’s a problem. I am just observing some facts.

---

Next steps could be to redact a proposal for lowering downtime for green farming.
And to explore the commercial grid service as proposed above.

[ParkerS]

Indeed if we’re talking about data someone is okay with it taking 48 hours to be able to retrieve it I guess that’s fine.

[scott]

This doesn’t take into account the distribution of the farms. For a very simple counter-example, if the 3 farms have their downtime at the same time, it will still be a 10% downtime.

The calculation correctly shows the probability that three nodes with 90% uptime will all be offline at the same time, assuming their downtimes occur randomly. In practice, there will be some correlation in energy availability for any nodes relying on solar power which are in adjacent time zones, but to the extent they also rely on wind and have significant geographical separation, this correlation would be reduced. Solar only farms won’t hit 90% uptime.

One question about the type of green farming: Do we consider it to be green farming if one’s farm is using 100% hydroelectricity?

As a majority, but not exclusive, consumer of hydroelectric power, I’ve pondered this quite a bit. Dams impact the local environment in ways that are harmful, including greenhouse gas emissions from decaying biomatter in the flooded area upstream (could actually be worse than burning fossil fuels for the same energy in some cases).

Every method of harvesting energy has impacts that are greater than what’s observed at the site and time of production. The total impact of any choice around using energy becomes a very complicated question. That’s why I don’t think we should take the role of judging the relative impact of various farms. Rather, I think we can take the more neutral stance of accommodating farming done with intermittent non polluting energy sources, and leave it to consumers to choose farms that align with their values.

[Mik]

“assuming their downtimes occur randomly”

Yes. This is exactly my point. The 0.001 comes from a random distribution.
I think we are saying the same thing here.

---

Also, 100% agree with this: “accommodating farming done with intermittent non polluting energy sources”

This is way more feasible.

Just a side note, dams in colder climate will produce less decaying biological matter in the flooded area, compared to dams in a tropical region. Also, the decaying biological matter will happen mostly in the first years of the dam. After several decades of construction, like here in Canada, the decaying matter will be minimal.

[jakubprogramming]

Instead I once more suggest to lower the uptime requirements for farmers using physically green energy.

This is the part of the proposal that I’m most comfortable supporting. Personally, I think that any boost to rewards should come through a utilization booster that is available to everyone. If there is indeed demand for renewable powered IT, then it will pay off for the farmers.

Otherwise, we get mired in a lot of very difficult questions about what is good enough to be “green”. Adding a booster just for electricity type opens the door for farmers to go buy some renewable energy credits/offsets and show up with their receipt asking for the extra TFT. This of course isn’t the intention, and I don’t think the team or the DAO should get caught up in judging where different arrangements fall on the spectrum.

While having higher rewards for green farmers would be a nice bonus, especially as an incentive for others, this has never been my primary goal. When I joined this community about a year ago and made my first post on this topic, higher rewards were not even something I had in mind. So personally, I am completely fine with receiving default rewards.

From my viewpoint the primary goal has always been to lower uptime requirements to such a degree, that farmers relying on 100% renewable energy with intermittent supply are enabled to participate in this project in the most sustainable way. For me this also means using as few backup batteries as possible and actively researching and working on solutions that enable workload execution to adapt to the nature of fluctuating green energy supply. All of this works better, if more people from different parts of the world participate. That’s why higher rewards could be beneficial. But they certainly are not the main focus here.

On the other hand, a farming class with 90% uptime requirement for farmers who can show that their nodes are connected directly to a renewable energy source is much simpler. There’s still a small incentive to try to abuse this arrangement, but much less than with boosted rewards.

Also, 100% agree with this: “accommodating farming done with intermittent non polluting energy sources”

This is way more feasible.

I also agree that without boosted rewards the incentive to try and abuse the system is much lower. The only way people could benefit from claiming that their nodes are powered by phsically green energy is by saving a few bucks on electricity if they turn off their nodes the last 2-3 days of each month or so.
I don’t think many people would bother going through all this trouble. Especially if they have to some degree (fake)-proof beforehand that their power really is renewable on a physical level.

[jakubprogramming]

The calculation correctly shows the probability that three nodes with 90% uptime will all be offline at the same time, assuming their downtimes occur randomly. In practice, there will be some correlation in energy availability for any nodes relying on solar power which are in adjacent time zones, but to the extent they also rely on wind and have significant geographical separation, this correlation would be reduced. Solar only farms won’t hit 90% uptime.

Yes. This is exactly my point. The 0.001 comes from a random distribution.
I think we are saying the same thing here.

Yes of course this is only true when assuming a random distribution. In the real world, depending on the type of power and the geographical separation, numbers will be a bit different. Though not necessarily worse. We ourselves have sustainable power available in Germany, France, Poland and South Africa. I think that could be a good starting point. The more people participate in this around the world the better it will work. Once a certain number of super green nodes have joined the grid, deployments could be done is such a way that they are distributed across nodes with different usual downtime schedules.

[jakubprogramming]

One question about the type of green farming: Do we consider it to be green farming if one’s farm is using 100% hydroelectricity?

I would say yes. I’d be curious to see what others think.

That being said, hydroelectricity farms can easily have 100% uptime (mine does ).
Still farmers in my situation wouldn’t get anything from the program if we don’t have boosters, but only lower uptime requirement for green farming, since uptime isn’t a problem with hydroelectricity (generally speaking, of course). Not that it’s a problem. I am just observing some facts.

If you can reach 100% uptime using exclusively physical hydro energy with your nodes that’s super sweet. Since the consensus seems to shift to not giving additional rewards to green farmers there would indeed not be any benefits. At least not in terms of higher monthly payouts. Those nodes could still be considered green and work together with other super green nodes to make IT-Systems and deployments more available. This way those nodes could potentially benefit from higher utilization rewards.

[jakubprogramming]

No worries, and I do stand by the statement, as my untested opinion

ParkerS:

I dont think this would be a good use of qsfs, even though it is capable of rebuilding shards, we don’t want to make it do that intentionally.

There’s no need to rebuild shards until some threshold of tolerance has been crossed for data availability. In the case of wind/solar powered nodes, you expect they will have some amount of downtime and thus that threshold is higher. It’s also possible to make the reconstruction algorithm smarter for this use case. For example, check if other nodes at the same site have powered on, then conclude that it’s a node failure rather than power outage and prioritize reconstruction.

Seems we should make the move from an untested opinion to a proof of concept then. While we are still figuring out the exact details on how to include nodes with intermittent power into the grid I’d be happy to dive deeper into this. It will probably be quite a challenge initially. Maybe I could get started by figuring out how to deploy a basic distributed qsfs solution to some of my own DIY nodes first. I suppose it would be best if connect them to the dev-grid for this purpose? Going from there I could simulate power outages and see how it all plays out.

Once the PoC is estabished further optimizations such as the ones @scott has suggested could be implemented.

⠀
⠀
By the way: ChatGPT also agrees that storage systems relying on erasure coding such as qsfs are well suited for these kinds of circumstances.

[scott]

Maybe I could get started by figuring out how to deploy a basic distributed qsfs solution to some of my own DIY nodes first. I suppose it would be best if connect them to the dev-grid for this purpose? Going from there I could simulate power outages and see how it all plays out.

Devnet would allow you to test without cost, but getting some Zdb HDD storage to test is already rather inexpensive. For testing, I think it’s best to invoke QSFS manually, either in a VM or on your local machine. That way you have direct control over the configuration and can access internals that would normally be hidden when you deploy on the Grid (removing the cache to force reconstructing data from backends, for example).

I ran some tests like this when I was first getting to know QSFS—distributing some data over a set of backend nodes, simulating a failure by blocking network access to some subset of those nodes, then reconstructing the data from the remaining nodes. Short answer is that that this works as expected.

Happy to support your efforts in testing. Would be nice to document the process too in case others have the same interest.

[Mik]

Would be nice to document the process too in case others have the same interest.

Will your next video be on QSFS ?

[jakubprogramming]

For testing, I think it’s best to invoke QSFS manually, either in a VM or on your local machine.

Yes that does seem like a good way to get started. I assume the quantum-storage repository on github contains everything I need to dive into it. Can you point me in the right direction? Maybe you still even have some config files from your own testing that you could share with me. VMs on Apple silicon don’t work too well… lets see how far I can get without having to dust off my rather old windows PC.

Happy to support your efforts in testing. Would be nice to document the process too in case others have the same interest.

I’ve been working on a video about Kubernetes on the Grid. First time I’ve done something like that and it still feels a bit weird. Hope to publish soon. I am considering doing the same for qsfs if I can get something done that is worth sharing. Or maybe a written guide as an alternative.

[jakubprogramming]

Summary and Interim conclusion

Based on the fruitful discussion in the last few days I think we have somewhat reached a consensus. In the current project stage additional rewards for green farming may be too prone to abuse and deciding on the exact definitions of what may or may not be considered green on a global scale is presently beyond the capabilities of the DAO. Instead the main focus should be on something that is technically and bureaucratically achievable at the current stage of this project. Luckily the first and also most important proposal of my GEP seems to be suited for implementation.

Uptime Requirements
Since renewable energy is not available 100% of the time, lower uptime requirements are needed for 100% green farming. Based on my analysis the following uptimes could be realistically achieved:

• 90% Uptime using a combination of wind & solar power, or a combination of wind & backup batteries

I think we can all agree with @scott that green farmers still need to show that their nodes are actually indeed powered by phyiscally sustainable sources in order to qualify for lower uptime requirements.

On the other hand, a farming class with 90% uptime requirement for farmers who can show that their nodes are connected directly to a renewable energy source is much simpler. There’s still a small incentive to try to abuse this arrangement, but much less than with boosted rewards.

How exactly a green farmer can show that his nodes are directly connected to a renewable energy source will be different on a case by case basis. For now let’s just say that if it’s supported by sufficient evidence and looks overall convincing it will be fine.

As I’ve already exemplified, the incentive to cheat this system should indeed be very low.

The only way people could benefit from claiming that their nodes are powered by phsically green energy is by saving a few bucks on electricity if they turn off their nodes the last 2-3 days of each month or so.
I don’t think many people would bother going through all this trouble. Especially if they have to some degree (fake)-proof beforehand that their power really is renewable on a physical level.

My personal goal is to actively work on solutions that can be deployed on single or multiple green nodes with lower uptime, without significant disadvantages. Starting with file storage using the Quantum Safe File System. @Mik has laid out a great vision for this that I hope we can further pursue all together.

Furthermore, I hope one day we will be able to include some of the more advanced features of my original proposal into the grid. Such as making better use of excess energy and the whole concept around smart contracts for green IT.

Next steps

• Implement a new type of farming class with lower uptime requirements for farmers that are physically connected to renewable power sources.
• Optional: Enhance the Farmerbot with its WOL features to power off/on super green nodes based on power availability.

@weynandkuijpers Since this whole idea really took form by us two talking, can you help get this thing rolling? Maybe a second, much slimmer GEP is needed? Let’s finally get it done!

[Mik]

As I’ve already exemplified, the incentive to cheat this system should indeed be very low.

Indeed, especially considering the farmerbot coming soon.