As a note, this proposal was closed and executed at Block 3508279. Next steps to follow!
[Approved & Executed] The First Vote of the ThreeFold TF Chain (L1) DAO: Gold Certified Farming (CLOSED)
2 Likes
I’ve been discussing this with the team for the last few days, and secure boot will indeed be a requirement for Gold Certified nodes. Furthermore, we’ll begin by offering gold certification to farmers with nodes purchased from our partner vendor HPE, as outlined in the spec, and new vendors may be added later through a DAO process.
Secure boot is important for providing assurance that nodes are running an unmodified version of Zero OS. In combination with a TPM chip, it’s possible to verify the entire boot sequence of a device and ensure the integrity of the code that’s running. This is the highest level of assurance that nodes can execute sensitive workloads in a secure way. It will also require that someone either from ThreeFold or a third party will need to configure the BIOS in these nodes, as our certified vendors do.
I know this will be disappointing news for some farmers hoping to become Gold Certified. The good news is that there will be other ways to increase your farming rewards according to your infrastructure investments. We have planned, but not yet implemented, boosters for this purpose, which could be based on uptime, bandwidth, IP addresses offered, or utilization of nodes. If you have proposals for the first boosters we should bring online, please create a topic here in the forum so we can discuss it further.
1 Like
So what’s the point of voting if specs are set in stone upfront?
1 Like
Is this how we must expect future DAO approvals to be proceeded? Just posting some unspecified thoughts into forum and make it a poll … declare it as “approved” after you guys let the community discuss for month, telling them to take those comments into consideration … than just ignore every single contribution and put the same imprecise so-called “agreed-upon” requirements to a TF-Chain based vote (to be approved by just 10 voters. really?!) …. and then - ”tata” - here comes the magic: “…let’s just do what we want and specify details afterwards.” Is this what you guys call “transparency” and “community first”?!? I’m stunned! Didn’t see this one coming and would never have expected how this went.
There are so many imprecise expressions, completely unspecified parameters and unanswered questions with what is “DAO approved” here.
You are asking for 5 public IPs per node. It took me month to figure out how public IPs need to be configured because you guys couldn’t really tell. I was asking so many times for knowledge base and support. Sometimes I was waiting days or even weeks for an answer on most trivial questions. Sometimes I didn’t even get an answer and we still don’t have proper documentation on networking in detail (not only public IP configs).
You never made clear how gold certified farming networking infrastructure should be configured. There is no whitepaper, best-practise or basic concept at all, or it has never been communicated. I “wrote a book” about the single-point-of-failure problem and I even didn’t get a technical based answer. You expect us to work with phrases like “Two routers per rack” and “At least Two Internet Service Provider connections”?! What is this?!? How can you base such an important program on so imprecise expressions? That’s not ok!
And now you come up with secure boot and TP-module?! NOW?!?! That was never mentioned anywhere, anytime!! Is this a joke?! Have you guys ever spent a second together and even tried to think this through??? You can’t tell me you forgot about that…and you also can’t tell me that we (farmers) could have known this from the once more imprecise phrase of “3nodes need to be certified”. Most refurbished servers don’t come along with TP-modules. There are hundreds of those servers running on the grid right now. Many of them waiting to be moved to DCs. And TF should be happy to have them as nodes. Apart from it seems like you are trying to sell us secure boot and TP-modules as the holy grail of security and that it would be indispensable. That’s just not true! There are countless hacks on TPM and proves of secured booting sequences being manipulated. Apart from… you are dealing with farmers not enemies! What do you think we are going to do with the nodes after being certified? Or is it just ZERO-OS being so vulnerable that it must be hardware based secured?! I thought it would be secure, self-healing, autonomous and so on?!? I’d love to get detailed information on that, but sources are very much limited.
I mean I understand that you guys might have a specific idea of how gold certified farming should look like. BUT!!! That’s maybe not the most important thing to take care of right now?? Now TF should be appreciating every node that’s being run in a DC. But instead of getting support, you cut us of.
THAT’S NOT OK! YOU CAN’T DO THAT!!!
We relied on you. You made us believe that we will figure this out together and that we will find a solution how to get it done. Instead, we are left with a big “here…eat this shit!”. And you guys just do this by the way. And then this is called “maybe disappointing”?!? That’s not disappointing…
…that’s a big fucking mess!!!
sorry if you feel this way, everything was done with best intentions,
we have indeed been talking about this for months
as far as I know we integrated all info we could and integrated in proposal, see the forum
and we together agreed on the very basic specs for certified gold, this is by no means complete enough, over next weeks we will have to complete this specs to something very concrete (e.g. a contract).
its important to realize there are 2 different ways how rewards are given for farmers who do the extra mile:
- certified farming has been from day 1 a service which ThreeFold Technology company would be occupied with, it means users will need license agreement, support. We have quite some commercial farmers who spend lots of money (some of them millions) since years to make it all possible. The certified farming has been developed for them. In future more commercial companies can do this.
- then there is rewards for people hosting in datacenters and doing their best to have best uptime, bandwidth, … for this we have the boosters, which need to be defined properly
We are in the middle of defining specs for TFGrid 4.0 which will have hopefully more details for all of above and gives us something to discuss about for the next steps.
If your worry is being rewarded in line with your effort, I think we all together want to exactly accomplish that. Let’s make sure this happens, but for non commercial farmers this will be done by means of boosters, because we need a decentralized way how we can do this. Certification is not decentralized and never has been intended to be decentralized, this is done by means of contracts, payments, license agreements … everything any commercial provider would expect from us.
thank you
Kristof
1 Like
ps secure boot was always mentioned, certified farming means, using certified nodes, certified nodes means the nodes need to be locked by a certification partner (today threefold tech) by means of bios, secure boot and in future TPM.
This is the only way how we can provide security for the TFGrid users and thats the point of the certified farming & certified nodes.
2 Likes
@Dany I fully understand your points and I agree with some but I will only talk about technical points and not about the voting decision.
Keep in mind that we try to provide a secure environment to customers, hosted by farmers who have physical access to the nodes. And you probably know that it’s nearly impossible to have a perfect solution when physical access is possible.
Using SecureBoot and TPM based feature to increase the security is the only way you can put in place easily for mostly everyone already on the grid. Theses technologie are available on almost all hardware attached to the grid and is the most common for no-cost solution right now. Other method you would put in place always depend on external hardware (and usually expensive).
Btw, arguing about hardware security or security hole in theses technology when talking about using 15 years old hardware is quite irrelevant, there are way more security issue with theses old hardware.
So well Secure Boot and TPM based solution are maybe not the best practice we can put in place but at least it’s one security layer we can apply to mostly everybody without any additional cost. There is no real other way we can use to store encryptions keys and/or node id to make an autonomous system working securely, being able to work without human intervention when people you don’t know can have physical access to node.
Even if it’s not perfect it will make way harder any way to corrupt, exploit, extract or tamper data and the OS 
3 Likes
I need to understand quickly; as an official 3Node supplier and partner we have access to certifying nodes. We are producing and selling the Peking (EU) for instance.
- We are about to be a partner with a cooperation and become the preffered suplier for one or more datacenters. We would like to convince them for buying our 3Nodes and get them involved in TF grid. We have access to some DL360 and 380’s from which we have received approval from Iouri to get them certified. However, the above "(today threefold tech)"worries me. I need to know today that we are able and allowed to produce and sell certified HP rackservers!
- I assume this also means we are partnering with TF to certify current 3 Nodes soon, after it has been worked out what the procedure and costs are made clear, correct?
Not getting an answer is worrisome…
Dear @kristof,
it took me some time to try to get back to objectivity, if you know what I mean. I have to admit that I am very angry about how this went. Of course my feelings and expressions are mostly based on my/our personal view, but I am still struggling to understand what is going on here, even if I try to not focus on our individual considerations.
First of all, I do not doubt that everything was done with best intentions . It is incredible what you and your folks have accomplished so far. I feel pleased to be part of TF and will always try to contribute.
I do realize that we have agreed on basic specs but I had a completly different understanding of what you are aming for here. And to be honest… it was not possible to extraxt that out of your posts. The fact that gold certified farming has been developed for commercial farmers who have spend millions already is a completely new information. The way how you make a difference between so called “commercial farmers” and “people doing their best” is unacceptable to me. It looks like you never intended to let farmers like us to participate in that program. Thats what I am struggling with the most. I mean it’s absolutly ok with me to reward those with whatever you think is appropriate. Give them 100x times rewards… I don’t care, we would support this. But please also take care of other DC farmers. We also consider us as commercial farmers since we had to register a business, will have to pay taxes and have spend serious amounts of money to get this done. We do contribute much and it’s also us who make the grid more decentralized and stable. And please… let’s be absolutly clear: it’s not the money that got us here! We are here because we believe in TF…and its mission… and its dedication… nothing else! But at the moment we just feel to be left alone… and that’s really sad.
It don’t matter how bad our hardware migth look to you. There are thousends of servers like ours being run in DCs used by commercial cloud service providers. There are even hundreds of serves likes this providing capacity to the grid rigth now. For most workloads this hardware is perfectly fitting. Once again…TF should be more than happy of any node being run in DCs. I mean this discussion is not about the hardware itself… it’s about environmental conditions which results in uptime and bandwith only. That costs… and I know that you must be aware of that!! At the moment we have to push much money in every month in order to cover costs for DC farming. Not even to mention not to get a single cent as a return of invest rigth now. You are talking about that we would have to realize that there are different ways how DC farming would be rewared and that for people like us doing their best (which feels some kind of being belittled) would have to count on boosters. Actually this is the first time hearing about this. Where are those boosters? When can we expect this to be happening? Do we have to wait another couple of month until we migth participate from that?! How much additional rewards can we expect? We were counting on the gold certified farming program. There are countless indicators in forum where you (or other staff members) could have taken knowledge of us trying to participate. Why didn’t you let us know in the first place? We would have found a way to deal with this on our own. Instead you made us believe we are going to find a solution and would figure out together. That’s obviously not the case. But we need a solution now! We are bleeding out!!! I don’t understand why you would ignore this.
Let me tell you what we are being forced to do right now: For now it would be the best to dismantle our DC farm and put it into a basement without any UPS/PDU, AC, professional networking infrastucture, secured physical access, dust-free and other environmental conditions and so on… and hide those nodes behind an un insufficient internet connection in order to not attract any workloads so power consumption would stay low. Is this what you want? huge amounts of unreliable and more or less useless capacity rewarded every month? Thats excatly what many others are doing rigth now who pretend to be waiting for the “rigth time” to move to DC?
The time we made the decission to go for DC farming was long ago before gold certified farming was even anounced. We thought that this is what the grid needed and it would be the rigth thing to do (maybe we thought wrong). With 8 cent per token (or even a bit less) we could have been able to handle this. But the moment we got on that road the token went down and is now cut in half. The gold certified farming programm was our drinking straw and we hoped that this would rescue us from drowning. We are bleeding out!!! We need your help… otherwise we won’t survive! I am perfectly fine if you come up with a time limited booster or what ever you want to call it. Maybe on a monthly to be GEP approved base until regular rewards would cover costs. But we need this NOW!!
We can’t wait!! We need something we can rely on… something that we can grab.
I please you… take our concerns seriously … and adress our problems urgently. We can’t wait any longer!
Thanks
Dany
2 Likes
The TLDR of this is those of us in DC’s and those of us planning to move to DC’s need the incentive to stay there/move there. Even without shiny new Gold Certified level servers, the grid desperately needs those public IP’s with good bandwidth to make for some marketable capacity.
I suppose this is the “booster” program discussed. How should we start this discussion? Originally token lockup was designed to make us create marketable capacity right? Token lockup forcing us to make our capacity more marketable won’t work in the current market. At 8+ cents it could, but right now I’d just declare bankruptcy and use the servers for targets.
2 Likes
Thanks for jumping in here. Public IPs is just one example in this scenario. We have a full /24 IPv4 block ready… and I would love to provide them to our farm. But at the moment that would be suicide. Every IP costs us 1.50 EUR (without TAX). And public IPs do gain attraction. We can’t cover costs of utilization. Every watt in power consumption (and corresponding cooling power) is giving us more costs than we already can’t cover with rewards. Apart from the actual rewarding structure does not value public IPs accordingly. The most you can get of public IPs is a couple of bucks per month, only if those are used. increasing power consumption according to increased utilization is not even considered and does not has any effect on rewards.
2 Likes
Same decision here on the public ip’s for now, definitely not worth the current rewards.
I can only imagine how hard the TF team is working to make this project successful and also the many unexpected or unanticipated matters that arose. Since this project started with the best intentions in mind I’m sure those intentions are still there.
I can only assume it must be a lot for TF to take care of, but they only need to draw in some of us enthusiasts and partners and 1) let us help them grow and 2) give us really the feeling we’re all part of it.
Again, it can’t be easy, but i frowned my eyebrows when we pretend to be autonomous by voting about whether Gold Farming is a good idea without telling us the specs and conditions upfront. Now that we voted yes, we are being disappointed.
I believe the point of autonomous voting is that we should have ALL information available to make a thoughtful decision.
I do agree on much of the above but especially the fact that it can take weeks to get a response, even if it’s just to explain the current process, while folks are making investments with 'logical assumptions ’ in mind.
2 Likes
Hey Robert, the forum is generally not the best place to get timely answers on important matters. Telegram (better) or email (okay) are the ways to go. If this hasn’t been resolved, please connect with me.
Dany, I understand why you are frustrated, and I want to say first that I’m committed to resolving the very real issues you’re noting about inverse incentives for farmers with public IPs, data center hosting, adequate bandwidth, and active workloads. More on that below.
I’ve reviewed the discussion in the GEP thread. When it comes to certifying hardware not originally sold as certified, this was left open ended. As for the definition of certified nodes, our wiki is the “single source of truth” and that’s covered on this page:
Plug and Farm and pre-configured 3Nodes are certified meaning they went through a secure boot process to guarantee increased security on a heardware level.
“Pre-configured” here refers to the rack mount nodes offered by HPE, as seen further down the page. If any conflicting information was ever given, I’m not seeing it, and I don’t see any discussion of changing the definition of certified either.
To be honest, I was surprised to find that so many rack mount servers in use on the Grid don’t support secure boot, since the technology has been standard in consumer machines since at least 2012. I’m not super knowledgeable about the capabilities of different generations of server hardware. Even if I were, I don’t know the model numbers of your nodes and I don’t see that you mentioned this (aside from the unit you use as a router) in the other thread.
As I said before, mentioning the secure boot requirement as the definition of a certified node in the Gold Certified specs would have been the best thing to do to provide the most clarity, absolutely. However, I just don’t agree that this specification was ambiguous or that the team should be responsible for knowing what hardware farmers participating in the Gold Certified discussion were planning to apply with.
Anyway, I hope that we’re all ready to move on to discussion of how we can align incentives with the extra investments involved in DC farming with public IPs. I see a few possible boosters we can pursue for this purpose, which I’ve outlined in a new topic. Let’s continue the conversation over there:
2 Likes
First of all I’d like to thank you @scott for giving additional and clarifying information on this. I think that the most of our disagreements here are based on lack of communication.
I have to admit that I have only overviewed the wiki page about the definition of certified nodes. I didn’t made a mandatory connection with what is mentioned on that wiki post in regard of what will be required as “certified” when it comes to gold farming. There is no generly confilcting information but we thougth that this post
would give an open discussion about what hardware would be fitting here and how the certification process would look like. My understanding of “gold certified farming” was, that this is about DC farming in general. That assumption was obviously wrong.
You are rigth…it was not particularly mentioned in this thread. Well… it is somehow possible to know what servers we (and @FLnelson f. e.) are using from other posts but I agree that you are not in charge of keeping track of what hardware farmers are using.
Apart from I would like point out that I absolutly agree that secure boot (based on TPM) is a reasonable way of adding another secure layer to OS/nodes (shout out to @maxux42).
Anyway… let’s go on… looks like we won’t change on what is decided already. I am happy that you @Scott feel in charge on how to figure another way out to make DC farming effordable. So let’s get it on here…
1 Like
Sorry for late response, I have been very busy last couple of weeks and didn’t find the time to read the forum, I am sorry about that. We are working on some suggestions which might resolve some of the points as mentioned above, more to come very soon.
We would like to introduce the first farming boosters to give more reward for farmers who can and want to achieve good uptime and good performance in relation to nr or CU and SU.
2 Likes
we need to find a way to continue with the farming boosters, I don’t have much time to follow up, who can own this?
@scott and me will own this and drive this forward. Also - for all future booster implementations we will take a leading role…