Incentive for farmers to attract workloads

#1

Hi everyone.

There have been good improvements to attract users to the Grid like a default discount on deployments, and extra discounts if you have a certain amount of tokens. Which is very positive since an attractive price can bring lots of utilization. Also the boosters are good to bring attractive nodes online for dedicated renting.

There is still a pressing issue I believe regarding the tokenomics for rewarding farmers to host workloads. Providing grid resources is one, no Grid to deploy on otherwise. Then next would be running workloads on them to actually utilize this already big network.
Currently there is little incentive for a farmer to attract any kind of workload to his/her node(s). Only if you can provide some public IP’s (most can’t) or get lucky to have a workload that uses a lot of bandwidth. The extra you can get does not seem to be enough to try to get your nodes filled with workloads.
If you don’t have public IP’s or a workload that uses a lot of bandwidth, getting for example a miner workload could decrease your profit since your node would start to use more electricity.

Of course we can’t just keep on increasing rewards for farmers every time we want to resolve something. Tokenomics are very complex to design, also very sensitive to personal opinion. But rewards can come in many different flavors forms and even combinations of things.

The following are not suggestions, just ideas as to how we could resolve the issue. To get you guys going with ideas! :wink:

  • A farmer could get some more rewards over time, if they can hold workloads for a minimum amount of time. For example first month they get 1% extra for each workload they had for min of 3 weeks. Next month 2% if the same workload is still there for a total of 2 months, … with a max of 5%.
  • A farmer could build some reputation count based on workload age that gives him bigger advantages when using the gird.
  • Soon there will be the introduction of Solution Providers, which is awesome! A farmer and a solution provider could sign some % agreement in a smart contract. Then if the farmer received a workload with this solution provider ID in it, the farmer receives the agreed % from the solution provider. For example a workload costs 10TFT/h, the solution provider receives 4TFT/h and the farmer 1TFT/h. This way the solution provider can provide known reliable nodes from a known farmer for the workloads of his users.
  • We could introduce some node benchmark, which is used to rewards an extra specific amount based on the performance of the node vs workload age
  • …

Do you think something like this is lacking? What would work the best for you? Maybe we create a whole lot of other problems trying to implement something like this?

The idea is to not just hunt for profits as a farmer (no offence to anyone here) but provide valuable hardware that user want to use, and get rewarded fairly for your effort.
Now, the hardest problem with this suggestion I think is the current nodes that are online on the Grid. A lot of them have been build to generate the most amount of tokens (following conversations on the farmers chat :wink: ). Meaning an old cpu with lots of cores would create more tokens then a brand new one with less cores (simply put). While any user would prefer the newer cpu, especially at the same price.
It made more sense in the past to build a big machine, regardless of the age of the hardware. So if we want to reward farmers with something extra for workloads, we have to be careful to not create a disadvantage towards the already running nodes. So early adopters do not feel left out.

Difficult question, for sure, but I think very necessary to build a reliable grid with a working ecosystem for the benefit of all participants.

p.s.: Not a farmer myself

5 Likes
#2

That’s a novel idea, I like that one.

Doesn’t the original proposal to lock up the rewards unless you get 30% utilization for 3 months kinda solve this issue? However, I do fear that if this project catches on heavy, that would get voted down in a DOA fast.

I’m one of the people who has helped create that environment of slightly older servers. I’m personally ok with being left behind in some regard if it means the grid is more appealing (as long as the plug isn’t pulled on me, and at some point it honestly should be when bandwidth monitoring comes into play).

#3

Don’t worry Nelson, if you run out of bandwidth I have lots of room!

#4

As I understand it, your rewards are unlocked at 30% utilization, your not rewarded extra for the utilization which is what this issue is about. You get the same amount of rewards, only more early if you have the utilization.

I’m not in favor of creating any kind of disadvantage of being an early adopter, you guys made it possible for the Grid to start in the first place.
Also think that older hardware has it’s place, and if it could be cheaper to rent it can even be more attractive for a lot of use cases (all sorts of testing, light unimportant workloads, an old cpu works fine for storage, …)

#5

Yep, it runs my websites just fine. I don’t need much for some wordpress sites.

#6

It could be hard for smal farmers to sign a contract with a solution provider. For big farmers this could be something. But the dao could also decide to arange this for every farmer. Now 50% of the utilization tokens is going to the solution provider, it may could be splitted in 40% to solution provider and 10% to the farmer. In this case every farmer provits from utilization

2 Likes
#7

Wouldn’t it make a lot of sense for larger (or small, but may not be worth all the effort) farmers to become their own solution providers? We’re planning to build a farm that is powered on 100% renewable energy from our own production and I have already been speaking with some potential customers that would be interested in booking our green capacity. If we market our capacity ourselves, I think benefiting from additional utilization rewards can be a great incentive (Thanks to @Dany for pointing this out to me).

3 Likes
#8

I think you could have the 2 roles at the same time: solution provider and farmer. In practical terms you (as a solution provider) could establish an agreement with yourself (as a farmer). This solution should be accepted as good and I am sure it would create new possibilities for those who want to start a provider business. If in the future they need more capacity they could invest on their own farms to scale or go with an hybrid model.

3 Likes
#9

Good idea indeed. True, most farmers will not spend the time and effort to look for a solution provider and try to get a deal with them (% wise).
But if we would do it by default, this could have a lot of impact on token distribution. Which would be good and possibly bad, but would need to be thought true very well. There would be less, by default, for a solution provider to earn for example. Or what would be a good % for the farmer to create enough incentive to attract workloads?

#10

That is currently perfectly possible I believe. If your approval for solution provider is approved, regardless if your a farmer.
But there is a difference between a farmer and a solution provider.

  • Farmer: providers capacity
  • Solution provider: provides workloads on that capacity

So just being a farmer will not get you approved as a solution provider. You will need to be able to show how you are actively adding workloads to the grid, with a solution you created like a UI to deploy on the grid, a custom created flist, maybe trough education of how to use the grid, … etc

1 Like
#11

Yes that should be perfectly possible. This way you can earn quite some extra on your farms. You would get the monthly minting payout of your nodes, and 50% of each deployment on your nodes.

#12

Hmmmm, I like the sound of that

#13

This conversation has some overlap with our boosters discussion, but with the additional possibility of routing some portion of the solution provider rewards to farmers. I personally like the latter idea and seem to remember discussing it before. I think it would be very cool to see some kind of “farmer’s grid capacity marketing collective” which onboards users and generates extra revenue through the solution provider fee.

While I think that this was already an important imbalance in the rewards model to address, it’s getting perhaps even more important when nodes are able to be fully powered off while idle. The cost benefit ratio is stacked even higher against having workloads on your nodes, if they would otherwise be mostly powered off instead of just idle.

Indeed, this is the originally conceived way for v3 tokenomics to solve this issue. I’d be curious to see how the vote falls in our existing farmer community, since at least many of our more vocal members seem to be hodlers. I’ll leave details of that discussion to this other thread, but just say here that I don’t think it’s the solution to pursue right now.

Boosters provide a simple way to at minimum keep things fair for farmers who pay more for power because their nodes are utilized. Of course this calculation depends heavily on how much a farmer pays for electricity, but we can try some math…

Let’s say we have a rack style server with 64 CU that draws 150w idle and 300w full loaded (based on some example figures I found for Dell R620s–let me know if this doesn’t look right). It earns 2080 TFT per month. With an electricity cost of $.1/kWh or $.3/kWh:

kWh/month Monthly cost ($.1) Monthly cost ($.3)
Idle 108 $10.80 $32.4
Loaded 216 $21.60 $64.8

Meanwhile, the value of TFT farmed every month is $62.4. So our farmer paying $.1/kWh needs to be boosted ~17% to break even, while our farmer paying $.3/kWh needs a ~52% boost.

I only checked after choosing these example numbers, but the world’s average electricity price was $.13/kWh and the highest was $.412/kWh as of June 2022 (the reality is currently changing fast, however, especially in Europe).

Again, that’s comparing loaded to idle. Comparing to nodes that are using negligible electricity simply proving that they still exist by powering up on a daily basis is a bit different. At least one node needs to be powered on at all times to be able to wake up the others. This means that there’s a significant jump in power consumption whenever a new node needs to wake up and start running workloads. I won’t try to full model this case until I have more details on the implementation.

1 Like
#14

Let’s consider another approach to this that keeps things pretty simple and doesn’t involve minting any more tokens than we already are. That is, simple directing some of the sales channel fee toward the farmer. In order to be a net positive for the farmer, the additional payment they receive would need to be more than the additional money they spend on electricity.

I did a bit of analysis in an attempt to find out how this would work in practice. It’s pretty tricky, because there are quite a few variables and not so many examples of measured power consumption for given setups. If you can provide any figures for how much power your nodes draw under different loads, or any links to resources like this, please provide them in the replies.

Comparing power cost to available sales channel revenue

Working with sales channel revenue is simple, because it’s always the same USD figure, regardless of TFT price. Therefore, we can just look at expected cost of electricity compared to what the grid user is paying for their deployment. The downside is that any discounts applied to the price also discount the available sales channel revenue as a percentage of deployment cost.

Discounts

It’s not ideal for farmers revenue to depend on whether the users of their nodes take advantage of discount opportunities. For larger farms, this may tend to average out. For single node farmers, having their node rented as dedicated, with a substantial discount, is an obvious disadvantage in the context of this proposal. Likewise for the case where a single node farmer has their node reserved at any level by someone utilizing a “staking” discount.

I’m not going to try to fully solve concerns around discounts here. Rather, I’ll just check how the situation stacks up when the largest discounts are in effect, and whether there’s still enough revenue to at least pay for farmers’ electricity. One potential solution would be to rebalance the payments to farmers according to the effective average discount across the grid for a period of time.

How much energy does a server consume?

The real answer here is that for every possible permutation of hardware configurations and workloads, there’s a different level of energy consumption. Some vendors have at some points in time published some measured power consumption figures, but they’re not common. Again, if you can point me to any resources around this or quote any figures you’ve measured yourself, please let me know.

For this analysis, I’m going to use an example from Serve The Home. In this article, they provide some figures for server power consumption under heavy CPU load. Here’s the test configuration:

  • Server : Dell EMC PowerEdge R640 1U 10x 2.5″ Chassis
  • CPUs : 2x Intel Xeon Platinum 8180
  • RAM : 384GB in 12x 32 DDR4-2666MHz RDIMMs
  • Storage Controller : PERC H740P
  • SAS3 Storage : 5x Samsung PM1635a 400GB Mixed Use SAS3 12gbps SSDs
  • NVMe Storage : 2x Samsung PM1725 1.6TB U.2 NVMe SSDs
  • OS Storage : BOSS controller card with 2x 120GB m.2 sticks
  • Networking : Mellanox ConnectX-4 Lx dual-port 25GbE
  • Power Supplies: 2x Dell 1100W

These are older, but not ancient CPUs, released about four years ago, which are reasonably efficient. The test workload ran the CPUs at 100%. This is unlikely to happen in practice, but it represents the upper bound of what a reservation could drive this node to consume. Measured power consumption was 500w, which corresponds to about 360kWh/month.

Here’s the total cost per month at a few different energy pricings. I’ve also included a 50% figure for reference, not suggesting the 50% CPU load corresponds precisely to half the power usage:

$.1/kWh $.3/kWh $.5/kWh
100% $36 $108 $180
50% $18 $54 $90

How much is the sales channel fee for this node?

If this same server were attached to the grid, the cost to rent it and associated sales channel fee looks like this (this assumed 50% staking discount is enabled by default, as it currently is). I’ve also computed half of the sales channel fee, assuming we want to leave at least half to be consumed by sales channels.

Default discount Dedicated node + max staking
Total Cost $751.22 $165.89
Sales channel fee $375.61 $82.95
1/2 sales channel $187.80 $41.47

Looking at the extremes here, this tells us that a farmer paying $.1/kWh who has this node rented as a dedicated node with maximum discount can still turn a profit getting half of the sales channel fee when the node is using its maximum power draw. That’s pretty encouraging, especially since the node would probably use much less energy in reality. If this farmer was able to claim the whole sales channel, by marketing their own capacity as a non dedicated node, they could earn an extra $300 in TFT per month!

On the other side, a farmer paying $.5/kWh can also make a small profit earning half of the sales channel discount when the node is fully rented as a non dedicated node and using max power. When the node is rented as dedicated node with maximum staking discount, even the entire sales channel fee won’t cover the farmers electricity when the node is drawing 50% of its maximum energy.

Conclusions

Overall, I think that allocating some part of the sales channel fee to farmers is a promising way to ensure that running workloads is a net positive. Even with a very simple scheme as analyzed above, this would work for a wide array of cases. By pooling the funds together to smooth out the dips caused by discounts, even farmers with expensive electricity could profit when their node is rented as dedicated with a discount.

With a pooling scheme a larger percentage, up to the entirety, of the sales channel fee could be allocated to the pool in cases where a sales channel is not involved. Successful sales channels would then divert some revenue from farmers, but assuming they are driving demand for TFT and its price increases, this should be a net positive for the ecosystem. Farmers who act as sales channels would get their sales channel cut before it hit the pool, while the rest would be pooled and distributed as normal.

The proof of utilization distribution would look instead like this:

Percentage Description
35% Burned
10% ThreeFold Foundation
5% Validator Staking Pool
0-25% Solution providers & sales channel
25-50% Farmers

Some final thoughts:

  • If the price of TFT goes up and the entry price for farmers is increased, pool distribution could be scaled down for farmers with a lower entry price, who are already earning a better profit
  • Less efficient nodes may still be net negative when they are utilized, especially where electricity is expensive. This is to be expected for gear of a certain age, but efficiency must be measured in energy per unit of performance, not just energy per core or GB of disk
  • Locking tokens earned in this way could also help to delay any sell pressure from this scheme, and provide a further incentive for farmers to keep their nodes up when they are running workloads
3 Likes
#15

Interesting fact, 45&49 on Devnet both see usage up to their capacity regularly, i have not seen sustained cpu load above 30% nearly at all on either of them.

#16

Awesome post! If implemented in this way I think it could be very beneficial to farmers and the project as a whole. Seems like you’ve already thought everything through to a great extend.
I especially like the idea with the pooling to keep rewards the same for all types of utilization (dedicated, discounted etc)

Just one question: with the statement below you mean farmers will receive their cut of the sales channel or that their sales channel tokens will be cut off? (Sorry if this should be clear)

Edit: now that I think about I’m pretty sure you mean farmers will receive their cut beforehand. Otherwise farmers marketing their own capacity would be disadvantaged, which probably makes very little sense.

1 Like
#17

A great post. In my opinion, this is the solution to compensate farmers for the increasing energy consumption due to the use of the server. At the moment you prefer not to have utilization on your node, as this will increase the costs. As a result, it is now more profitable for farmers to make their node as unattractive as possible. While nodes should be made more attractive, for example by investing in IP addresses. In addition, I never understood that farmers do not receive any rewards for using their node, since this costs more electricity and the hardware will also wear out faster. This will also contribute to the problem of farmers shutting down their node because something has been deployed on their node.

2 Likes
#18

Hi @liqearce. Some good points in your message. We have been at both ends of the farming token earning spectrum. V1 of the farming tokonomy included a fraction of the usage tokens going to the farmer. This had two downsides:

  • being recognized as a utility token cannot include payment income for the “utility” owner. This would make the token a payment token, which is far more regulated in certain parts of the world.
  • many farmers felt like they were “missing out” if their nodes were not used, and were looking at ThreeFold to promote, market and in the end create usage on their nodes. Some farmers (and farmer collectives) were actively looking into creating use cases and becoming service providers. While possible, this is a very different involvement and requires different skills.

In order to overcome this difficulty te tokenomics where changed such that the new token economics for farming only wooud match the old farming + utilsation income for farmers. This took away the feeling of missing out and made farmers, farmers.

Now with a changed world in terms of costs involved to operate a farm, an unused farm is the lowest possible cost level incurrent to be involved in Threefold farming. And with this changed world, we are (again) changing and improving things:

  1. unused servers can the turned off. More details are here. This needs some more advanced workload planning inside a farm to fill the used server up first and then spin up a new server when up and running capacity is maxed out.
  2. looking to create an affiliate program, where individuals (anyone basically) that present additional capabilities (an flist, a program to report on usage, anything) can claim a percentage of the utilization fee. the community could

Blockquote

consider including an affiliate fee for the high price for power.

#19

This is certainly a helpful data point. Since devnet capacity is free to use, I think it’s likely undershooting what a mainnet farmer would see with their node rented by paying customers who will be less likely to leave idle VMs up or reserve more capacity then they actually need.

#20

I’ve actually been considering moving them to test net for that reason, I feel like there may some ips being used by stale deployments since they don’t bill real tft