CPU benchmarking for reward calculations

#1

Following questions and discussion on Telegram, I’m opening this thread to discuss CPU benchmarking and how it could impact farming rewards in the future. Ever since I started farming almost two years ago, there’s been mention of passmark benchmarking in the wiki as a future factor in farming rewards.

The original spec, which can still be found on the current wiki here, look like this:

CU (Compute Unit)
GB Memory 4 8 2
nr vCPU 2 1 4
Passmark Minimum (expected is double) 500 250 1000 CPU performance

Then in January 2022, this note was added on another page:

  • please note minimal passmark per CU (with 4GB mem), needs to be 1000 passmark at farming side, this is not being checked today but might be done in future. If your chosen CPU has less than 1000 passmark per CU (of 4 GB mem), it could be your final CU’s will be lower once that feature is introduced.

The table above seems to imply a minimum of 250 per thread, with 500 being expected. The note below sets a higher standard. A node with 8 threads, for example, can have up to 16 CUs. That’s a big spread of 2000 minimum to 16,000 minimum.

The goal of this policy, like any other, is to make the Grid a useful utility while rewarding farmers in a fair way for the capacity they provide. Different CPUs have different performance levels, and this must be eventually be considered some way in the model. Farmers shouldn’t need the latest and fastest gear. Midrange systems produced in the last 5 (maybe 10) years should be fine.

Examples

Part of my goal here is to gather some data which will help us dial in a reasonable requirement. Feel free to add your nodes specs in the replies to help build a sense of how benchmark requirements will impact farmers existing (or planned) setups.

Titan

The Titan is configured with 32GB of RAM and 8 threads, which means it has 7.75 CU. The CPU in older AMD based models was a Ryzen 5 3400G with a passmark rating of ~8000.

My node

My HPE EC200a has a lower power, SOC Xeon D-1518 with 8 threads and a passmark score of 4,784. I installed 64GB of RAM, giving it 15.75 CU.

How much speed does a server need?

Like in our storage/bandwidth ratio discussion, reasonable limits here depend on what a system is actually used for. Since I don’t have much experience managing servers, especially not those running demanding applications, I don’t have much perspective on what’s appropriate here. But maybe you do?

3 Likes
#2

Hi Scott,

Excellent start of the thread and thanks for opening it based on the discussions in Threefold. For me, there was a lot of confusion for what unit the 1,000 passmark requirement would be. Core? Threads? CU?
I understand from your example that it’s going to be CU…which is the CU value that is generated in the simulator.

In my case, I have a HP Z840 Workstation with dual E5-2698v3 (16core/32threads each) processor, 512GB DDR4 Ram and 7TB SSD. According to the simulator, that’s giving 127.75 CU.

If we have a required benchmark of 1,000 passmark per CU, that setup will never be enough: it would need 127,750 points. The setup (looking for dual CPU setup for this system) gives 31.489 points. It would never be enough for the 1,000/CU requirement.

From seeing the various DIYs out there it this would be a requirement, 90% of the DIY will not be up to the task.

I’m no server expert, but if I had to chose where to deploy a VM, I would rather have the DIY as mentioned above (that would not fit the 1,000 requirement) then a Titan (that would fit the requirement).

2 Likes
#3

so there will be 3 different CU types? one “balanced” and two focused on MRU or CPU?

And now:

  • MRU type (8:1 MRU:CPU ratio) will need 250 passmark?
  • CPU type (2:4 MRU:CPU) will need 1000 passmark?
  • “balanced” type (4:2 MRU:CPU) will need 500 passmark?

That what I understood from that table. Am i right?

Edit: btw i agree about that 1000 passmark per CU would kill most nodes. I have E7-4890v2 and it will be not suitable. But considering that I could put 8:1 MRU:CPU ratio it would be more than enough.
Edit2: if it was like having 3 different type of CUs with passmark from that table (or even a little higher) i’d vote YES.

1 Like
#4

Hi,
here are two examples of my farm - seems the passmark is not very kind to a higher count of cores!

old system would pass:
Dell R710 - dual XEON X5660
24 Threads & 192GB RAM & 6TB SSD
CU = 47.75
Passmark = 12266

new system would fail:
DIY with EPYC 7551
64 Threads & 512GB RAM & 8TB SSD
CU = 127.75
Passmark = 23002

#5

Scott, could you please write out the formula that you’re applying to arrive at the CU. It might be straightforward for some but it’s not exactly clear for everyone…for example, what formula arrives at 7.75CU for the Titan? This way anyone can take the formula, apply it to their node parameters and effortlessly figure out where they fit in the 250/500/1000 passmark spectrum. Or are we simply relying on the simulator…plug your RAM and thread count and take whatever shows as CU? Thanks.

1 Like
#6

Yes, if you write your specs on the simulator.grid.tf it should give you the correct CU.

Hope it helps. The simulator uses the formula:

cu = min((mru - 1) / 4, cru * 4 / 2, sru / 50)

1 Like
#7

I’m a little clearer on some aspects, but not 100%. Here is what I have planned at scale:

Dell Rx20(12G) and HP DL3x0 G8
2x E5-2650L V2
320GB RAM
4TB SSD
CU = 79.75
Dual CPU Passmark = 13,597

If I’m understanding the formula correctly, I’m in the 8:1 ratio based on RAM to thread. At a minimum of 250 * 40 = 10,000, I technically pass just above minimum with an overall PassMark of 13,597.

I chose the E5-2650L V2 based on optimal thread count to TDP since the simulator only accounted for thread count and did not mention PassMark. TDP is 70W

A good middle ground in TDP to PassMark ratio would be the E5-2660 V2 with a dual CPU PassMark score of 17,585. It doesn’t quite hit the expected number of 20,000, but its close, and TDP is at 95W

To meet the expected PassMark in this generation of hardware I’m using, I would need to opt for E5-2690 V2 with a dual CPU PassMark of 22,690, but at the high TDP cost of 130W.

TDP factor is considerable at scale. I am building infrastructure to support 100 amps of sustained draw @ 120V.
2650L V2 = 42 servers
2660 V2 = 36 servers
2690 V2 = 30 servers

I believe I’m doing the math correct on this whole thought process. Someone please correct me if I’m wrong.

Again building at scale to a 40 node farm, I would certainly want the build to be viable for at least 2-3 years. If I can start on 2650L V2 procs and run those for the expected 2-3 years, this would give me ample funding to then overhaul and expand my farm. Is this path viable?

2 Likes
#8

Excellent post. This is THE question we need to settle.

I’d say that first, if we come up with numbers and it disqualifies farmers that have already invested in the hardware, that is not good at all. We really need to make sure early farmers and investors are not left behind with new decisions. (When this happens in any blockchain project, it really hurts the community trust.) I know the passmark information was there but it’s not very obvious for everyone, and there’s so much ambiguity already. That being said, I’m sure we can find a good middle ground that is good for farmers and users. Once it’s settled, it needs to be added to the simulator. People refer to the simulator mostly.

From what I read and saw on the TG farmer chat, I think that between 500 and 1000 passmark per core would work for most of the farming set up. (250 to 500 passmark per thread.)

I think what could be done is to reward positively the farmers that go beyond these numbers. With the upcoming tools showing Grid and 3nodes stats, passmark could be put forward and users that need high passmark with be able to choose the appropriate farms. We have to take advantage of the Grid. If we can have different options of 3nodes for different kind of uses on the Grid, we cover more ground and this also gives the opportunity for more people to farm. People’s Internet, you know!

This would not penalized farmers who already built a farm between 500 and 1000 passmark per core.
I would not penalize farmers with below 500 passmark per core if their farm is already setup. Future farmers (when the numbers are settled) that would go below 500 passmark per core would be penalized. This ensures a minimum of efficiency for the whole Grid.

I have a R720 256GB ram 4Tb ssd 32vcores with 2xE5-2640v2, thus 63.75 CU in total.

The CPU passmark is 7730 for 8 cores, so almost 1000 passmark per core. This would fit in the range of 500 to 1000 passmark.

We have to be realistic. I don’t think us farmers would have invested so much if we needed 1000 passmark per CU. I would need with the R720 example 64 000 passmark, or 32 000 passmark per CPU. This is not possible with the RX10, RX15, RX20 Series, and perhaps also the HP Proliant Gen 8 and Gen 9. (Correct me if I’m wrong!)

Just to make this comment even longer: I hope you guys are doing fine. Happy farming!

4 Likes
#9

Great insight and I agree that some serious settlement with early adopters is imperative for the project’s future. I’d be remiss if I didn’t say that I’m quite frankly taken aback that such a consequential element of the project was heretofore buried deep in reading material. Yet it’s conspicuously missing in the simulator that remains the flag bearing go-to vehicle for reference. This can’t bode well for trust and effective communication in any project.

#10

Yes, dell r920 is also out of range for 1000 passmark per CU.

1 Like
#11

Good to know for documentation. I added this in the comment. (Nice machine!)

#12

From Passmark webpage:

Intel Xeon E5-2698 V3, single thread rating is 1990 which is enough

Screenshot 2022-03-16 at 21.30.13

#13

But are we going off of a single-thread rating? That is what I would like to know.

#14

Per thread rating assumes you are only running a single process. While this is a useful metric for many consumer uses that don’t involve heavy multitasking, nodes on the Grid are expected to do parallel processing. Therefore, we’re looking at the total passmark score.

Thanks everyone for your input. I hear what you’re saying about “moving the goalposts” and I totally agree. It also seems pretty obvious to me that 1000 passmark per CU is generally unattainable for high thread count systems with large amounts of RAM.

High end server CPUs

  • Consider the AMD Ryzen Threadripper PRO 3995WX, a top of the line server CPU. It has 123,631 passmark and 128 threads. That means its max CUs is 256 and passmark per CU in such a configuration is closer to 500.

  • Down the list a bit we see AMD EPYC 7543, which has 64 threads (128 possible CU) and a passmark of 103,492. Still short of 1000 per CU, but close.

  • Then there’s AMD EPYC 74F3 with 48 threads and 94,726 passmark.

  • And AMD Ryzen Threadripper PRO 3955WX with 32 threads and 63,885 passmark.

Based on what we see here, only the highest end CPUs can come close to 1000 passmark per CU in configurations with high RAM and SSD.

On the other hand, it’s hard to find any processor which actually has less than 250 passmark per thread, so I’m not sure this represents a meaningful minimum.

Per CU or per Core?

Whether to consider per core or per CU is interesting. A per CU requirement effectively puts a cap on the amount of RAM it’s profitable to install in systems with slower CPUs. This makes sense in a way, but then again, my laptop tends to get RAM bound way more often than it gets CPU bound (different use case, of course). A per thread or per core requirement, on the other hand, might penalize a slower CPU with less RAM attached that’s under its max possible CUs.

I’ll leave it here without a conclusion for now, but I feel like we’re getting somewhere :slight_smile:

3 Likes
#15

Personally, I think that running slower CPUs that draw less power while we scale up the Grid and collect data on usage makes a lot of sense. If such configurations turn out to be sufficient to gain significant utilization, then no problem.

Finding a way to pay more to farmers who run faster CPUs and potentially pay higher power bills in the meantime makes sense too. If we can do this without penalizing anyone who checked the simulator and invested in gear based on what they saw there, all the better.

4 Likes
#16

Why is anyone getting worried? This doesn’t seem to be a problem. If you follow the 8gb ram per thread then you need at least 250 passmark per thread to operate but 500 per thread is better. My i7-6700t lenovo falls in that range but a little under the “expected”. My workstations with v2 E5 xeons that many of us are using exceeds the “expected” by a little bit

3 Likes
#17

It will be not fair for old Threefold farmers to introduce passmark values - they invested a lot of money already and now they will be penilalised for that. If you want to loose a part of the community, penalizing old farmers is a way to do it.
It will be more fair to introduce those passmark limits for new farmers that will come on the future, leaving old farmers with their servers on the threefold grid, with same amount of contracted coins like they had before.
You can’t tell people that they will earn X and then add penalties for too slow cores - it’s just not a good thing for community.

1 Like
#18

The minimum requirements seem pretty low though so I don’t think the any purchased equipment (including used stuff) is going to be made obsolete anytime soon from my understanding of the figures. It’s going to be people running really outdated hardware that won’t meet minimums. I have a desktop I built 15 years ago that I fixed up recently for fun. It sure is slow though. That’s like 8 generations of Moore’s law. It technically would meet the minimums. Most equipment that old is probably getting recycled for raw materials with little to no resale value.

2 Likes
#19

Hey guys,

Any consensus on this? Here are two options.

  1. 1000 passmark per core

Or perhaps

  1. a) Between 500 and 1000* passmark per core is necessary, and over this number you get bonus in rewards
    b) Under 500 passmark per core you get penalized.
    c) Under a certain threshold, there are no rewards (too old system). Say 250 passmark per core.
    d) We do not penalize old 3nodes.
    d) It’s only for 3nodes that are connected once this decision is clear for everyone.

(It could be between 750 and 1000 instead, etc.)

What do you think?

#20

That sounds good.
No instant penalties are somehow important for existing nodes and people who are just about to bring nodes online.
Anyway there should be a limit when also the rewards for existing nodes start to decrease. 24 months maybe?
Making it interesting to invest in new hardware.

Oh, and maybe a performance index could be shown when renting capacity. For some it might be important to get faster CPUs.