Current SU design ? How does this work exactly ?

Hi there,

I’d like to plan the storage configuration of my one node farm and for that I’d need to have a clear statement on how SU are actually used :

Different wiki statements for 1 SU :

Wiki states that the price for January 2021 is for actual consumption and not reservation. How does that work exactly ? Some real case examples and clarifications would be appreciated :slight_smile:

1 Like

Hmmm, and here it says 200gb SSD for an SU. I see that @weynandkuijpers was just working on this and might be able to weigh in on what was decided.

1 Like

Seems your link does not work anymore @scott :slight_smile:

Could someone from the TF team could answer these questions ? I think these are critical pieces of information for people looking to get involved with TF (from a farming or consumer perspective)

It is a mistake on our end. The definition of an SU is the first one: 1200GB of HDD and 300 GD od SSD resulting in 1000GB of net usable storage. The inner workings of the storage solution use “disk managers” to manage slices if disks throughout the TF Grid that manage the selected slices of disk space. These disk managers (container with 0-DB’s) use SSD’s to treat the spinning disk in the “nicest way possible” collecting and storing information in larger chunks (in a smart way) that typical disk block sizes.

The second definition is an old one and should have been deleted. Will do that. Apologies for the late response, will better my response times going forward :slight_smile:

1 Like

@weynandkuijpers thank you for the informations !

So if I understand correctly, if someone get 1 SU, the 300 GB SSD are not directly usable for container deployments ?

Instead this space is reserved for the 1 TB HDD usage, and used like a sort of cache ?

For the price depending of the actual consumption, how does this work exactly ? Because if I remember correctly, we are paying for the whole SU when we make the reservation ?

We combine the benefits of both types of storage to give consumers the best possible experience. Lower cost (although that is changing rapidly) HDD but slow with higher cost but fast) SSD. You can reserve SSD only, but then the price of 300GB os SSD is one SU.

The second part of the question: you can rent fractions of SU (theoretically down to MB’s).

These disk managers (container with 0-DB’s) use SSD’s to treat the spinning disk in the “nicest way possible” collecting and storing information in larger chunks (in a smart way) that typical disk block sizes.

Is there a technical documentation which go through this algorithm ? I’d like some more precise answers than “nicest way possible” and “in a smart way” if you don’t mind :slight_smile: I think I get how works most of the TF solution but the storage part is like a a big mistery for me.

Right now, If I’m using solutions like Nutanix for work and I got confidence in it, it’s because I read all of the Nutanix Bible ! If you could provide something similar to this which goes in deep technical details I’m sure it would help people gain trust for TF :slight_smile:

And I’m sorry if I don’t get your answers but I’m still not sure what is the behavior if I buy one SU :

  • I got 1 TB HDD and 300 GB SSD are used by 0-DB containers for accessing these 1 TB HDD (I hope it doesn’t work like that :D, but I’d like to know which percentage of SSD space is used for this TB HDD)
  • I got 1 TB HDD OR 300 GB SSD and the first that is depleted consumes my SU
  • If I consume only 500 GB of HDD for one month but I paid one SU for a month, will I have another month of my 500 GB used space ?

I’m sure a detailed practical example would help me a lot to understand if you don’t mind :slight_smile:

Ok, so essentially, we work with the concept of SU and CU. Whenever you want to deploy a workload, you first need a pool. A pool contains an amount of CU and SU you can use. Essentially, the pool represents your purchased capacity.

When you deploy a workload, we calculate how much CU and SU this workload consumes per second. Once the node reports the workload is deployed, this amount will be drained from the pool every second, until the pool is empty, or the workload is deleted.

Exactly how 1 SU (or CU for that matter) is defined, is depending on the user’s configuration. Basically a SU is a combination of both HDD and SSD (though SSD counts for more than HDD). Should you only require one of the 2, then that is of course possible. In this way, some types of workloads allow you to mix and match. For instance, containers allow you to specify exactly how much cores and memory you want, and depending on that, the amount of CU used is calculated.

Right now, the payment is done based on the reserved amounts. I.e if you reserve 10 SU worth of disk space, 10 SU will be deducted from your pool every second, even if you only use 1 SU.

Formula wise - this is (currently) how unit consumption are being calculated:

cu = round(min(MRU/4, CRU/2)*1000) / 1000
su = round((HRU/1200+SRU/300)*1000) / 1000

The reserved raw units are directly usable and addressable.

Whoops, looks like I linked to an issue on a private repository on Github… I’ve removed that link to avoid any future confusion. Thanks for pointing that out :slight_smile:

I’m hoping that things are more clear with Weynand’s input. Also hoping that I can get some testing in soon to give some more concrete examples of how this is currently operating in practice.