Twins, and node twins
On ThreeFold’s Grid version 3, we have the concept of a twin on the TF Chain blockchain. Every farmer and user of the Grid has a twin, which provides a very basic form of identity for determining who owns farms and who owns workloads. Each twin is linked to exactly one TF Chain wallet address, so it can also be thought of as an account for the purposes of holding and using TFT.
Besides controlling TFT, farms, and contracts, twins are also used as communication endpoints. A twin is assigned a unique twin id when it is created. Using the Reliable Message Bus, any twin can attempt to send messages to any other twin on the Grid, by referencing its twin id. Since this part is optional, not every twin will actually receive or respond to messages over RMB. However, for nodes this is essential—to be able to run workloads a node needs to receive instructions over RMB.
Here we run into a common point of confusion, especially for farmers. Each node on the Grid is associated with two twin ids. One is the twin that owns the farm the node belongs to and another is the twin of the node itself. Let’s look at a quick example. Here’s one of my nodes on testnet, as seen in the dashboard:
This node has a twin id of 831, and it belongs to farm id 42. Let’s check the farm:
The farm belongs to twin id 142. At first glance, we might think something is wrong here. I know my twin id is 142, but this node has a twin id of 831! Armed with the knowledge that each node also has its own twin, we can understand that this is normal. Each node needs a unique twin id to send and receive messages over RMB, and the Grid couldn’t work without this arrangement.
Node wallets
You might have also noticed that each twin shown in the screenshots above has its own account id too. I mentioned before that each twin has a linked account on TF Chain, and node twins are no exception. This means that the node has its own wallet, and as it turns out, that’s essential to the function of the Grid too.
Nodes need to execute transactions on TF Chain as part of their normal operations. When a node first boots up, it will create its node object on chain containing its node id and the farm it belongs to (based on the farm id specified in the bootstrap media used to boot the node). If a node is moved to a different farm, it must submit a transaction to update its farm id. Nodes also submit their uptime reports, as well as some information about their physical location and configuration to TF Chain.
So it’s important that nodes be able to execute on chain transactions. In the course of normal operations, TF Chain actually refunds any transaction fees paid by nodes, so that they can continue operation without needing a TFT top up. Nodes just get funded with a very small amount of TFT when they are created and farmers should never need to worry about this.
What could go wrong?
Well, it turns out that node wallets have sometimes run out of TFT due to issues in the system. The first hint is that error messages appear on the node’s console, something like: failed to submit extrinsic. In this case, an extrinsic is just a type of transaction, and it’s failing because there’s not enough funds to pay for the transaction fee.
Knowing how to check a node’s wallet balance is handy trick to have in the bag for troubleshooting. Let’s see how to do that now. Back on the dashboard, here’s a more complete view on how to find a node’s wallet address:
Use the filters if necessary to find the node, then select its row to see details. Here, for example, is node id 10:
Now copy the account id under Node Twin Details (in this case it’s 5CJDoxjwCBw5UwdyYZeVp6uqenfLSog5gQqvJUkqBBKHPXZP), and head to the Polkadot UI for TF Chain. The simplest way I’ve found to view another account’s balance is to use the Address Book, click here:
Then use the Add Contact button in the upper right hand corner of the screen, and fill in the node’s wallet address from earlier in the pop up menu:
Once you hit Save, you’ll see the balance:
This node has almost 1 TFT, which is good. Newer nodes only get funded with .1 TFT, so this is also okay:
If the node has a balance smaller than this, it could indicate an issue. You can contact our support team or open an issue on the Zos repository in that case for further investigation.
Wrapping up
In this post we covered some of the details about how nodes on the ThreeFold Grid use a twin and that twin’s associated wallet in the course of operation. I hope that reading this post can help to alleviate some confusion about the role of twins and provide some insight into how the node’s twin can be useful to reference when troubleshooting nodes.
Any questions? Drop it in a reply and we’ll take it from there 