GRANT SUGGESTION: Deploy a HYDRA node on TFGrid

[joris]

Present: Oele, Mik-idrnd, Joris
Date: 07/ april 2022 (5 - 6h20)
Purpose: Poc run hydra node on TF grid
Result of meeting:

• First goal: proof that Hydra node can run on the TF grid

• How: running hydra node on VM (Oele) and produce documentation (Mik)

• Question

  • what are minumum VM requirements knowing that hydra node runs on Raspberry Pi node
  • how much TFT would be needed to run a VM with the recommended hardware config for having a Hydra node for 1 day

• Resources needed:

  • TFT to run VM on testgrid (Joris)

Next steps

• FUP meeting (between 18/4 - 22/4) Doodle to setup (Joris)
• evaluate poc
• evaluate more user friendly way on seting up hydra nodes for enduser
  => create weblet where user can enter his own configuration and hydra node is automatically deployed

[oele]

Helpfull info (for my info - i’ll write an extensive how to after my POC):

Question: Do we need an IPv4 or is a Planetary IP enough?

How to setup a VM

From Zero to Hero: Deploying a Virtual Machine on TF Grid 3 in under 10 minutes!

I wanted to share this demo done by @weynandkuijpers where he attempts to go from “zero” (nothing installed) to deploying a Virtual Machine (VM) on the ThreeFold Grid in under 10 minutes. Will he make it?! There’s only one way to find out: For our technical audience, you’ll already be able to understand what is happening throughout the video and you already know what a VM is and why this is so cool. But for others, here is a little bit more context: A VM is no different than any o…

Guide for Linux

Documentation that needs to be created:

• How to install the Threefold Wallet App on android
• How to generate a private/public key on Linux

Currently struggling with Disk Quota that is being exceeded:

Strange because on each drive I still have enough space.

I think it may be related to the reduced ThreeFold ubuntu image, but I’m not sure.

[Mik]

Maybe I’m crazy, but would this work?

Take the Hydra docker image

Docker image for Hydra Node

Use the Docker Hub Converter of Threefold to turn the docker image into flist

threefoldfoundation/info_manual3/blob/development/wiki/flist_hub/flist_docker_convert.md

Looks almost too easy. There must be a catch!

---

Then,

We can follow these steps to get the flist on play.grid.tf:
https://github.com/threefoldtech/publish-flist

“This step can be used to pack your stuff and upload the resulting flist into the hub (playground).”

Then we could have it on play.grid.tf

What do you think?

[oele]

I’ll need to test this.
If using docker, maybe additional steps are required for ssh access in order to be able to execute all the command line options.

How to SSH into a Running Docker Container and Run Commands

[weynandkuijpers]

@oele. Nice try. I have looked at the install of the hydra node in Linux and it does contain a large number of steps. I think the docker-image present a simpler solutions. Two options with that:

• the first option is to create a VM and install Docker into the VM. It’s less efficient but does get you up and running in no time with the docker image.
• second option is what @Mik suggested - transponse the dockerimage into an flist. This is also easy and straightforward.

Let’s try the second option first:

1. import the docker image (easy in the HTTP://hub.grid.tf).
   
   When done -> this is the result;
   

2. So the URL to start the machine with is: https://hub.grid.tf/weynandkuijpers.3bot/hydra.flist

3. go to play.grid.tf: and use the deploy virtual machine with a custom list:
   

This will deploy the hydra docker image (+ I added SSH access, so if you have set your SSH_KEY right in the portal, you will be able to ssh in).

I did not check if the hydrad is running as expected. It might need some tweaks:

➜  hydra git:(master) ✗ ssh root@185.69.166.156
Welcome to Ubuntu 20.04.2 LTS (GNU/Linux 5.12.9 x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

This system has been minimized by removing packages and content that are
not required on a system that users do not log into.

To restore this content, you can run the 'unminimize' command.
Last login: Mon Apr 11 12:02:47 2022 from 2.51.127.183
root@hydranoe:~# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 11:57 ?        00:00:01 /bin/bash /root/Hydra/bin/run-hydrad.sh
root         2     0  0 11:57 ?        00:00:00 [kthreadd]
root         3     2  0 11:57 ?        00:00:00 [rcu_gp]
root         4     2  0 11:57 ?        00:00:00 [rcu_par_gp]
root         5     2  0 11:57 ?        00:00:00 [kworker/0:0-events]
root         6     2  0 11:57 ?        00:00:00 [kworker/0:0H-events_highpri]
root         7     2  0 11:57 ?        00:00:00 [kworker/u4:0-events_unbound]
root         8     2  0 11:57 ?        00:00:00 [mm_percpu_wq]
root         9     2  0 11:57 ?        00:00:00 [rcu_tasks_kthre]
root        10     2  0 11:57 ?        00:00:00 [rcu_tasks_rude_]
root        11     2  0 11:57 ?        00:00:00 [rcu_tasks_trace]
root        12     2  0 11:57 ?        00:00:00 [ksoftirqd/0]
root        13     2  0 11:57 ?        00:00:00 [rcuc/0]
root        14     2  0 11:57 ?        00:00:00 [rcu_preempt]
root        15     2  0 11:57 ?        00:00:00 [rcub/0]
root        16     2  0 11:57 ?        00:00:00 [migration/0]
root        17     2  0 11:57 ?        00:00:00 [idle_inject/0]
root        18     2  0 11:57 ?        00:00:00 [kworker/0:1-ipv6_addrconf]
root        19     2  0 11:57 ?        00:00:00 [cpuhp/0]
root        20     2  0 11:57 ?        00:00:00 [cpuhp/1]
root        21     2  0 11:57 ?        00:00:00 [idle_inject/1]
root        22     2  0 11:57 ?        00:00:00 [migration/1]
root        23     2  0 11:57 ?        00:00:00 [rcuc/1]
root        24     2  0 11:57 ?        00:00:00 [ksoftirqd/1]
root        25     2  0 11:57 ?        00:00:00 [kworker/1:0-rcu_gp]
root        26     2  0 11:57 ?        00:00:00 [kworker/1:0H-events_highpri]
root        27     2  0 11:57 ?        00:00:00 [kdevtmpfs]
root        28     2  0 11:57 ?        00:00:00 [netns]
root        29     2  0 11:57 ?        00:00:00 [inet_frag_wq]
root        30     2  0 11:57 ?        00:00:00 [kauditd]
root        31     2  0 11:57 ?        00:00:00 [kworker/1:1-events]
root        32     2  0 11:57 ?        00:00:00 [khungtaskd]
root        33     2  0 11:57 ?        00:00:00 [oom_reaper]
root        34     2  0 11:57 ?        00:00:00 [writeback]
root        35     2  0 11:57 ?        00:00:00 [kcompactd0]
root        36     2  0 11:57 ?        00:00:00 [ksmd]
root        37     2  0 11:57 ?        00:00:00 [khugepaged]
root        54     2  0 11:57 ?        00:00:00 [cryptd]
root        64     2  0 11:57 ?        00:00:00 [kintegrityd]
root        65     2  0 11:57 ?        00:00:00 [kblockd]
root        66     2  0 11:57 ?        00:00:00 [blkcg_punt_bio]
root        67     2  0 11:57 ?        00:00:00 [md]
root        68     2  0 11:57 ?        00:00:00 [edac-poller]
root        69     2  0 11:57 ?        00:00:00 [devfreq_wq]
root        70     2  0 11:57 ?        00:00:00 [watchdogd]
root        72     2  0 11:57 ?        00:00:00 [kworker/0:1H]
root        75     2  0 11:57 ?        00:00:00 [kswapd0]
root        77     2  0 11:57 ?        00:00:00 [kthrotld]
root        78     2  0 11:57 ?        00:00:00 [irq/1-ACPI:Ged]
root        79     2  0 11:57 ?        00:00:00 [acpi_thermal_pm]
root        80     2  0 11:57 ?        00:00:00 [hwrng]
root        81     2  0 11:57 ?        00:00:00 [drbd-reissue]
root        82     2  0 11:57 ?        00:00:00 [rbd]
root        83     2  0 11:57 ?        00:00:00 [raid5wq]
root        84     2  0 11:57 ?        00:00:00 [bch_btree_io]
root        85     2  0 11:57 ?        00:00:00 [bcache]
root        86     2  0 11:57 ?        00:00:00 [bch_journal]
root        87     2  0 11:57 ?        00:00:00 [dm_bufio_cache]
root        88     2  0 11:57 ?        00:00:00 [kmpathd]
root        89     2  0 11:57 ?        00:00:00 [kmpath_handlerd]
root        90     2  0 11:57 ?        00:00:00 [kworker/1:1H]
root        91     2  0 11:57 ?        00:00:00 [ipv6_addrconf]
root        96     2  0 11:57 ?        00:00:00 [kstrp]
root        97     2  0 11:57 ?        00:00:00 [ceph-msgr]
root       113     2  0 11:57 ?        00:00:00 [zswap1]
root       114     2  0 11:57 ?        00:00:00 [zswap1]
root       115     2  0 11:57 ?        00:00:00 [zswap-shrink]
root       116     2  0 11:57 ?        00:00:00 [kworker/u5:0]
root       204     1  0 11:57 ?        00:00:00 sshd: /usr/sbin/sshd [listener] 0 of 10-100 startups
root       235     2  0 12:00 ?        00:00:00 [kworker/u4:2-events_unbound]
root       278     2  0 12:07 ?        00:00:00 [kworker/u4:1-events_power_efficient]
root       286     1  0 12:10 ?        00:00:00 sleep 30
root       287   204  1 12:10 ?        00:00:00 sshd: root@pts/0
root       299   287  0 12:10 pts/0    00:00:00 -bash
root       308   299  0 12:10 pts/0    00:00:00 ps -ef
root@hydranoe:~# 

Details:

FList: https://hub.grid.tf/weynandkuijpers.3bot/hydra.flist
Entry Point: /root/Hydra/bin/run-hydrad.sh
Root File System (GB): 5
CPU: 2
Memory (MB): 2048

Happy hydra’ing!

[Mik]

Oh yeah! Amazing.

We got it! Simpler than we first thought. Long Live the Grid!
We’ll have to make sure everything is fine in the details and then make some documentation out of it.

It’ll be nice to have this as a one-click solution on play.grid.tf in the Deployment section.

Thanks @weynandkuijpers

Good work @oele and @joris. Let’s do it!

EDIT: Well once it’s available on play.grid.tf as a flist, it should be very easy to install for the Hydra community. Sweet.

[Mik]

@oele @joris

Hey guys!

Do you want to have a meeting soon to check what are the next steps?

Tell me what you think.

Have a good day

[joris]

@weynandkuijpers @oele, @trout2, @Mik
Here is a new doodle for a follow up meeting to see how far we got and what are next steps.…
https://doodle.com/meeting/participate/id/erk6802a
Feel free to forward

[oele]

Hi all,

This evening between 5pm and 6pm (Brussels Time) we’ll have another meeting via Google: meet.google.com/dkm-akft-wwv

Feel free to join.

We’d like to setup a Hydra node on the threefold grid before a new, so called delegated staking, will be released.

Delegated staking allows others to stake on your node, you’ll get a certain (configurable) percentage of the earnings. It’s to avoid staking on CEX’es.

See you this evening.

[joris]

Hydra Node on TF grid

available:

• flist containing hydra docker image on the hub.grid.tf

Action/demo by Oele

• start virtual machine/load hydra-docker flist - play.grid.tf
• config used here
  1. 2CPU
  2. 2 GB of ram
  3. 50 GB of SSD
• start ssh connection
• execute some hydra commands to test (test passed)
• no staking tested yet
• Current cost for hydra node on tf vm
  14-17$ per month + discounts of TFT provisioned for longer period

todo/next steps

1. find how much it costs in TFT for the Hydra node with optimal configuration (ask in hydra comunity what minimal requirements are)
2. find if the TF VM is optimal (can a virtual machine be upgraded once its deployed?)
3. test staking via command lines
4. ask hydra team to validate the hydra node runing on TF grid to see if all functionalities work
   => give access to hydra team member
5. make things easier:
   need to import the seedphrase of hydra (documentation for this)
   can this be done by importing environment variables entered in play.grid.tf and is this secure enough to contain seedphrase
   get env variables from the play.grid.tf page to hydra command lines, in order to have basic functionalities via environment variables, or via weblet
6. what if hydra node goes down? Possible causes

• if no more TFT on the 3node?
• hydra app crashes
• 3node goes down?
  1. telegram bot
  2. Caprover solution?

7. add a folder in github
   1. branch on threefold
      1. hydra folder
   2. give access to oele + joris + hydra team member to edit

Next meeting with Weynand (TF cofounder) See doodle
https://doodle.com/meeting/participate/id/axkrk43e

It’s Alive!! Hydra node is runing on the decentralized ThreeFold grid

[weynandkuijpers]

Sorry for not making it to the meeting last night - family obligations steered me away from the 19:00 (my time) meeting. Let’s make it work early next week. Impressed with the progress of the group/community

[oele]

Hello,

Let’s hope to meet a Hydra core member, that would be amazing.
I’ll try to update the Doodle today.

Enjoy the day!

[Mik]

It’s ok @weynandkuijpers! We do understand family obligations.

I hope you cooked something real good for your family so it was worth it :P.

Let’s hope we can all make it for the next meeting. Such a great project.

Threefold-Hydra will be an amazing collaboration.

What is your timezone, to be sure we set the next meeting at a proper time?

Thanks.

[hannahcordes]

Very impressive, guys! This is so cool to see!

[joris]

Meeting will be today at 4 PM CET/CEST

Meet

Everybody welcome to listen or to contribute

[weynandkuijpers]

Was a short (but great) meeting. WE have agreed to:

• ask some more in-depth questions to the Hydra community
• post a story in the forum that we ar helping hydray to escape from the google/AWS walled garden
• have a barebone hydra flist available, now need to work on how we can get to a single click deployment (and passing some vital data to the deployment).
• find out how the public staking model works for Hydra.

Looking forward to more updates!

[joris]

Just heared that Hydra does not have plans to bridge with Stellar or Polkadot Substrate.
Currently the only bridge they have is with Ethereum