Hey guys,
I just want to give you some news on the FAQ Bot.
The Threefold FAQ Bot is Alive
The Simple Bot
How to Use It
The Threefold FAQ Bot is now deployed here: https://t.me/TF_FAQ_Bot
There are few commands:
-
/start
-
start or restart the bot
-
/ask
-
ask a question to the bot
-
/more
-
once you asked a question and got answered, you can ask to see more related questions (3 new questions at a time)
On both cellphone and computer, you can either click on the command as presented by the bot, or you can write the command (/start, /ask, /more).
The Threefold Grid Data and the Bot Architecture
The Threefold FAQ Bot is currently running on test net, on the node 20.
It is using an IPv4 address. It is running on a full virtual machine (vm) and it uses Ubuntu 22.04.
The code is written in python and the main AI/ML library is the almighty sentence-transformers.
First the bot scraps the raw FAQ version on the Github page of the Threefold manual.
Then, it presents it to the AI/ML.
The AI/ML ranks from best to worst the most related Q&As compared to the question asked by the user. Once it has the ordered list of Q&A, the bot gives the answer in markdown through the telegram chat.
Limitations of the Bot
The facts: AI/ML workloads are most easily done by GPUs (graphical processing units).
In short (and in a simplified presentation), natural language processing language turns words into tokens (part/segment of words) and those tokens are turned into numbers. Then the computer compares numbers and gives answers. The answers are converted back into letters or the related words are presented.
Since AI/Ml takes a lot of workload, and that it needs to compute a lot of things in parallel, the more cores there is, the more efficient the AI/ML is.
For this reason, GPU are preferred over CPU (central processing unit). Even if CPU are in essence faster, GPU can compute a greater quantity of work in parallel.
If you want to, picture this: CPU can calculate quickly lines of numbers, while GPU can calculate more slowly mountains of numbers. The mathematics behind AI/ML is that it uses tensors. And tensors are very cool. They are basically matrices at each point of a mathematical grid.
Imagine 10 runners running 10 km per hours. How many kms are they running within an hour? 10 * 10 = 100 kilometers. That would be the CPU.
Imagine now 50 runners running 5 km per hours. How many kms are they running within an hour? 50 * 5 = 250 kilometers. That would be the GPU. Each runner runs slower, but the total distance made is greater.
In essence, GPU can do more different calculations, but those calculations are slower.
So a CPU is good to compute quickly some calculations. But to compute lots of data, GPU will do better.
In short, GPU would compute the answers faster, but it is not yet available on the Threefold Grid. This current version of the bot is thus running on CPU.
Also, note that there are now TPU: tensor processing unit. Those are made specifically for AI/ML compute workload.
Enough nerdy stuff for now. Let us test and let us see how the bot fares.
Basic Tests for the FAQ Bot
Here are some basic tests to show the speed of the bot. More tests would be clearly more precise, but the results are significant enough and in line with the literature to settle the score.
We present here two different deployments of a full virtual machine deployed on the Threefold Grid test net.
Note that in both cases, the bot used around 7.2gb of storage. The minimum storage is 15gb with a full VM, so this parameter was constant.
Let’s go!
4vcores, 8gb of ram, 15gb of storage
-
The first tested deployment was with 4vcores and 8gb of ram.
-
It used 2.4gb of ram maximum and the vcores were maxed out pretty good (all 95%+ during peaks).
-
One question got answered in 2m12s in average (n=2, just basic testing).
-
On test net, without yet any large amount of TFT to get 60% discount, it was 0.944 TFT/hour to run with some Internet traffic.
8vcores, 4gb of ram, 15gb of storage
-
The second tested deployment was with 8vcores and 4gb of ram.
-
It still uses around 2.4gb of ram and the vcores are still maxed out around 95% and more during peaks.
-
One question got answered in 1m12s (n=3).
-
Two questions, from two different Telegram accounts, got answered in 1m06s (n=1).
-
On test net, without TFT for the 60% discount, it is 1.136 TFT/hour to run with some Internet traffic.
The Brief Analysis
In short, doubling the vcores more or less doubles the speed. Even with two queries at at time, it took less than 1m20s. This is normal. AI/ML needs cores to do different calculations. The more cores, the quicker the bot.
For around 1 TFT/hour, anyone can run on the Threefold test net an AI/ML bot running on CPU, and you would count 2 TFT/hour on main net. Of course, the cost will increase as there is more Internet traffic with the bot.
To get more speed, you need more vcores. For 8vcores, you nearly double the speed, compared to 4vcores.
GPU and Future Testings
Of course, when GPU is implemented it will be very interesting to do some more testing here.
AI/ML is a huge use case and the market will demand a lot of this type of cloud service.
The Bot: Now and Later
We could have the FAQ bot as is, and then add more vcores if there is a need.
Of course, it depends on how much the Threefold community uses the Threefold FAQ Bot. We will check the pulse of the community in terms of interest in using the FAQ Bot.
With (e.g.) 10 TFT per hour, we could surely have a crazy fast FAQ Bot with a lot of vcores.
If we can have a FAQ bot that answers questions within 5-15 seconds, it would be a highly effective tool.
But still, waiting 1 minute to get an answer, that is convenient.
Also, note that what takes time is to compute the best answers in order.
So, once the calculations are done, the user can then ask to see three (3) more questions (/more on the bot), and 3 questions then appeared quickly. What takes time is really to take the user’s question and compare it with all the FAQ questions, then ranking them. Once this is done, it is very fast to print answers.
When GPUs are implemented on the Threefold Grid, the Threefold FAQ Bot would gain much more speed.
Closing Words
The Threefold FAQ Bot will evolve as the Threefold community gives feedback and as we make improvements with the bot.
You are invited to try it and tell us what you think.
Is this the first AI/ML bot deployed by the Threefold community?
Is it the last? Most certainly not, as AI/ML and the Threefold Grid go very well together.
The future of Threefold, with GPUs, is brighthly shining towards the AI/ML world.
In the meantime, we will explore this world gladly with CPUs.
Let us know what you think and tell us if you have ideas for future AI/ML tools for the Threefold community!
Note: A tutorial could be done on how to build such an AI/ML bot on the Threefold Grid. Let’s see if there is some interest first!
