Hi Farmers,
If you’re using the new Farmerbot and you set it up using one of our guides or the install script, we recommend you check that your Systemd unit files match the latest versions published this week and update them if necessary.
Why?
TLDR
Our Systemd unit file templates were not configured to ensure that the system continues trying to start up the bot in certain failure cases.
The gory details
While we included:
Restart=always
Systemd does not interpret this in the same common sense manner as, say, Docker (that is, to always restart the service). Rather, repeated rapid failures can cause a rate limit to be hit, after which Systemd simply stops trying to start the service.
The conditions for this to happen are relatively rare, but for example could happen if:
- A power outage brings down the Farmerbot
- When the power is restored, network connectivity is not available so the Farmerbot fails to fully start and enters a crash loop
Ask me how I know
Fix
To address this, all that’s required is to edit the Systemd unit file(s) for the Farmerbot and make Systemd aware of the changes with: systemctl daemon-reload
There’s two lines we’ll add to the file. First make sure that retries are done indefinitely:
[Unit]
StartLimitIntervalSec=0
And then also slow the rate of retrying a bit (mainly to cut down the log spam a bit if the bot gets into a crash loop):
[Service]
RestartSec=5
A complete example unit file is shown below. It’s also possible to use the setup script to update your unit file (even if you didn’t use the script originally). Skip ahead for more details on that.
Manually
To update the unit file manually, just open it in an editor:
sudo nano /etc/systemd/system/farmerbot.service
Add the two lines so that your files looks like this. Take care if you are copying and pasting that the paths to your config.yaml
and .env
files are correct:
[Unit]
Description=ThreeFold Farmerbot
StartLimitIntervalSec=0
[Service]
Restart=always
RestartSec=5
StandardOutput=append:/root/farmerbotfiles/farmerbot.log
StandardError=append:/root/farmerbotfiles/farmerbot.log
ExecStart=/usr/local/bin/farmerbot run -e /root/farmerbotfiles/.env -c /root/farmerbotfiles/config.yml -d
[Install]
WantedBy=multi-user.target
Save the file and then run:
sudo systemctl daemon-reload
You don’t need to restart the bot for this to take effect.
Script
The setup script has been updated to generate unit files with the specified changes. You can use it to overwrite your existing unit file without needing to complete the entire setup process again.
First, make sure you’re in the same folder with config.yaml
and .env
. For example:
cd /root/farmerbot
ls -a
# config.yaml .env
Now download the updated script (this link always points to the latest version):
# First remove the old version if it's present
rm farmerbot-setup.sh
# Download the latest version
wget https://gist.githubusercontent.com/scottyeager/b4e4504f037841da165e7f759976a86c/raw/farmerbot-setup.sh
When you run the script it will offer to update the bot for you, if applicable, and then offer to regenerate configuration files. Just skip the config files part unless you need to update something in your config.
After that it will offer to generate a unit file. Enter “y” at this step, and then again to overwrite the existing one.
Run the script like this:
sudo bash farmerbot-setup.sh
The script will take care of the daemon-reload
step. There’s no need to restart the bot if you only needed to update the unit file.
That’s all
With this change Systemd will continue trying to start up your Farmerbot until the end of time. Cheers, and happy farming