Checking Software Status
Monitoring the monitors
Checking Software Status
systemctl
The quick way to check the status of all of the supporting software is to run systemctl status on all of the services.
sudo systemctl status prometheus grafana-server node_exporter json_exporter eth-metricsYou will see output like the following.
If you would like to see the logs individually for the supporting software, use the following commands to see the last 100 lines of log files.
sudo journalctl -u prometheus -n 100
sudo journalctl -u grafana-server -n 100
sudo journalctl -u node_exporter -n 100
sudo journalctl -u json_exporter -n 100
sudo journalctl -u eth-metrics -n 100Replace "100" in the commands above with any number of lines you would like to see.
To see a never-ending stream of logs, use these commands, and use CTRL-C to exit.
sudo journalctl -fu prometheus
sudo journalctl -fu grafana-server
sudo journalctl -fu node_exporter
sudo journalctl -fu json_exporter
sudo journalctl -fu eth-metricsHealthy Startups
Here is what a healthy start looks like in the logs for the supporting software.
Logs
Prometheus
Grafana
node_exporter
json_exporter
Ethereum Metrics Exporter
Prometheus Targets
One way to check if your supporting software is running is to see whether Prometheus is able to scrape the software.
Go to http://127.0.0.1:9090, replacing 127.0.0.1 with the IP address at which Prometheus is running.
At the top of the screen you should see a Status menu. Hover over that menu and click on the Targets link.
The Targets screen will appear listing all of the Prometheus targets we previously set up.
You may need click Clear Selection in the targets filter in the upper-left corner of the screen.
Each line of blue or red text represents a single Prometheus job serving one or more targets.
Blue means Prometheus has successfully scraped data recently
Red means Prometheus has not successfully scraped data recently from one
The numbers in parentheses show how many of the targets within each group Prometheus has successfully scraped recently.
If Prometheus has not had any data from a target within the last 15 minutes, it will show that target in the color red on this screen. Two of the jobs will often show a red color on this screen:
consensus_beacon_genesis - This job runs every few hours, because the genesis information never changes. Therefore, Prometheus often shows this job in red.
github_latest_release - This job runs every few hours, because GitHub has a low rate limit. Therefore, Prometheus often shows this job in red. Additionally, it may take a couple hours after a new release for that release to be reflected on the Prometheus Targets screen.
The last job that may be red is blackbox_exporter. That job is not presently covered in these instructions.
Last updated