> For the complete documentation index, see [llms.txt](https://docs.stakelocal.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.stakelocal.io/dashboard-installation/enable-and-start-restart-services/checking-software-status.md).

# Checking Software Status

Monitoring the monitors

## Checking Software Status

### systemctl

The quick way to check the status of all of the supporting software is to run `systemctl status` on all of the services.

```bash
sudo systemctl status prometheus grafana-server node_exporter json_exporter eth-metrics
```

You will see output like the following.

<div><figure><img src="https://971382438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZON6xeJcH6NDhbzjDAWf%2Fuploads%2Fc5hlBQPvGWtSeu9CFuvv%2Fsystemctl-status-prometheus-grafana.png?alt=media&amp;token=1a508ed4-ff15-46db-8ab5-2b6185ef822c" alt=""><figcaption><p>Prometheus and Grafana status</p></figcaption></figure> <figure><img src="https://971382438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZON6xeJcH6NDhbzjDAWf%2Fuploads%2Fwujq8u7SDLe1Cxnetmco%2Fsystemctl-status-node_exporter-json_exporter.png?alt=media&amp;token=06c291f6-d57d-4668-900f-2b067842d3cb" alt=""><figcaption><p>node_exporter and json_exporter status</p></figcaption></figure> <figure><img src="https://971382438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZON6xeJcH6NDhbzjDAWf%2Fuploads%2F13JWaLhedfW3RpdploAM%2Fsystemctl-status-eth-metrics.png?alt=media&amp;token=f800c4c5-56d0-453b-920b-c635ae5e83ec" alt=""><figcaption><p>Ethereum Metrics Exporter status</p></figcaption></figure></div>

If you would like to see the logs individually for the supporting software, use the following commands to see the last 100 lines of log files.

```
sudo journalctl -u prometheus -n 100
sudo journalctl -u grafana-server -n 100
sudo journalctl -u node_exporter -n 100
sudo journalctl -u json_exporter -n 100
sudo journalctl -u eth-metrics -n 100
```

Replace "100" in the commands above with any number of lines you would like to see.

To see a never-ending stream of logs, use these commands, and use CTRL-C to exit.

```bash
sudo journalctl -fu prometheus
sudo journalctl -fu grafana-server
sudo journalctl -fu node_exporter
sudo journalctl -fu json_exporter
sudo journalctl -fu eth-metrics
```

Healthy Startups

Here is what a healthy start looks like in the logs for the supporting software.

### Logs

#### Prometheus

<figure><img src="https://971382438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZON6xeJcH6NDhbzjDAWf%2Fuploads%2FWC9RSRJj2CHauQQ7SH4g%2Fhealthy-prometheus.png?alt=media&amp;token=beaab1fb-ca99-45a7-b1a8-6e75f122e0a4" alt=""><figcaption><p>Logs for a healthy Prometheus start</p></figcaption></figure>

#### Grafana

<figure><img src="https://971382438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZON6xeJcH6NDhbzjDAWf%2Fuploads%2FulVJ7KGyFJs5Cbd0LEmE%2Fhealthy-grafana.png?alt=media&amp;token=d18bf5d5-3468-41d2-a1bd-4c069a08447c" alt=""><figcaption><p>Logs for a healthy Grafana start</p></figcaption></figure>

#### node\_exporter

<figure><img src="https://971382438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZON6xeJcH6NDhbzjDAWf%2Fuploads%2FrD8IL4cXHGP6uyuhSwPs%2Fhealthy-node_exporter.png?alt=media&amp;token=6c9ae3a4-3bd9-4871-b306-8da92052a900" alt=""><figcaption><p>Logs for a healthy node_exporter start</p></figcaption></figure>

#### json\_exporter

<figure><img src="https://971382438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZON6xeJcH6NDhbzjDAWf%2Fuploads%2Fcp0Bg9jrRQc6teVsqPsj%2Fhealthy-json_exporter.png?alt=media&amp;token=28795943-3d4e-4191-96fd-c0ef164358bf" alt=""><figcaption><p>Logs for a healthy json_exporter start</p></figcaption></figure>

#### Ethereum Metrics Exporter

<figure><img src="https://971382438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZON6xeJcH6NDhbzjDAWf%2Fuploads%2FYLie4kh1hHBGsEkcOFXR%2Fhealthy-eth-metrics.png?alt=media&amp;token=db161d31-14c4-4de1-9a47-5108b4eced3e" alt=""><figcaption><p>Logs for a healthy Ethereum Metrics Exporter start</p></figcaption></figure>

Prometheus Targets

One way to check if your supporting software is running is to see whether Prometheus is able to scrape the software.&#x20;

Go to `http://127.0.0.1:9090`, replacing `127.0.0.1` with the IP address at which Prometheus is running.

At the top of the screen you should see a Status menu. Hover over that menu and click on the Targets link.

<figure><img src="https://971382438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZON6xeJcH6NDhbzjDAWf%2Fuploads%2FXsb1xs6djObGjCK1Qlwv%2Fprometheus-ui-status-menu.png?alt=media&amp;token=cab83d7b-4630-4546-97d9-95dfcccb02bb" alt=""><figcaption><p>Find the Targets link under the Status menu</p></figcaption></figure>

The Targets screen will appear listing all of the Prometheus targets we previously set up.

You may need click Clear Selection in the targets filter in the upper-left corner of the screen.

<figure><img src="https://971382438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZON6xeJcH6NDhbzjDAWf%2Fuploads%2FEsMTj2jtRVi0JLqzwtEQ%2Fprometheus-ui-clear-selection.png?alt=media&amp;token=20178034-a89b-4bd4-98b4-ff61167b0743" alt=""><figcaption><p>Clear the selection to see all Prometheus targets</p></figcaption></figure>

Each line of blue or red text represents a single Prometheus job serving one or more targets.

* Blue means Prometheus has successfully scraped data recently
* Red means Prometheus has not successfully scraped data recently from one

The numbers in parentheses show how many of the targets within each group Prometheus has successfully scraped recently.

<figure><img src="https://971382438-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZON6xeJcH6NDhbzjDAWf%2Fuploads%2FvyyFDazaQF8cq6mBdUNP%2Fprometheus-ui-targets.png?alt=media&amp;token=02beea6e-868b-40d7-bac6-c0edb41ce03d" alt=""><figcaption><p>The Targets screen showing the status of various Prometheus jobs</p></figcaption></figure>

{% hint style="warning" %}
If Prometheus has not had any data from a target within the last 15 minutes, it will show that target in the color red on this screen. Two of the jobs will often show a red color on this screen:

* consensus\_beacon\_genesis - This job runs every few hours, because the genesis information never changes. Therefore, Prometheus often shows this job in red.
* github\_latest\_release - This job runs every few hours, because GitHub has a low rate limit. Therefore, Prometheus often shows this job in red. Additionally, it may take a couple hours after a new release for that release to be reflected on the Prometheus Targets screen.

The last job that may be red is blackbox\_exporter. That job is not presently covered in these instructions.
{% endhint %}
