Additional Staking Groups

Set up Prometheus to monitor multiple groups of consensus/execution clients

Additional Staking Groups

The Stake Local Dashboard can monitor multiple consensus/execution pairs at a time. Each consensus and execution client pair is assigned to a group. A new client group can be created as simply as configuring Prometheus targets for the new clients and assigning them a unique group label.

Configure Prometheus Targets

To start adding a new client: Create new consensus client targets, execution client targets, and validator client targets. Return to the these three previous sections and add a new targets sections to the client-specific configuration files.

• Consensus Client Targets

• Execution Client Targets

• Validator Client Target

Copy the existing targets in the files, or copy the default targets from this documentation.

Multiple targets sections can appear in the same file, but they should always be for the same client type.

For each new targets section created, modify the IP address, port, and label values.

IP Address and Port

The new client will have a unique IP address or port. Update the IP address and port of the new targets line to the correct IP address and port for the new client.

Labels

Labels help classify data from different client instances. Depending on your architecture, one or more of these labels may need to change for your new targets section.

See About Stake Local Labels to refresh to refresh your knowledge of how the Stake Local Dashboard uses Prometheus labels.

Below the labels line in the targets section for the new client:

• Update the network label for the new client. The network of the new client may/may not be the same as for the original client. Examples: Mainnet, Goerli

• Update the group label for the new client. The group of the new client is most likely a different from that of the original client. Examples: Mainnet Prysm/Geth, Goerli Teku/Besu, Group A

• Update the host label for the new client. The host of the new client may/may not be the same as for the original client. Examples: nuc, node.stakelocal.io, srv-lax-03932, trogdor

• Update the service label for the new client. The service label is used to distinguish individual instances of a client type. With only one instance of each type, you can keep it simple with "Nimbus" and "Nethermind", but with multiple instances of each type the service values should be unique. Examples: Mainnet Lodestar, Goerli Erigon,, Backup Lighthouse, srv-lax-03932-besu

After adding a second Besu client, one target file might look like the following.

    - targets: ['127.0.0.1:8545']
      labels:
        network: 'Mainnet'
        host: 'srv01'
        service: 'Mainnet Besu'
        group: 'Mainnet Teku/Besu'
        explorer: 'etherscan.io'
        json_exporter: '127.0.0.1:7979'
        client: 'Besu'
    - targets: ['192.168.0.50:8545']
      labels:
        network: 'Goerli'
        host: 'srv02'
        service: 'Goerli Besu'
        group: 'Goerli Teku/Besu'
        explorer: 'goerli.etherscan.io'
        json_exporter: '127.0.0.1:7979'
        client: 'Besu'

Prometheus targets are read in from files that follow a naming convention. All files that begin with the name of a client in lower case are read in by Prometheus and associated with jobs appropriate for that client.

The default file names provided in this guide use only the client name in the file name (e.g. teku.yml, nethermind.yml), but Prometheus will also read in files such as the following:

• prysm-mainnet.yml

• erigon-goerli.yml

• nimbus-srv-lax-039302.yml

The only requirement is that the name of the client be in lower case at the beginning of the filename.

This provides greater flexibility in organizing Prometheus target configurations.

Why so many target files in the first place? Not all clients support all API calls. Separating targets by client type allows us to configure API-specific Prometheus jobs to execute for only those clients that correctly support the end point.

Configure Ethereum Exporter

Ethereum Metrics Exporter can only support a single consensus/execution client pair. For full dashboard support of a new pair, a new instance of Ethereum Metrics Exporter must be run.

To add another Ethereum Metrics Exporter instance, you must either run Ethereum Metrics Exporter on a separate system from your first instance, or you must configure Ethereum Metrics Exporter to run more than once on the same system under a different port.

New Instance on Separate System

To install a new instance of Ethereum Metrics Exporter on a separate system, follow all of the Ethereum Metrics Exporter instructions on the new system.

1. Install Ethereum Metrics Exporter on the new system.

2. Configure Ethereum Metrics Exporter on the new system.

3. Configure the Prometheus target for the new Ethereum Metrics Exporter.

4. Enable and start the Ethereum Metrics Exporter software on the new system.

5. Add a new Ethereum Metrics Exporter target to your eth-metrics.yml file in /etc/prometheus/files_sd/stakelocal/other/ directory. Modify the IP address, port, and labels, as necessary.

You do not need to restart Prometheus. Prometheus will pick up the changes shortly after you save the target file.

Second Instance on Same System

To run a second instance of Ethereum Metrics Exporter on the same system:

1. Skip the installation steps, but follow the steps to Configure Startup for Ethereum Metrics Exporter from the Install Ethereum Metrics Exporter page.

   1. Modify the commands on that page to use a unique filename for the second instance of Ethereum Metrics Exporter. For example: eth-metrics-goerli.service, eth-metrics-backup.service

   2. Change the port in the ExecStart line to the port on which the new instance of Ethereum Metrics Exporter is running

   3. Change the name of the configuration file in the ExecStart line to a name unique to the new instance of Ethereum Metrics Exporter. For example: eth-metrics-goerli.yml, eth-metrics-backup.yml.

2. Repeat the steps to Configure Ethereum Metrics Exporter, using the configuration filename configured in the previous step. For example: eth-metrics-goerli.yml, eth-metrics-backup.yml

3. Repeat the steps to configure Ethereum Metrics Exporter Targets, creating a new targets section in the /etc/prometheus/files_sd/stakelocal/other/eth-metrics.yml file

4. Enable and start the new Ethereum Metrics Exporter service with sudo systemctl enable --now eth-metrics-goerli, using the name of the .service file created above.