Skip to main content
This appendix walks you through the process for setting up a robust monitoring system for the Google EKM integration with CryptoHub by using Cloudprober, Prometheus, and Grafana. The ability to promptly detect and respond to potential issues is crucial in maintaining the integrity and reliability of your security infrastructure. Effective monitoring offers real-time visibility into system operations, ensuring anomalies are quickly identified and addressed, minimizing downtime, and maintaining optimal performance. Key metrics such as latency and response codes are at the heart of this monitoring process, providing vital insights into your systems’ health and efficiency. By actively monitoring these metrics, you can ensure that the Futurex CryptoHub operates at peak performance, delivering the security and reliability that your enterprise demands.

​
Google EKM tools

To achieve this level of monitoring, we leverage three powerful tools: Cloudprober, Prometheus, and Grafana, which are used in tandem to create a comprehensive monitoring and alerting system.

​
Cloudprober

Cloudprober is a monitoring software developed by Google that actively tests and measures the availability and performance of services. It operates on the principle of proactive probing, meaning it continuously checks and verifies whether various system components are functioning as expected. Cloudprober supports multiple protocols like HTTP, UDP, and ICMP, and is highly configurable, enabling you to define custom probes based on your specific monitoring needs.

​
Prometheus

Prometheus is an open-source monitoring system with a powerful time-series database at its core. It is particularly well-suited for capturing and analyzing metrics in a high-uptime environment. Prometheus collects and stores metrics as time series data, meaning each metric is stored along with its timestamp and optional key-value pairs called labels. Its query language, PromQL, enables you to select and aggregate data, offering significant flexibility for viewing and analyzing data.

​
Grafana

Grafana is a popular open-source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources, the most common of which are time series databases like Prometheus. Grafana has powerful and easy-to-use dashboarding capabilities that enable you to create and share various types of visualizations to better understand complex data.

​
Integration of Cloudprober, Prometheus, and Grafana

You can use these three tools together to create a comprehensive monitoring and alerting system. You can configure Cloudprober to check the health of various system components continuously. Prometheus scrapes an endpoint that Cloudprober exposes to share the metrics it collects and then stores and processes this data, offering insights and the ability to trigger alerts based on predefined conditions. Grafana integrates seamlessly with Prometheus to visualize this data. This combination enables proactive monitoring (with Cloudprober), data storage and alerting (through Prometheus), and sophisticated data visualization (by using Grafana), making it an ideal setup for maintaining system health and performance.

​
Set up Cloudprober

To set up Cloudprober, perform the following tasks:
  1. Install Cloudprober.
  2. Configure Cloudprober to monitor a CryptoHub device.
  3. Test and verify the Cloudprober configuration.

​
Install Cloudprober

Refer to the Getting Started documentation on the official Cloudprober website for instructions on installing Cloudprober ( cloudprober.org/docs/overview/getting-started/).

​
Configure Cloudprober

After installing Cloudprober, you must configure it to monitor specific CryptoHub metrics, such as latency and response codes.
1
Create a configuration file: Cloudprober requires a configuration file (typically named cloudprober.cfg) to define what to monitor and how. Create this file in a directory of your choice.
2
Define probes for latency: In your configuration file, define aprobe block for latency. This instructs Cloudprober to periodically send requests to the CryptoHub and measure the time taken for a response. The following example shows a sample configuration:
None
probe {
    name: "google_ekm_latency"
    type: HTTP
    interval_msec: 5000 # Probe every 5 seconds
    timeout_msec: 5000 # 5 seconds timeout
    targets {
        host_names: "fx-ekm-load.cryptohub-demo.virtucrypt.com"
    }
    http_probe {
        relative_url: "/" # Replace with the specific path for the EKM status check
        protocol: HTTPS
        port: 443
    }
    latency_distribution {
        explicit_buckets: "50,150,1200,10000" # Buckets for latency distribution in milliseconds
    }
    latency_unit: "ms"
}
3
Configure probes for response codes: Similarly, add another probe block in your configuration file to monitor HTTP response codes. This helps in identifying error responses or any unexpected status codes.
None
probe {
    name: "data_center_network_connection"
    type: HTTP
    targets {
        host_names: "fx-ekm-load.cryptohub-demo.virtucrypt.com"
    }
    interval_msec: 60000 # Probe every 60 seconds
    timeout_msec: 5000 # 5 seconds timeout
    http_probe {
        relative_url: "/" # Path to probe without triggering crypto operation
        protocol: HTTPS
        port: 443
    }
    validator {
        name: "http_response_validator"
        http_validator {
            # Log any HTTP response, not just 200 or 400
            success_status_codes: "100-599"
        }
    }
}

surfacer {
    type: PROMETHEUS
}
4
Start Cloudprober: With your configuration file ready, start Cloudprober by using the following command:
Shell
cloudprober --config_file=cloudprober.cfg
Replace cloudprober.cfg with the path to your actual configuration file.
5
Monitor logs: Initially, keep an eye on the Cloudprober logs to ensure it operates as expected. You can find the logs in the default logging directory or the one specified in your configuration.

​
Test and verify the Cloudprober configuration

After Cloudprober is up and running, you should verify that it’s correctly monitoring the CryptoHub.
1
To check probing status, examine the Cloudprober admin interface or log files. Ensure that they are active and not reporting any errors.
2
Manually validate the latency and response code metrics collected by Cloudprober to ensure accuracy.
3
Adjust the configuration as needed. If you notice any discrepancies or want to modify the frequency of probes, adjust your configuration file accordingly and restart Cloudprober.
Following these steps enables you to successfully set up Cloudprober to monitor the performance metrics of CryptoHub and lays the foundation for a comprehensive monitoring setup.

​
Integrate Prometheus with Cloudprober

After setting up Cloudprober, the next critical step is to integrate it with Prometheus, which serves as the primary tool for storing and querying the collected monitoring data.

​
Download and install Prometheus

Perform the following steps to download and install Prometheus:
1
Visit the Prometheus download page (prometheus.io/download/) and download the current version of Prometheus for your operating system.
2
Unpack the downloaded file and run the Prometheus binary by executing ./prometheus in your command line in the directory where you unpacked Prometheus.

​
Configure Prometheus

Perform the following steps to configure Prometheus to scrape metrics from Cloudprober:
1
Prometheus uses a configuration file (usually named prometheus.yml) to define the sources from which it scrapes data. Open this file in a text editor.
2
Add Cloudprober as a scrape target in thescrape_configs section of the Prometheus configuration. The configuration looks similar to the following sample:
None
scrape_configs:
  - job_name: 'cloudprober'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9313']
3
In this configuration, localhost:9313 is the default address where Cloudprober exposes its metrics. Adjust this if you have configured Cloudprober differently.

​
Start Prometheus

Perform the following steps to start Prometheus:
1
Run the Prometheus binary with the configuration file:
Shell
./prometheus --config.file=prometheus.yml
2
Ensure that Prometheus is running without errors and can connect to Cloudprober.

​
Set up basic Prometheus alerts

To leverage Prometheus’s alerting capabilities, perform the following tasks:

​
Define alert rules

In the Prometheus configuration directory, create a file named alert.rules to define your alert conditions. For instance, you can set an alert for high latency or frequent error responses, similar to the following sample alert rule:
None
groups:
- name: cloudprober_alerts
  rules:
  - alert: HighLatency
    expr: probe_duration_seconds{job="cloudprober"} > 0.5
    for: 1m
    labels:
      severity: "critical"
    annotations:
      summary: "High latency detected in CryptoHub"

​
Add alert rules to the Prometheus configuration

Include the alert rules file in your prometheus.yml under the rule_files section:
None
rule_files:
  - "alert.rules"

​
Reload the Prometheus configuration

After updating the configuration, reload Prometheus to apply the changes. This can usually be done by sending a SIGHUP signal to the Prometheus process or by restarting Prometheus. These steps successfully integrate Prometheus with Cloudprober for advanced monitoring of the CryptoHub. This setup enables you to collect and store valuable performance data and also set up alerts for proactive monitoring of system health.

​
Visualize metrics with Grafana

Perform the following tasks to visualize the collected metrics with Grafana.

​
Install Grafana

Perform the following steps to install Grafana:
1
Visit the Grafana download page (grafana.com/grafana/download) and select the appropriate version for your operating system. Then, follow these installation instructions:grafana.com/docs/grafana/latest/
2
After youstart the Grafana server, access the web interface of the Grafana Dashboard, usually available athttp://localhost:3000.
3
Log in by using the default credentials (admin/admin) and change the password when prompted.

​
Integrate Grafana with Prometheus

Perform the following steps to integrate Grafana with Prometheus:
1
To add Prometheus as a data source, first, in the Grafana dashboard, go to Configuration (gear icon) and select Data Sources.
2
Then, select [ Add Data Source ], select Prometheus as the type, and configure the Prometheus server details. By default, the URL should behttp://localhost:9090, if it runs on the same machine.
3
To test and save the data source, ensure Grafana can connect to Prometheus by selecting**[ Save & Test ]**.
You should see a message confirming the data source is working.

​
Create dashboards

Perform the following steps to create dashboards for visualizing CryptoHub metrics:
1
To create a new dashboard in Grafana, select the**+** icon and select Dashboard. Then, select [ Add new panel ] to start customizing your first metric visualization.
2
Select Prometheus as the data source and use a query to display latency metrics, such as: probe_duration_seconds{job="cloudprober"}.Customize the panel with appropriate visualization types (such as graphs, gauges) and settings (such as time ranges, thresholds).
3
To design a response codes dashboard, perform the following steps:
  1. Add another panel to the dashboard or create a new dashboard specifically for response codes.
  2. Use a query that captures response code metrics, such as: probe_http_response_code{job="cloudprober"}.
  3. Configure the panel to display the data effectively, possibly using bar charts or tables.
4
After setting up the panels, save the dashboard and give it a meaningful name.You can share dashboards with team members or export them for use in other Grafana installations.