Skip to main content
Version: NG-2.16

ClickHouse Monitor

Introduction​

The ClickHouse Monitoring O11ySource enables real-time collection of ClickHouse database metrics including queries, memory, connections, replication and storage performance via the Telegraf clickhouse input plugin.

Getting Started​

Compatibility​

ClickHouse version 22.x and above

Data Collection Method​

vuSmartMaps supports the monitoring of your ClickHouse via the Vunet internal collector deployed on the vuSmartMaps server. The Vunet internal collector connects to the ClickHouse HTTP interface (default port 8123, or 8443 for HTTPS/TLS) and queries system tables including system.metrics, system.events, system.asynchronous_metrics, system.replication_queue, system.tables, system.zookeeper, system.processes, system.dictionaries, system.mutations, system.disks, system.detached_parts, and system.text_log to collect server, query, connection, replication, and storage metrics.

Prerequisites​

Dependent Configuration​

To configure this O11ySource, create a 'credential' of type 'user' under the 'Definition' tab.

Inputs for Configuring Data Source​

  • Source Name: Unique name to identify this ClickHouse monitoring source
  • Server URLs: ClickHouse HTTP(s) endpoint URLs, one per line (e.g. http://host:8123 or https://host:8443). TLS is activated automatically when https:// is used.
  • Authentication Method: Select the Authentication Method
  • Credential: Credentials to access the storage Array endpoint
  • Query Timeout: HTTP timeout for metric collection including connection, redirects and response body (e.g. 5s, 10s)
  • Server Variant: Set to "Managed Cloud" for ClickHouse Cloud instances — this skips queries on tables not available in cloud environments (e.g. Zookeeper metrics)
  • Auto Discover Cluster Nodes: When enabled, Telegraf queries system.clusters to auto-discover and connect to all nodes in the cluster using the same credentials
  • Cluster Filter (Include): Comma-separated list of cluster names to include when auto-discovery is enabled. Leave blank to discover all clusters.
  • Cluster Filter (Exclude): Comma-separated list of cluster names to exclude when auto-discovery is enabled.

Firewall Requirement​

To collect data from this O11ySource, ensure the following ports are opened:

Source IPDestination IPDestination PortProtocolDirection
vuSmartMaps IPClickHouse server IP8123TCPOutbound
vuSmartMaps IPClickHouse server IP8443TCPOutbound

*Before providing the firewall requirements, please update the port based on the customer environment.

Configuring the Target​

Before configuring the O11ySource, ensure that:

The ClickHouse HTTP interface is enabled and accessible on port 8123 (or 8443 for HTTPS). Check http_port (or https_port) in config.xml or config.d/.

A dedicated monitoring user is created with read-only access to system tables: CREATE USER vunet_collector IDENTIFIED WITH sha256_password BY '' SETTINGS PROFILE 'readonly'; GRANT SELECT ON system.* TO vunet_collector;

If HTTPS is used, ensure the ClickHouse server certificate is signed by a CA trusted by the OS on the vuSmartMaps host.

Configuration Steps​

  • Enable the O11ySource.
  • Select the sources tab and press Add Source.
  • Enter a unique Source Name.
  • Enter one or more Server URLs (one per line, e.g. http://192.168.1.10:8123 or https://192.168.1.10:8443). TLS is activated automatically when https:// is used.
  • Select the Authentication Method (no_authentication or basic).
  • If basic auth is selected, choose the stored Credential.
  • Set the Query Timeout (default 5s). Format: number followed by 's', e.g. 10s.
  • Select the Server Variant: Self-hosted (default) for on-premises deployments, or Managed Cloud for ClickHouse Cloud instances (this skips queries on tables unavailable in cloud environments such as Zookeeper metrics).
  • Set Auto Discover Cluster Nodes to Yes to enable automatic node discovery via system.clusters using the same credentials.
  • If auto-discovery is enabled, optionally enter Cluster Filter (Include) and/or Cluster Filter (Exclude) as comma-separated cluster names to control which clusters are discovered.
  • Click Save to begin monitoring.

Metrics Collected​

NameDescriptionData Type
hostHost that collected the metricLowCardinality(String)
nameMetric measurement nameLowCardinality(String)
sourceSource of the metric (e.g. system.events)String
targetTarget ClickHouse instance being monitoredString
kafka_topicKafka topic the metric was received fromLowCardinality(String)
timestampDetailed timestamp with millisecondsDateTime64(3)
queryNumber of queries to be interpreted and potentially executedUInt64
select_queryNumber of SELECT queriesUInt64
insert_queryNumber of INSERT queriesUInt64
failed_queryNumber of failed queriesUInt64
failed_select_queryNumber of failed SELECT queriesUInt64
failed_insert_queryNumber of failed INSERT queriesUInt64
inserted_rowsNumber of rows inserted into all tablesUInt64
inserted_bytesNumber of bytes (uncompressed) inserted into all tablesUInt64
selected_rowsNumber of rows selected from all tablesUInt64
selected_bytesNumber of bytes (uncompressed) selected from all tablesUInt64
network_receive_bytesTotal bytes received from the networkUInt64
network_send_bytesTotal bytes sent to the networkUInt64
memory_worker_runNumber of runs done by the memory workerUInt64
memory_worker_run_elapsed_microsecondsTotal time spent by the memory worker in microsecondsUInt64
query_time_microsecondsTotal time of all queries in microsecondsUInt64
select_query_time_microsecondsTotal time of SELECT queries in microsecondsUInt64
insert_query_time_microsecondsTotal time of INSERT queries in microsecondsUInt64
mergeNumber of launched background mergesUInt64
merge_total_millisecondsTotal time spent on background merges in millisecondsUInt64
disk_read_elapsed_microsecondsTotal time spent waiting for disk reads in microsecondsUInt64
disk_write_elapsed_microsecondsTotal time spent waiting for disk writes in microsecondsUInt64
os_read_bytesNumber of bytes read from disks or block devicesUInt64
os_write_bytesNumber of bytes written to disks or block devicesUInt64
zoo_keeper_transactionsNumber of ZooKeeper/Keeper operationsUInt64
zoo_keeper_wait_microsecondsTime spent waiting for ZooKeeper/Keeper responses in microsecondsUInt64
_raw_messageRaw message received from Kafka before parsingString
_errorParsing/ingestion error message, if anyString
typeMetric typeString
log_uuidUnique record identifier generated at insertUUID