AWS MSK

Introduction

Amazon MSK O11ySource enables monitoring of AWS-managed Kafka clusters by collecting key broker, topic, partition, consumer, and cluster health metrics. It helps teams track Kafka availability, performance, throughput, lag, and resource utilization from a centralized vuSmartMaps dashboard, supporting faster issue detection, troubleshooting, and operational visibility for real-time streaming workloads.

Getting Started

Compatibility

The MSK metricsets were tested and validated with Amazon Managed Streaming for Apache Kafka (MSK) clusters running Apache Kafka versions 4.0.x.

Data Collection Method

vuSmartMaps collects AWS MSK metrics through a data collector deployed within the vuSmartMaps platform, which uses AWS CloudWatch to retrieve cluster, broker, topic, and partition performance metrics.

Prerequisites

Dependent Configuration

To configure this O11ySource, create a 'credential' of type 'aws' under the 'Definition' tab.

Inputs for Configuring Data Source

• Data Source Name: Enter a unique name for this AWS MSK Source.
• AWS Region: AWS Region where the instance of this component is running. For eg: Asia Pacific (Mumbai), the region would be ap-south-1.
• AWS Credential: AWS credential that provides the access key and secret key for CloudWatch.
• Period (in minutes): Specifies the interval in minutes at which data is collected. Data collection occurs once every specified period. The period should be between 1 - 60 minutes.

Firewall Requirement

To collect data from this O11ySource, ensure the following ports are opened:

Source IP	Destination IP	Destination Port	Protocol	Direction
vuSmartMaps Collection/Ingress node IP	Service URL	443*	TCP	Outbound

*Before providing the firewall requirements, please update the port based on the customer environment.

Configuring the Target

Health and performance metrics of AWS MSK are collected through AWS CloudWatch service. Therefore, AWS CloudWatch monitoring must be enabled for your MSK cluster in the AWS account. An IAM role or user with the following permissions is required to access CloudWatch metrics and MSK cluster details: cloudwatch:GetMetricData cloudwatch:ListMetrics kafka:DescribeCluster kafka:ListClusters kafka:GetBootstrapBrokers tag:GetResources

Configuration Steps

• Enable the O11ySource.
• Select the Sources tab and press the + button to add a new AWS MSK cluster to be monitored.
• Populate all the configurations. Click on Save to create the Cluster.

Metrics Collected

Name	Description	Data Type
Cluster Name	Cluster name (name of the MSK cluster)	String
Time Stamp	Timestamp when the metric was recorded	DateTime64(3, 'UTC')
Host	Broker host machine generating the metric	String
Region	AWS region where the cluster is running	String
Broker ID	Unique identifier of the Kafka broker	String
Burst Balance Average	Average EBS burst credit balance available for disk performance	Float64
Burst Balance Maximum	Maximum observed EBS burst credit balance	Float64
Burst Balance Minimum	Minimum observed EBS burst credit balance	Float64
Burst Balance Sample Count	Number of samples used for burst balance calculation	UInt32
Burst Balance Sum	Total sum of burst balance values across samples	Float64
Bandwidth In Allowance Exceeded Average	Average number of inbound network bandwidth allowance violations	UInt32
Bandwidth In Allowance Exceeded Maximum	Maximum inbound bandwidth allowance exceeded events observed	UInt32
Bandwidth In Allowance Exceeded Minimum	Minimum inbound bandwidth allowance exceeded events observed	UInt32
Bandwidth In Allowance Exceeded Sample Count	Number of samples used for inbound bandwidth violation metric	UInt32
Bandwidth In Allowance Exceeded Sum	Total inbound bandwidth allowance exceeded occurrences	UInt32
Bandwidth Out Allowance Exceeded Average	Average number of outbound network bandwidth allowance violations	UInt32
Bandwidth Out Allowance Exceeded Maximum	Maximum outbound bandwidth allowance exceeded events observed	UInt32
Bandwidth Out Allowance Exceeded Minimum	Minimum outbound bandwidth allowance exceeded events observed	UInt32
Bandwidth Out Allowance Exceeded Sample Count	Number of samples used for outbound bandwidth violation metric	UInt32
Bandwidth Out Allowance Exceeded Sum	Total outbound bandwidth allowance exceeded occurrences	UInt32
Bytes In Per Second Average	Average incoming data rate (producer traffic) in bytes per second	Float64
Bytes In Per Second Maximum	Peak incoming data rate in bytes per second	Float64
Bytes In Per Second Minimum	Lowest incoming data rate in bytes per second	Float64
Bytes In Per Second Sample Count	Number of samples used for incoming throughput calculation	UInt32
Bytes In Per Second Sum	Total bytes received per second into Kafka brokers	Float64
Bytes Out Per Second Average	Average bytes sent out per second from Kafka brokers	Float64
Bytes Out Per Second Maximum	Maximum bytes sent out per second from Kafka brokers	Float64
Bytes Out Per Second Minimum	Minimum bytes sent out per second from Kafka brokers	Float64
Bytes Out Per Second Sample Count	Number of samples used for outbound byte rate calculation	UInt32
Bytes Out Per Second Sum	Total bytes sent out per second aggregated over time window	Float64
Client Connection Count Average	Average number of active client connections to brokers	Float64
Client Connection Count Maximum	Maximum number of active client connections to brokers	UInt32
Client Connection Count Minimum	Minimum number of active client connections to brokers	UInt32
Client Connection Count Sample Count	Number of samples used for client connection metric	UInt32
Client Connection Count Sum	Total client connection count aggregated over time window	UInt32
Connection Close Rate Average	Average rate of connection closures per second	Float64
Connection Close Rate Maximum	Maximum connection close rate observed per second	Float64
Connection Close Rate Minimum	Minimum connection close rate observed per second	Float64
Connection Close Rate Sample Count	Number of samples used for connection close rate metric	UInt32
Connection Close Rate Sum	Total number of connection closures over time window	Float64
Connection Count Average	Average total number of active connections to broker	Float64
Connection Count Maximum	Maximum number of active connections observed	UInt32
Connection Count Minimum	Minimum number of active connections observed	UInt32
Connection Count Sample Count	Number of samples used for connection count metric	UInt32
Connection Count Sum	Total number of connection counts aggregated over time window	UInt32
Connection Creation Rate Average	Average rate of new connections created per second	Float64
Connection Creation Rate Maximum	Maximum connection creation rate observed	Float64
Connection Creation Rate Minimum	Minimum connection creation rate observed	Float64
Connection Creation Rate Sample Count	Number of samples used for connection creation metric	UInt32
Connection Creation Rate Sum	Total number of connection creations over time window	Float64
Conntrack Allowance Exceeded Average	Average number of conntrack (network tracking table) limit violations	UInt32
Conntrack Allowance Exceeded Maximum	Maximum conntrack limit exceeded events observed	UInt32
Conntrack Allowance Exceeded Minimum	Minimum conntrack limit exceeded events observed	UInt32
Conntrack Allowance Exceeded Sample Count	Number of samples used for conntrack metric	UInt32
Conntrack Allowance Exceeded Sum	Total conntrack limit exceeded occurrences	UInt32
CPU Credit Balance Average	Average CPU credit balance available for burstable instances	Float64
CPU Credit Balance Maximum	Maximum CPU credit balance observed	Float64
CPU Credit Balance Minimum	Minimum CPU credit balance observed	Float64
CPU Credit Balance Sample Count	Number of samples used for CPU credit balance metric	UInt32
CPU Credit Balance Sum	Total CPU credit balance accumulated over time window	Float64
CPU Credit Usage Average	Average CPU credit usage consumed by broker	Float64
CPU Credit Usage Maximum	Maximum CPU credit usage observed	Float64
CPU Credit Usage Minimum	Minimum CPU credit usage observed	Float64
CPU Credit Usage Sample Count	Number of samples used for CPU credit usage metric	UInt32
CPU Credit Usage Sum	Total CPU credit usage over time window	Float64
CPU Idle Average	Average CPU idle percentage of broker	Float64
CPU Idle Maximum	Maximum CPU idle percentage observed	Float64
CPU Idle Minimum	Minimum CPU idle percentage observed	Float64
CPU Idle Sample Count	Number of samples used for CPU idle metric	UInt32
CPU Idle Sum	Total CPU idle accumulated over time window	Float64
CPU I/O Wait Average	Average CPU time spent waiting on I/O operations	Float64
CPU I/O Wait Maximum	Maximum CPU I/O wait observed	Float64
CPU I/O Wait Minimum	Minimum CPU I/O wait observed	Float64
CPU I/O Wait Sample Count	Number of samples used for CPU I/O wait metric	UInt32
CPU I/O Wait Sum	Total CPU I/O wait time accumulated	Float64
CPU System Average	Average CPU time spent in system processes	Float64
CPU System Maximum	Maximum system CPU usage observed	Float64
CPU System Minimum	Minimum system CPU usage observed	Float64
CPU System Sample Count	Number of samples used for system CPU metric	UInt32
CPU System Sum	Total system CPU time accumulated	Float64
CPU User Average	Average CPU time spent on user processes	Float64
CPU User Maximum	Maximum user CPU usage observed	Float64
CPU User Minimum	Minimum user CPU usage observed	Float64
CPU User Sample Count	Number of samples used for user CPU metric	UInt32
CPU User Sum	Total user CPU time accumulated	Float64
Memory Buffered Average	Average buffered memory usage on broker	UInt64
Memory Buffered Maximum	Maximum buffered memory usage observed	UInt64
Memory Buffered Minimum	Minimum buffered memory usage observed	UInt64
Memory Buffered Sample Count	Number of samples used for buffered memory metric	UInt32
Memory Buffered Sum	Total buffered memory usage over time	UInt64
Memory Cached Average	Average cached memory usage on broker	UInt64
Memory Cached Maximum	Maximum cached memory usage observed	UInt64
Memory Cached Minimum	Minimum cached memory usage observed	UInt64
Memory Cached Sample Count	Number of samples used for cached memory metric	UInt32
Memory Cached Sum	Total cached memory usage over time	UInt64
Memory Free Average	Average free memory available on broker	UInt64
Memory Free Maximum	Maximum free memory observed	UInt64
Memory Free Minimum	Minimum free memory observed	UInt64
Memory Free Sample Count	Number of samples used for free memory metric	UInt32
Memory Free Sum	Free memory available on broker	UInt64
Memory Used Average	Average memory used by broker	UInt64
Memory Used Maximum	Maximum memory used by broker	UInt64
Memory Used Minimum	Minimum memory used by broker	UInt64
Memory Used Sample Count	Number of samples used for memory usage calculation	UInt32
Memory Used Sum	Total memory usage over time window	UInt64
Leader Count Average	Average number of partitions where broker is leader	UInt32
Leader Count Maximum	Maximum leader count observed on broker	UInt32
Leader Count Minimum	Minimum leader count observed on broker	UInt32
Leader Count Sample Count	Number of samples used for leader count metric	UInt32
Leader Count Sum	Total leader count aggregated over time window	UInt32
Partition Count Average	Average number of partitions assigned to broker	UInt32
Partition Count Maximum	Maximum partition count observed on broker	UInt32
Partition Count Minimum	Minimum partition count observed on broker	UInt32
Partition Count Sample Count	Number of samples used for partition count metric	UInt32
Partition Count Sum	Total partition count aggregated over time window	UInt32
Fetch Consumer Local Time Ms Mean Average	Average time taken for consumer fetch request local processing	Float64
Fetch Consumer Local Time Ms Mean Maximum	Maximum consumer fetch local processing time observed	Float64
Fetch Consumer Local Time Ms Mean Minimum	Minimum consumer fetch local processing time observed	Float64
Fetch Consumer Local Time Ms Mean Sample Count	Number of samples used for consumer fetch local time metric	UInt64
Fetch Consumer Local Time Ms Mean Sum	Total consumer fetch local processing time over window	Float64
Fetch Consumer Request Queue Time Ms Mean Average	Average time consumer requests spend in queue before processing	Float64
Fetch Consumer Request Queue Time Ms Mean Maximum	Maximum consumer request queue time observed	Float64
Fetch Consumer Request Queue Time Ms Mean Minimum	Minimum consumer request queue time observed	Float64
Fetch Consumer Request Queue Time Ms Mean Sample Count	Number of samples used for consumer request queue metric	UInt64
Fetch Consumer Request Queue Time Ms Mean Sum	Total consumer request queue time over window	Float64
Fetch Consumer Response Queue Time Ms Mean Average	Average time consumer responses spend in queue before sending	Float64
Fetch Consumer Response Queue Time Ms Mean Maximum	Maximum consumer response queue time observed	Float64
Fetch Consumer Response Queue Time Ms Mean Minimum	Minimum consumer response queue time observed	Float64
Fetch Consumer Response Queue Time Ms Mean Sample Count	Number of samples used for consumer response queue metric	UInt64
Fetch Consumer Response Queue Time Ms Mean Sum	Total consumer response queue time over window	Float64
Fetch Consumer Response Send Time Ms Mean Average	Average time spent sending consumer responses	Float64
Fetch Consumer Response Send Time Ms Mean Maximum	Maximum consumer response send time observed	Float64
Fetch Consumer Response Send Time Ms Mean Minimum	Minimum consumer response send time observed	Float64
Fetch Consumer Response Send Time Ms Mean Sample Count	Number of samples used for consumer response send metric	UInt64
Fetch Consumer Response Send Time Ms Mean Sum	Total consumer response send time over window	Float64
Fetch Consumer Total Time Ms Mean Average	Average total time for consumer fetch request completion	Float64
Fetch Consumer Total Time Ms Mean Maximum	Maximum consumer fetch total time observed	Float64
Fetch Consumer Total Time Ms Mean Minimum	Minimum consumer fetch total time observed	Float64
Fetch Consumer Total Time Ms Mean Sample Count	Number of samples used for consumer fetch total metric	UInt64
Fetch Consumer Total Time Ms Mean Sum	Total consumer fetch total time over window	Float64
Fetch Follower Local Time Ms Mean Average	Average local time for follower replica fetch requests	Float64
Fetch Follower Local Time Ms Mean Maximum	Maximum follower fetch local time observed	Float64
Fetch Follower Local Time Ms Mean Minimum	Minimum follower fetch local time observed	Float64
Fetch Follower Local Time Ms Mean Sample Count	Number of samples used for follower fetch local time metric	UInt64
Fetch Follower Local Time Ms Mean Sum	Total follower fetch local time over window	Float64
Fetch Follower Request Queue Time Ms Mean Average	Average time follower fetch requests spend in queue	Float64
Fetch Follower Request Queue Time Ms Mean Maximum	Maximum follower request queue time observed	Float64
Fetch Follower Request Queue Time Ms Mean Minimum	Minimum follower request queue time observed	Float64
Fetch Follower Request Queue Time Ms Mean Sample Count	Number of samples used for follower request queue metric	UInt64
Fetch Follower Request Queue Time Ms Mean Sum	Total follower request queue time over window	Float64
Fetch Follower Response Queue Time Ms Mean Average	Average time follower responses spend in queue	Float64
Fetch Follower Response Queue Time Ms Mean Maximum	Maximum follower response queue time observed	Float64
Fetch Follower Response Queue Time Ms Mean Minimum	Minimum follower response queue time observed	Float64
Fetch Follower Response Queue Time Ms Mean Sample Count	Number of samples used for follower response queue metric	UInt64
Fetch Follower Response Queue Time Ms Mean Sum	Total follower response queue time over window	Float64
Fetch Follower Response Send Time Ms Mean Average	Average time spent sending follower responses	Float64
Fetch Follower Response Send Time Ms Mean Maximum	Maximum follower response send time observed	Float64
Fetch Follower Response Send Time Ms Mean Minimum	Minimum follower response send time observed	Float64
Fetch Follower Response Send Time Ms Mean Sample Count	Number of samples used for follower response send metric	UInt64
Fetch Follower Response Send Time Ms Mean Sum	Total follower response send time over window	Float64
Fetch Follower Total Time Ms Mean Average	Average total time for follower fetch completion	Float64
Fetch Follower Total Time Ms Mean Maximum	Maximum follower fetch total time observed	Float64
Fetch Follower Total Time Ms Mean Minimum	Minimum follower fetch total time observed	Float64
Fetch Follower Total Time Ms Mean Sample Count	Number of samples used for follower fetch total metric	UInt64
Fetch Follower Total Time Ms Mean Sum	Total follower fetch total time over window	Float64
Fetch Message Conversions Per Second Average	Average message conversion rate per second (fetch path)	Float64
Fetch Message Conversions Per Second Maximum	Maximum message conversion rate observed	Float64
Fetch Message Conversions Per Second Minimum	Minimum message conversion rate observed	Float64
Fetch Message Conversions Per Second Sample Count	Number of samples used for message conversion rate metric	UInt64
Fetch Message Conversions Per Second Sum	Total message conversions over time window	Float64
Fetch Message Conversions Time Ms Mean Average	Average time taken for message conversions during fetch	Float64
Fetch Message Conversions Time Ms Mean Maximum	Maximum message conversion time observed	Float64
Fetch Message Conversions Time Ms Mean Minimum	Minimum message conversion time observed	Float64
Fetch Message Conversions Time Ms Mean Sample Count	Number of samples used for message conversion time metric	UInt64
Fetch Message Conversions Time Ms Mean Sum	Total message conversion time over window	Float64
Fetch Throttle Queue Size Average	Average size of fetch throttle queue	Float64
Fetch Throttle Queue Size Maximum	Maximum fetch throttle queue size observed	Float64
Fetch Throttle Queue Size Minimum	Minimum fetch throttle queue size observed	Float64
Fetch Throttle Queue Size Sample Count	Number of samples used for fetch throttle metric	UInt64
Fetch Throttle Queue Size Sum	Total fetch throttle queue size over time window	Float64
Heap Memory After GC Average	Average heap memory usage after garbage collection	Float64
Heap Memory After GC Maximum	Maximum heap memory usage after GC	Float64
Heap Memory After GC Minimum	Minimum heap memory usage after GC	Float64
Heap Memory After GC Sample Count	Number of samples used for heap memory GC metric	UInt64
Heap Memory After GC Sum	Total heap memory after GC over time window	Float64
IAM Number Of Connection Requests Average	Average number of IAM connection requests	Float64
IAM Number Of Connection Requests Maximum	Maximum IAM connection requests observed	Float64
IAM Number Of Connection Requests Minimum	Minimum IAM connection requests observed	Float64
IAM Number Of Connection Requests Sample Count	Number of samples used for IAM connection request metric	UInt64
IAM Number Of Connection Requests Sum	Total IAM connection requests over time window	Float64
IAM Too Many Connections Average	Average number of IAM rejected connections due to limits	Float64
IAM Too Many Connections Maximum	Maximum IAM connection rejection events observed	Float64
IAM Too Many Connections Minimum	Minimum IAM connection rejection events observed	Float64
IAM Too Many Connections Sample Count	Number of samples used for IAM rejection metric	UInt64
IAM Too Many Connections Sum	Total IAM rejected connections over time window	Float64
Kafka App Logs Disk Used Average	Average disk usage for Kafka application logs	Float64
Kafka App Logs Disk Used Maximum	Maximum disk usage for Kafka application logs	Float64
Kafka App Logs Disk Used Minimum	Minimum disk usage for Kafka application logs	Float64
Kafka App Logs Disk Used Sample Count	Number of samples used for app log disk usage metric	UInt64
Kafka App Logs Disk Used Sum	Kafka application log disk usage (total sum of disk used by Kafka app logs)	Float64
Kafka Data Logs Disk Used Average	Kafka data log disk usage (average disk usage of Kafka data logs on brokers)	Float64
Kafka Data Logs Disk Used Maximum	Kafka data log disk usage (maximum observed disk usage of Kafka data logs)	Float64
Kafka Data Logs Disk Used Minimum	Kafka data log disk usage (minimum observed disk usage of Kafka data logs)	Float64
Kafka Data Logs Disk Used Sample Count	Kafka data log disk usage (number of samples used for calculation)	UInt64
Kafka Data Logs Disk Used Sum	Kafka data log disk usage (total disk usage aggregated over time window)	Float64
Linklocal Allowance Exceeded Average	Link-local network allowance exceeded (average number of violations)	Float64
Linklocal Allowance Exceeded Maximum	Link-local network allowance exceeded (maximum violations observed)	Float64
Linklocal Allowance Exceeded Minimum	Link-local network allowance exceeded (minimum violations observed)	Float64
Linklocal Allowance Exceeded Sample Count	Link-local network allowance exceeded (number of samples used for calculation)	UInt64
Linklocal Allowance Exceeded Sum	Link-local network allowance exceeded (total violation occurrences)	Float64
Messages In Per Second Average	Messages received per second (average Kafka ingress message rate)	Float64
Messages In Per Second Maximum	Messages received per second (maximum observed message rate)	Float64
Messages In Per Second Minimum	Messages received per second (minimum observed message rate)	Float64
Messages In Per Second Sample Count	Messages received per second (number of samples used for calculation)	UInt64
Messages In Per Second Sum	Messages received per second (total messages over time window)	Float64
Network Processor Avg Idle Percent Average	Network processor average idle percentage (average CPU idle on network threads)	Float64
Network Processor Avg Idle Percent Maximum	Network processor average idle percentage (maximum observed idle percentage)	Float64
Network Processor Avg Idle Percent Minimum	Network processor average idle percentage (minimum observed idle percentage)	Float64
Network Processor Avg Idle Percent Sample Count	Network processor average idle percentage (number of samples used)	UInt64
Network Processor Avg Idle Percent Sum	Network processor average idle percentage (total aggregated idle time)	Float64
Network RX Dropped Average	Network receive packets dropped (average drop rate of incoming packets)	Float64
Network RX Dropped Maximum	Network receive packets dropped (maximum drop rate observed)	Float64
Network RX Dropped Minimum	Network receive packets dropped (minimum drop rate observed)	Float64
Network RX Dropped Sample Count	Network receive packets dropped (number of samples used)	UInt64
Network RX Dropped Sum	Network receive packets dropped (total dropped packets)	Float64
Network RX Errors Average	Network receive errors (average RX error rate)	Float64
Network RX Errors Maximum	Network receive errors (maximum RX errors observed)	Float64
Network RX Errors Minimum	Network receive errors (minimum RX errors observed)	Float64
Network RX Errors Sample Count	Network receive errors (number of samples used)	UInt64
Network RX Errors Sum	Network receive errors (total RX errors)	Float64
Network RX Packets Average	Network receive packets (average packets received per second)	Float64
Network RX Packets Maximum	Network receive packets (maximum packets received per second)	Float64
Network RX Packets Minimum	Network receive packets (minimum packets received per second)	Float64
Network RX Packets Sample Count	Network receive packets (number of samples used)	UInt64
Network RX Packets Sum	Network receive packets (total packets received)	Float64
Network TX Dropped Average	Network transmit packets dropped (average TX drop rate)	Float64
Network TX Dropped Maximum	Network transmit packets dropped (maximum TX drops observed)	Float64
Network TX Dropped Minimum	Network transmit packets dropped (minimum TX drops observed)	Float64
Network TX Dropped Sample Count	Network transmit packets dropped (number of samples used)	UInt64
Network TX Dropped Sum	Network transmit packets dropped (total TX drops)	Float64
Network TX Errors Average	Network transmit errors (average TX error rate)	Float64
Network TX Errors Maximum	Network transmit errors (maximum TX errors observed)	Float64
Network TX Errors Minimum	Network transmit errors (minimum TX errors observed)	Float64
Network TX Errors Sample Count	Network transmit errors (number of samples used)	UInt64
Network TX Errors Sum	Network transmit errors (total TX errors)	Float64
Network TX Packets Average	Network transmit packets (average packets sent per second)	Float64
Network TX Packets Maximum	Network transmit packets (maximum packets sent per second)	Float64
Network TX Packets Minimum	Network transmit packets (minimum packets sent per second)	Float64
Network TX Packets Sample Count	Network transmit packets (number of samples used)	UInt64
Network TX Packets Sum	Network transmit packets (total packets sent)	Float64
PPS Allowance Exceeded Average	Packets-per-second allowance exceeded (average violations)	Float64
PPS Allowance Exceeded Maximum	Packets-per-second allowance exceeded (maximum violations)	Float64
PPS Allowance Exceeded Minimum	Packets-per-second allowance exceeded (minimum violations)	Float64
PPS Allowance Exceeded Sample Count	Packets-per-second allowance exceeded (number of samples used)	UInt64
PPS Allowance Exceeded Sum	Packets-per-second allowance exceeded (total violation events)	Float64
Produce Local Time Ms Mean Average	Produce request local processing time (average time to process produce requests locally)	Float64
Produce Local Time Ms Mean Maximum	Produce request local processing time (maximum observed latency)	Float64
Produce Local Time Ms Mean Minimum	Produce request local processing time (minimum observed latency)	Float64
Produce Local Time Ms Mean Sample Count	Produce request local processing time (number of samples used)	UInt64
Produce Local Time Ms Mean Sum	Produce request local processing time (total processing time over window)	Float64
Produce Message Conversions Per Second Average	Produce message conversion rate per second (average conversion rate during produce path)	Float64
Produce Message Conversions Per Second Maximum	Produce message conversion rate per second (maximum observed conversion rate)	Float64
Produce Message Conversions Per Second Minimum	Produce message conversion rate per second (minimum observed conversion rate)	Float64
Produce Message Conversions Per Second Sample Count	Produce message conversion rate per second (number of samples used)	UInt64
Produce Message Conversions Per Second Sum	Produce message conversion rate per second (total conversions over time window)	Float64
Produce Message Conversions Time Ms Mean Average	Produce message conversion time (average time spent converting messages)	Float64
Produce Message Conversions Time Ms Mean Maximum	Produce message conversion time (maximum observed conversion time)	Float64
Produce Message Conversions Time Ms Mean Minimum	Produce message conversion time (minimum observed conversion time)	Float64
Produce Message Conversions Time Ms Mean Sample Count	Produce message conversion time (number of samples used)	UInt64
Produce Message Conversions Time Ms Mean Sum	Produce message conversion time (total conversion time over window)	Float64
Produce Request Queue Time Ms Mean Average	Produce request queue time (average time requests spend waiting in queue)	Float64
Produce Request Queue Time Ms Mean Maximum	Produce request queue time (maximum queue delay observed)	Float64
Produce Request Queue Time Ms Mean Minimum	Produce request queue time (minimum queue delay observed)	Float64
Produce Request Queue Time Ms Mean Sample Count	Produce request queue time (number of samples used)	UInt64
Produce Request Queue Time Ms Mean Sum	Produce request queue time (total queue time over window)	Float64
Produce Response Queue Time Ms Mean Average	Produce response queue time (average time responses wait before sending)	Float64
Produce Response Queue Time Ms Mean Maximum	Produce response queue time (maximum response queue delay observed)	Float64
Produce Response Queue Time Ms Mean Minimum	Produce response queue time (minimum response queue delay observed)	Float64
Produce Response Queue Time Ms Mean Sample Count	Produce response queue time (number of samples used)	UInt64
Produce Response Queue Time Ms Mean Sum	Produce response queue time (total response queue time over window)	Float64
Produce Response Send Time Ms Mean Average	Produce response send time (average time to send responses)	Float64
Produce Response Send Time Ms Mean Maximum	Produce response send time (maximum response send latency observed)	Float64
Produce Response Send Time Ms Mean Minimum	Produce response send time (minimum response send latency observed)	Float64
Produce Response Send Time Ms Mean Sample Count	Produce response send time (number of samples used)	UInt64
Produce Response Send Time Ms Mean Sum	Produce response send time (total response send time over window)	Float64
Produce Throttle Queue Size Average	Produce throttle queue size (average size of produce throttle queue)	Float64
Produce Throttle Queue Size Maximum	Produce throttle queue size (maximum observed queue size)	Float64
Produce Throttle Queue Size Minimum	Produce throttle queue size (minimum observed queue size)	Float64
Produce Throttle Queue Size Sample Count	Produce throttle queue size (number of samples used)	UInt64
Produce Throttle Queue Size Sum	Produce throttle queue size (total queue size over time window)	Float64
Produce Total Time Ms Mean Average	Produce total time (average end-to-end produce request time)	Float64
Produce Total Time Ms Mean Maximum	Produce total time (maximum produce latency observed)	Float64
Produce Total Time Ms Mean Minimum	Produce total time (minimum produce latency observed)	Float64
Produce Total Time Ms Mean Sample Count	Produce total time (number of samples used)	UInt64
Produce Total Time Ms Mean Sum	Produce total time (total produce processing time over window)	Float64
Replication Bytes In Per Second Average	Replication bytes in per second (average replication ingress rate between brokers)	Float64
Replication Bytes In Per Second Maximum	Replication bytes in per second (maximum replication ingress rate observed)	Float64
Replication Bytes In Per Second Minimum	Replication bytes in per second (minimum replication ingress rate observed)	Float64
Replication Bytes In Per Second Sample Count	Replication bytes in per second (number of samples used for calculation)	UInt64
Replication Bytes In Per Second Sum	Replication bytes in per second (total replication data received across brokers over time window)	Float64
Replication Bytes Out Per Second Average	Replication bytes out per second (average replication data sent from brokers to replicas)	Float64
Replication Bytes Out Per Second Maximum	Replication bytes out per second (maximum observed replication outbound throughput)	Float64
Replication Bytes Out Per Second Minimum	Replication bytes out per second (minimum observed replication outbound throughput)	Float64
Replication Bytes Out Per Second Sample Count	Replication bytes out per second (number of samples used for calculation)	UInt64
Replication Bytes Out Per Second Sum	Replication bytes out per second (total replication outbound bytes over time window)	Float64
Request Bytes Mean Average	Request bytes mean (average size of broker requests in bytes)	Float64
Request Bytes Mean Maximum	Request bytes mean (maximum request size observed)	Float64
Request Bytes Mean Minimum	Request bytes mean (minimum request size observed)	Float64
Request Bytes Mean Sample Count	Request bytes mean (number of samples used for calculation)	UInt64
Request Bytes Mean Sum	Request bytes mean (total request bytes aggregated over time window)	Float64
Request Handler Avg Idle Percent Average	Request handler average idle percent (average idle capacity of broker request threads)	Float64
Request Handler Avg Idle Percent Maximum	Request handler average idle percent (maximum idle percentage observed)	Float64
Request Handler Avg Idle Percent Minimum	Request handler average idle percent (minimum idle percentage observed)	Float64
Request Handler Avg Idle Percent Sample Count	Request handler average idle percent (number of samples used for calculation)	UInt64
Request Handler Avg Idle Percent Sum	Request handler average idle percent (total idle percentage over time window)	Float64
Request Throttle Queue Size Average	Request throttle queue size (average number of requests waiting due to throttling)	Float64
Request Throttle Queue Size Maximum	Request throttle queue size (maximum throttle queue size observed)	Float64
Request Throttle Queue Size Minimum	Request throttle queue size (minimum throttle queue size observed)	Float64
Request Throttle Queue Size Sample Count	Request throttle queue size (number of samples used for calculation)	UInt64
Request Throttle Queue Size Sum	Request throttle queue size (total throttle queue accumulation over time window)	Float64
Root Disk Used Average	Root disk used (average disk usage of broker root volume)	Float64
Root Disk Used Maximum	Root disk used (maximum disk usage observed on root volume)	Float64
Root Disk Used Minimum	Root disk used (minimum disk usage observed on root volume)	Float64
Root Disk Used Sample Count	Root disk used (number of samples used for calculation)	UInt64
Root Disk Used Sum	Root disk used (total root disk usage over time window)	Float64
Swap Free Average	Swap free (average free swap memory available on broker host)	Float64
Swap Free Maximum	Swap free (maximum swap free observed)	Float64
Swap Free Minimum	Swap free (minimum swap free observed)	Float64
Swap Free Sample Count	Swap free (number of samples used for calculation)	UInt64
Swap Free Sum	Swap free (total swap free over time window)	Float64
Swap Used Average	Swap used (average swap memory usage on broker host)	Float64
Swap Used Maximum	Swap used (maximum swap usage observed)	Float64
Swap Used Minimum	Swap used (minimum swap usage observed)	Float64
Swap Used Sample Count	Swap used (number of samples used for calculation)	UInt64
Swap Used Sum	Swap used (total swap usage over time window)	Float64
TCP Connections Average	TCP connections (average number of active TCP connections to broker)	Float64
TCP Connections Maximum	TCP connections (maximum TCP connections observed)	Float64
TCP Connections Minimum	TCP connections (minimum TCP connections observed)	Float64
TCP Connections Sample Count	TCP connections (number of samples used for calculation)	UInt64
TCP Connections Sum	TCP connections (total TCP connections over time window)	Float64
Traffic Bytes Average	Traffic bytes (average total network traffic handled by broker)	Float64
Traffic Bytes Maximum	Traffic bytes (maximum network traffic observed)	Float64
Traffic Bytes Minimum	Traffic bytes (minimum network traffic observed)	Float64
Traffic Bytes Sample Count	Traffic bytes (number of samples used for calculation)	UInt64
Traffic Bytes Sum	Traffic bytes (total network traffic over time window)	Float64
Traffic Shaping Average	Traffic shaping (average throttling applied due to traffic limits)	Float64
Traffic Shaping Maximum	Traffic shaping (maximum throttling observed)	Float64
Traffic Shaping Minimum	Traffic shaping (minimum throttling observed)	Float64
Traffic Shaping Sample Count	Traffic shaping (number of samples used for calculation)	UInt64
Traffic Shaping Sum	Traffic shaping (total throttling impact over time window)	Float64
Under Min ISR Partition Count Average	Under min ISR partition count (average number of partitions below minimum in-sync replicas)	Float64
Under Min ISR Partition Count Maximum	Under min ISR partition count (maximum observed under-replicated severity)	Float64
Under Min ISR Partition Count Minimum	Under min ISR partition count (minimum observed under-replication state)	Float64
Under Min ISR Partition Count Sample Count	Under min ISR partition count (number of samples used for calculation)	UInt64
Under Min ISR Partition Count Sum	Under min ISR partition count (total under-replication occurrences over time window)	Float64
Under Replicated Partitions Average	Under replicated partitions (average number of under-replicated partitions in cluster)	Float64
Under Replicated Partitions Maximum	Under replicated partitions (maximum under-replicated partitions observed)	Float64
Under Replicated Partitions Minimum	Under replicated partitions (minimum under-replicated partitions observed)	Float64
Under Replicated Partitions Sample Count	Under replicated partitions (number of samples used for calculation)	UInt64
Under Replicated Partitions Sum	Under replicated partitions (total under-replicated partition occurrences)	Float64
User Partition Exists Average	User partition exists (average check for existence of user-defined partitions)	Float64
User Partition Exists Maximum	User partition exists (maximum observed state indicator)	Float64
User Partition Exists Minimum	User partition exists (minimum observed state indicator)	Float64
User Partition Exists Sample Count	User partition exists (number of samples used for calculation)	UInt64
User Partition Exists Sum	User partition exists (total partition existence checks over time window)	Float64
Volume Queue Length Average	Volume queue length (average number of pending disk I/O operations)	Float64
Volume Queue Length Maximum	Volume queue length (maximum disk queue depth observed)	Float64
Volume Queue Length Minimum	Volume queue length (minimum disk queue depth observed)	Float64
Volume Queue Length Sample Count	Volume queue length (number of samples used for calculation)	UInt64
Volume Queue Length Sum	Volume queue length (total disk queue accumulation over time window)	Float64
Volume Read Bytes Average	Volume read bytes (average disk read throughput in bytes)	Float64
Volume Read Bytes Maximum	Volume read bytes (maximum disk read throughput observed)	Float64
Volume Read Bytes Minimum	Volume read bytes (minimum disk read throughput observed)	Float64
Volume Read Bytes Sample Count	Volume read bytes (number of samples used for calculation)	UInt64
Volume Read Bytes Sum	Volume read bytes (total disk read bytes over time window)	Float64
Volume Read Ops Average	Volume read operations (average number of disk read I/O operations per second)	Float64
Volume Read Ops Maximum	Volume read operations (maximum disk read IOPS observed)	Float64
Volume Read Ops Minimum	Volume read operations (minimum disk read IOPS observed)	Float64
Volume Read Ops Sample Count	Volume read operations (number of samples used for calculation)	UInt64
Volume Read Ops Sum	Volume read operations (total disk read operations over time window)	Float64
Volume Total Read Time Average	Volume total read time (average time spent on disk read operations)	Float64
Volume Total Read Time Maximum	Volume total read time (maximum disk read latency observed)	Float64
Volume Total Read Time Minimum	Volume total read time (minimum disk read latency observed)	Float64
Volume Total Read Time Sample Count	Volume total read time (number of samples used for calculation)	UInt64
Volume Total Read Time Sum	Volume total read time (total disk read latency over time window)	Float64
Volume Total Write Time Average	Volume total write time (average time spent on disk write operations)	Float64
Volume Total Write Time Maximum	Volume total write time (maximum disk write latency observed)	Float64
Volume Total Write Time Minimum	Volume total write time (minimum disk write latency observed)	Float64
Volume Total Write Time Sample Count	Volume total write time (number of samples used for calculation)	UInt64
Volume Total Write Time Sum	Volume total write time (total disk write latency over time window)	Float64
Volume Write Bytes Average	Volume write bytes (average disk write throughput in bytes)	Float64
Volume Write Bytes Maximum	Volume write bytes (maximum disk write throughput observed)	Float64
Volume Write Bytes Minimum	Volume write bytes (minimum disk write throughput observed)	Float64
Volume Write Bytes Sample Count	Volume write bytes (number of samples used for calculation)	UInt64
Volume Write Bytes Sum	Volume write bytes (total disk write bytes over time window)	Float64
Volume Write Ops Average	Volume write operations (average disk write IOPS)	Float64
Volume Write Ops Maximum	Volume write operations (maximum disk write IOPS observed)	Float64
Volume Write Ops Minimum	Volume write operations (minimum disk write IOPS observed)	Float64
Volume Write Ops Sample Count	Volume write operations (number of samples used for calculation)	UInt64
Volume Write Ops Sum	Volume write operations (total number of disk write IOPS over time window)	Float64
Cluster Name	Cluster name (name of the MSK cluster)	String
Time Stamp	Timestamp (time when metric was recorded)	DateTime64(3, 'UTC')
Host	Host (broker host generating the metric)	String
Region	Region (AWS region of the cluster)	String
Active Controller Count Average	Active controller count (average number of active Kafka controllers in cluster)	Float64
Active Controller Count Maximum	Active controller count (maximum observed active controller instances)	Float64
Active Controller Count Minimum	Active controller count (minimum observed active controller instances)	Float64
Active Controller Count Sample Count	Active controller count (number of samples used for calculation)	UInt32
Active Controller Count Sum	Active controller count (total active controller count over time window)	Float64
Global Partition Count Average	Global partition count (average total number of partitions in the cluster)	Float64
Global Partition Count Maximum	Global partition count (maximum partitions observed in cluster)	Float64
Global Partition Count Minimum	Global partition count (minimum partitions observed in cluster)	Float64
Global Partition Count Sample Count	Global partition count (number of samples used for calculation)	UInt32
Global Partition Count Sum	Global partition count (total partition count over time window)	Float64
Global Topic Count Average	Global topic count (average number of topics in cluster)	Float64
Global Topic Count Maximum	Global topic count (maximum topic count observed)	Float64
Global Topic Count Minimum	Global topic count (minimum topic count observed)	Float64
Global Topic Count Sample Count	Global topic count (number of samples used for calculation)	UInt32
Global Topic Count Sum	Global topic count (total topic count over time window)	Float64
Offline Partitions Count Average	Offline partitions count (average number of partitions unavailable/offline)	Float64
Offline Partitions Count Maximum	Offline partitions count (maximum offline partitions observed)	Float64
Offline Partitions Count Minimum	Offline partitions count (minimum offline partitions observed)	Float64
Offline Partitions Count Sample Count	Offline partitions count (number of samples used for calculation)	UInt32
Offline Partitions Count Sum	Offline partitions count (total offline partition occurrences)	Float64
ZooKeeper Request Latency Ms Mean Average	ZooKeeper request latency (average latency of ZooKeeper requests in ms)	Float64
ZooKeeper Request Latency Ms Mean Maximum	ZooKeeper request latency (maximum observed latency)	Float64
ZooKeeper Request Latency Ms Mean Minimum	ZooKeeper request latency (minimum observed latency)	Float64
ZooKeeper Request Latency Ms Mean Sample Count	ZooKeeper request latency (number of samples used for calculation)	UInt32
ZooKeeper Request Latency Ms Mean Sum	ZooKeeper request latency (total ZooKeeper request latency over window)	Float64
ZooKeeper Session State Average	ZooKeeper session state (average session state value indicating health)	Float64
ZooKeeper Session State Maximum	ZooKeeper session state (maximum session state observed)	Float64
ZooKeeper Session State Minimum	ZooKeeper session state (minimum session state observed)	Float64
ZooKeeper Session State Sample Count	ZooKeeper session state (number of samples used for calculation)	UInt32
ZooKeeper Session State Sum	ZooKeeper session state (total session state aggregation over time window)	Float64
Kafka Data Logs Disk Used Average	Kafka data log disk usage (average disk usage of Kafka data logs)	Float64
Kafka Data Logs Disk Used Maximum	Kafka data log disk usage (maximum disk usage observed)	Float64
Kafka Data Logs Disk Used Minimum	Kafka data log disk usage (minimum disk usage observed)	Float64
Kafka Data Logs Disk Used Sample Count	Kafka data log disk usage (number of samples used for calculation)	UInt32
Kafka Data Logs Disk Used Sum	Kafka data log disk usage (total disk usage over time window)	Float64
Client Connection Count Average	Client connection count (average number of client connections to brokers)	Float64
Client Connection Count Maximum	Client connection count (maximum client connections observed)	Float64
Client Connection Count Minimum	Client connection count (minimum client connections observed)	Float64
Client Connection Count Sample Count	Client connection count (number of samples used for calculation)	Float64
Client Connection Count Sum	Client connection count (total client connections over time window)	Float64
Cluster Name	Cluster name (name of the MSK cluster)	String
Time Stamp	Timestamp (time when metric was recorded)	DateTime64(3, 'UTC')
Host	Host (broker host generating the metric)	String
Region	Region (AWS region of the cluster)	String
Topic	Topic (Kafka topic name)	String
Consumer Group	Consumer group (Kafka consumer group name)	String
Estimated Max Time Lag Average	Estimated max time lag (average estimated consumer lag in time)	Float64
Estimated Max Time Lag Maximum	Estimated max time lag (maximum observed consumer lag)	Float64
Estimated Max Time Lag Minimum	Estimated max time lag (minimum observed consumer lag)	Float64
Estimated Max Time Lag Sample Count	Estimated max time lag (number of samples used for calculation)	UInt64
Estimated Max Time Lag Sum	Estimated max time lag (total lag over time window)	Float64
Max Offset Lag Average	Max offset lag (average maximum offset lag per partition)	Float64
Max Offset Lag Maximum	Max offset lag (maximum observed offset lag)	Float64
Max Offset Lag Minimum	Max offset lag (minimum observed offset lag)	Float64
Max Offset Lag Sample Count	Max offset lag (number of samples used for calculation)	UInt64
Max Offset Lag Sum	Max offset lag (total offset lag over time window)	Float64
Rolling Estimated Time Lag Max Average	Rolling estimated time lag max (average smoothed consumer lag estimate)	Float64
Rolling Estimated Time Lag Max Maximum	Rolling estimated time lag max (maximum rolling lag observed)	Float64
Rolling Estimated Time Lag Max Minimum	Rolling estimated time lag max (minimum rolling lag observed)	Float64
Rolling Estimated Time Lag Max Sample Count	Rolling estimated time lag max (number of samples used for calculation)	UInt64
Rolling Estimated Time Lag Max Sum	Rolling estimated time lag max (total rolling lag over time window)	Float64
Sum Offset Lag Average	Sum offset lag (average total lag across all partitions)	Float64
Sum Offset Lag Maximum	Sum offset lag (maximum total lag observed)	Float64
Sum Offset Lag Minimum	Sum offset lag (minimum total lag observed)	Float64
Sum Offset Lag Sample Count	Sum offset lag (number of samples used for calculation)	UInt64
Sum Offset Lag Sum	Sum offset lag (total lag aggregated over time window)	Float64
Cluster Name	Cluster name (name of the MSK cluster)	String
Time Stamp	Timestamp (time when metric was recorded)	DateTime64(3, 'UTC')
Host	Host (broker host generating the metric)	String
Region	Region (AWS region of the cluster)	String
Topic	Topic (Kafka topic name)	String
Broker ID	Broker ID (Kafka broker identifier)	String
Bytes In Per Second Average	Bytes in per second (average incoming data rate to brokers)	Float64
Bytes In Per Second Maximum	Bytes in per second (maximum incoming throughput observed)	Float64
Bytes In Per Second Minimum	Bytes in per second (minimum incoming throughput observed)	Float64
Bytes In Per Second Sample Count	Bytes in per second (number of samples used for calculation)	UInt32
Bytes In Per Second Sum	Bytes in per second (total incoming bytes over time window)	Float64
Bytes Out Per Second Average	Bytes out per second (average outgoing data rate from brokers)	Float64
Bytes Out Per Second Maximum	Bytes out per second (maximum outgoing throughput observed)	Float64
Bytes Out Per Second Minimum	Bytes out per second (minimum outgoing throughput observed)	Float64
Bytes Out Per Second Sample Count	Bytes out per second (number of samples used for calculation)	UInt32
Bytes Out Per Second Sum	Bytes out per second (total outgoing bytes over time window)	Float64
Messages In Per Second Average	Messages in per second (average message ingestion rate into Kafka)	Float64
Messages In Per Second Maximum	Messages in per second (maximum message ingestion rate observed)	Float64
Messages In Per Second Minimum	Messages in per second (minimum message ingestion rate observed)	Float64
Messages In Per Second Sample Count	Messages in per second (number of samples used for calculation)	UInt64
Messages In Per Second Sum	Messages in per second (total messages ingested over time window)	Float64