Skip to main content

Hubble Metrics

When Retina is deployed with Hubble control plane, the metrics include Node-level and Pod-level. Metrics are stored in Prometheus format, and can be viewed in Grafana.

Metrics Endpoints

The Hubble control plane exposes metrics on two separate ports:

  • Port 10093: Node-level metrics (networkobservability_* prefix)
  • Port 9965: Hubble pod-level metrics (hubble_* prefix)

Metrics

  • Node-Level Metrics: These metrics provide insights into traffic volume, dropped packets, number of connections, etc. by node. Available on port 10093.
  • Hubble Metrics (DNS and Pod-Level Metrics): These metrics include source and destination pod information allowing to pinpoint network-related issues at a granular level. Metrics cover DNS queries/responses, L4/L7 packet flows, dropped packets, and TCP flags. Available on port 9965.

Node-Level Metrics

The following metrics are aggregated per node and available on port 10093. All metrics include labels:

  • cluster
  • instance (Node name)

Retina provides metrics for both Linux and Windows operating systems. The table below outlines the different metrics generated.

Metric NameDescriptionExtra LabelsLinuxWindows
networkobservability_forward_countTotal forwarded packet countdirection
networkobservability_forward_bytesTotal forwarded byte countdirection
networkobservability_drop_countTotal dropped packet countdirection, reason
networkobservability_drop_bytesTotal dropped byte countdirection, reason
networkobservability_tcp_stateTCP currently active socket count by TCP state.state
networkobservability_tcp_connection_remoteTCP currently active socket count by remote address.address (IP:port)
networkobservability_tcp_connection_statsTCP connection statistics. (ex: Delayed ACKs, TCPKeepAlive, TCPSackFailures)statistic_name
networkobservability_tcp_flag_gaugesTCP packets count by flag.direction, flag
networkobservability_ip_connection_statsIP connection statistics.statistic_name
networkobservability_udp_connection_statsUDP connection statistics. Includes active socket count with statistic_name="ACTIVE".statistic_name
networkobservability_interface_statsInterface statistics.interface_name, statistic_name
networkobservability_dns_request_countTotal DNS request count
networkobservability_dns_response_countTotal DNS response count
networkobservability_windows_hns_statsWindows HNS statistics (packets sent/received)direction
networkobservability_node_connectivity_statusConnectivity status between nodes (1=connected, 0=not)source_node_name, target_node_name
networkobservability_node_connectivity_latency_secondsLatency in seconds between nodessource_node_name, target_node_name

Pod-Level Metrics (Hubble Metrics)

The following metrics are aggregated per pod (node information is preserved) and available on port 9965. All metrics include labels:

  • cluster
  • instance (Node name)
  • source
  • destination

For outgoing traffic, there will be a source label with source pod namespace/name. For incoming traffic, there will be a destination label with destination pod namespace/name.

Metric NameDescriptionExtra LabelsLinuxWindows
hubble_dns_queries_totalTotal DNS requests by querysource or destination, query, qtypes (query type), ips_returned, rcode
hubble_dns_responses_totalTotal DNS responses by query/responsesource or destination, query, qtypes (query type), rcode (return code), ips_returned (number of IPs)
hubble_drop_totalTotal dropped packet countsource or destination, protocol, reason
hubble_flows_processed_totalTotal network flows processed (L4/L7 traffic)source or destination, protocol, verdict, type, subtype
hubble_tcp_flags_totalTotal TCP packets count by flag.source or destination, flag
hubble_lost_events_totalTotal number of lost Hubble events