menu
techminis

A naukri.com initiative

google-web-stories
source image

Netflixtechblog

1M

read

370

img
dot

Image Credit: Netflixtechblog

How Netflix Accurately Attributes eBPF Flow Logs

  • Netflix uses eBPF and TCP tracepoints to capture TCP flow logs at scale for network insights.
  • FlowExporter generates flow log records including IP addresses, ports, and timestamps, with around 5 million records per second.
  • Attributing IP addresses accurately to workload identities is crucial for making flow logs useful.
  • Initially, misattribution issues arose due to delays in IP address change events reaching FlowCollector.
  • FlowCollector's new attribution method involves correctly attributing local and remote IP addresses.
  • For container workloads, IPManAgent assists in assigning IP addresses to workload IDs.
  • Accurate attribution of IP addresses is achieved by maintaining time ranges of IP address ownership.
  • A broadcasting mechanism using Kafka helps in sharing IP address time ranges among FlowCollector nodes.
  • Cross-regional IP address attribution is facilitated by forwarding flows to nodes in the corresponding region.
  • Verification of the new attribution method was done by analyzing flow logs of a large service with known dependencies.

Read Full Article

like

16 Likes

For uninterrupted reading, download the app