<ul><li>In a project at DataStream, two senior engineers, Alex and Taylor, had differing approaches in redesigning the event processing pipeline to handle increased data volumes.</li><li>Alex advocated for a batch-processing solution using Apache Spark for cost efficiency and simpler implementation, while Taylor pushed for a real-time streaming approach with Apache Kafka and Flink to minimize latency.</li><li>To resolve the disagreement, a structured decision-making workshop was organized, with a comparison framework to assess objective criteria like performance, complexity, scalability, and cost projections.</li><li>The workshop led to a hybrid approach, using Kafka for ingestion, processing time-sensitive events with Flink, and handling the majority with scheduled Spark jobs.</li></ul>

Strong Story Telling study for Software Engineer — Day 5

Discover more