Statistics is a practical tool for software engineers to analyze performance, reason about systems under load, and evaluate solutions objectively.
Using simple statistical metrics like mean, median, and percentiles can help in picking the best-performing implementation from a set of options.
Metrics such as mean can be misleading, especially in the presence of outliers, so it's essential to consider other metrics for a well-rounded view of system performance.
It's crucial to combine multiple metrics, avoid comparing different sample sizes directly, and normalize sample sizes for fair conclusions when evaluating different solutions.