Setting up Network Load Balancer (NLB), Ingress, and Services on a hybrid EKS infrastructure involves configuring routing rules, disabling kube-proxy on hybrid nodes, updating Cilium configuration, verifying Cilium agents, configuring static routes on the VPN router, and installing the Load Balancer Controller.
To enable communication between pods on AWS-managed and hybrid nodes, specific routing rules are configured on the VPN gateway/router, replacing kube-proxy functionality with Cilium and updating Cilium configuration.
Verification includes checking cilium-agent pods running on hybrid nodes and no kube-proxy pods present, configuring static routes based on subnet allocation, and installing the AWS Load Balancer Controller using Terraform.
Deployment of a demo app with 6 replicas helps observe pod distribution, setting up Ingress, Service, and Load Balancer Controller, and testing round-robin behavior through curl commands.
Pricing overview: while hybrid nodes do not incur EC2 costs, expenses include EKS control plane fees, data transfer, and hybrid node usage costs based on vCPU-hours.
In conclusion, running a hybrid EKS environment combining AWS-managed and on-prem nodes is feasible, providing flexibility for organizations with existing infrastructure or specific compliance needs, and laying a foundation for more advanced setups in production scenarios.