A naukri.com initiative
Amazon
2w
223
Image Credit: Amazon
Implement Amazon EMR HBase Graceful Scaling
- Amazon EMR with HBase on Amazon S3 provides random, strictly consistent real-time access for tables with Apache Kylin, ingesting data through spark jobs and querying HTables through Apache Kylin cubes.
- To scale in long-running Amazon EMR HBase clusters, consider issues like EC2 scheduling events, budget concerns, and Spot Instances usage for parallel computation power.
- Unexpected shutdown on HBase region servers can lead to WAL splits, causing extra load and potential inconsistencies in HTables in an Amazon EMR HBase cluster.
- Administrators aim to scale-in Amazon EMR HBase clusters gracefully by stopping all HBase region servers on task nodes.
- The solution includes using built-in scripts to move regions, raising decommission priority, and configuring software settings to prevent region server provisioning on task nodes.
- Method 1 involves decommissioning HBase region servers through resizing and manipulating decommission priority.
- Method 2 focuses on manually decommissioning target core nodes with careful consideration to avoid data loss and ensure HDFS block replication.
- Prevent Amazon EMR from provisioning HBase region servers on task nodes for new clusters by configuring HBase settings and scaling in task groups.
- Administrators can run scripts as Amazon EMR steps to gracefully scale in HBase clusters and handle provisioning of region servers on task nodes effectively.
- The post highlights detailed steps, syntax examples, and scripts for gracefully decommissioning target region servers in Amazon EMR HBase clusters.
Read Full Article
13 Likes
For uninterrupted reading, download the app