A naukri.com initiative
Amazon
2M
271
Image Credit: Amazon
How MuleSoft achieved cloud excellence through an event-driven Amazon Redshift lakehouse architecture
- MuleSoft implemented an event-driven strategy to achieve near real-time analytics enabling AWS Glue jobs through a layered approach that included collection, preparation, and enrichment to identify data accuracy areas.
- MuleSoft's cloud infrastructure data was prepared using the COE Framework, business goals were determined, and data sources were ingested after processing in the lakehouse.
- Cloudquery, AWS managed services, and data ingestion tools were used to collect cloud infrastructure data from multiple data sources in an S3 bucket for enrichment.
- An event-driven architecture comprising of EventBridge, Amazon SQS, and Amazon S3 Event Notifications triggered AWS Glue jobs that transformed and cleansed data so that it was ready for analysis using Amazon Redshift.
- File partitioning, Redshift Spectrum, and materialized views were techniques used to address storage, scalability, and efficiency challenges faced during the enrichment phase.
- The use of Amazon Redshift Materialized Views empowered MuleSoft Engineers to identify and tag assets and remediate non-compliant resources in real-time through an integrated portal, Cloud Central.
- The COE Framework has enabled near real-time streaming of MuleSoft's cloud infrastructure data, leveling up a COE Framework well into the future.
- MuleSoft is exploring new ways to improve the total cost of ownership, lower costs using Amazon Redshift Serverless for transient database workloads and exploring Amazon DataZone to democratize data sharing among lines of businesses.
- This solution can enable enterprises to understand their cloud infrastructure and run on a data modernization lakehouse applying Well-Architected concepts, best practices, and cloud-centric principles achieving their strategic goals.
- The technical and business impact of MuleSoft's COE Framework enables an optimization strategy and a cloud usage show back approach, driving continual maturity and benefits in cloud infrastructure centered around operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability.
Read Full Article
16 Likes
For uninterrupted reading, download the app