A naukri.com initiative
Amazon
2w
426
Image Credit: Amazon
How Firmex used AWS SCT and AWS DMS to move 65,000 on-premises Microsoft SQL Server databases to an Amazon Aurora PostgreSQL cluster
- Firmex is an online data room provider that offers a strict data isolation strategy of individual databases. They had around 65,000 Microsoft SQL Servers' databases deployed following this architecture and were fast approaching the database limit. This led Firmex to develop a solution that fit their data model in the cloud to contain customer data and eventually decided to migrate to Amazon Aurora PostgreSQL.
- Firmex’s engineers proceeded with the approach of shifting tenant isolation from a database-based isolation solution to a schema-based isolation solution, allowing the application to operate within database limit restrictions. They selected Aurora PostgreSQL as the migration option as it had very attractive price-performance ratios, multi-AZ failovers, and could use Graviton processors.
- Firmex migrated their Microsoft SQL Server databases from on-premises data centers to Amazon Aurora for PostgreSQL in AWS. They used AWS SCT and AWS DMS tools for this process, which allowed migration of data as well as transformation of stored procedures, DDL, and data isolation. They also refactored and re-targeted T-SQL stored procedures to increase efficiency.
- Firmex engineers created one-shot migration tasks on an as-needed basis using AWS DMS due to exceeding the quota of 600 tasks per account and limits on the number of tasks allowed to create. To work around API limit and bandwidth constraints, custom tools were developed. Migration windows were also carefully analyzed for minimizing downtime and SLA targets.
- Post-migration, Firmex was able to reduce its operating costs by using Amazon Aurora for PostgreSQL in AWS and Aurora’s automatic and fast failovers helped them meet their availability targets and simplify disaster recovery.
- Eric Boyer and Maria Hristova of Firmex, Kiran Singh and Radhika Chakravarty of AWS, and Dylan Souvage and Daniel Rios of Amazon worked collaboratively towards the solution.
- Future improvements include database connection management using RDS Proxy, reducing temporary table usage, auto-vacuum tuning, and using more expressive type systems.
Read Full Article
25 Likes
For uninterrupted reading, download the app