Migrating from AWS RDS MySQL to AWS Aurora Serverless MySQL Database
Database Management Woes
Taking away some of the responsibilities of database administration requires automating and removing complexities from every day DBA (Database Admin) work. AWS RDS did that a few years ago by providing users with a way to launch database instances without having to deal with the underlying operating system to host the instance. The ease of creating and automating backups were built into RDS along with other administrative tasks that would normally be handled by a database expert.
Automatic software patching, analyzing metrics, restoring snapshots/backups were all integrated functions that didn’t require specialized knowledge to manage. Recommendations would also be made when storage would reach limits, performance was impacted, or a DB engine version needed to be upgraded.
AWS RDS Instance vs Aurora Serverless
The limitations of RDS and other managed infrastructure run into are scalability, redundancy, and maintenance windows. Our corporate website has been running on RDS for a while now without any issues and no downtime since we have a multi-zone deployment. We had purchased reserved instances to cut back on costs but our main cost driver had been the need to statically set resource requirements for the instance (RAM, CPU, storage, etc). Although growth of storage was minimal for our database, other applications and organizations would be limited by storage constraints in RDS.
The whole concept of serverless computing is that infrastructure requirements can be met without thinking about servers and instead focusing on application functionality and results. On the backend of the cloud provider, in our case AWS, all of the infrastructure (CPU, memory, storage, redundancy, etc.) is handled for the consumer usually with very predictable responses and service levels.
One of Amazon’s database engines supported include Aurora, a MySQL & PostgreSQL compatible database that offers high availability and performance for a lower cost and more cloud focused infrastructure than traditional enterprise database engines. Amazon now offers the same Aurora DB engine without having to manage instances in a serverless environment. Consumers are billed on a per-second basis and when the DB engine is idle for some time, billing and the DB serverless instance is paused. Providing a low cost and easily deployable database solution for unpredictable workloads.
Migrating existing RDS instance to Aurora Serverless
The database we were running on AWS was an RDS instance (db.t2.medium) with support for MySQL 5.6.x. We had purchased 2 reserved instances for multi-zone redundancy and after the initial setup never had to perform any maintenance or upgrade the database. Our website is for the most part static, so data storage growth and capacity were not really a concern. Our focus when looking into serverless options was to lower the cost and enhance availability.
Running a standard RDS MySQL DB presents a few challenges when trying to migrate to Aurora Serverless. Those challenges being:
- Simple path to migrate schemas to Aurora
- Downtime and compatibility
- Reconfigure web instance(s) to connect to new cluster(s)
Our initial migration effort involved taking down the application/database, taking a snapshot of the instance, and trying to restore it into Aurora Serverless. AWS does not allow this migration path due to differences in configuration and schemas for the database engine.
Our next option was to create a Read Replica of the RDS database instance, allow replication to take place (essentially no time for our database since it was small), and redirect requests to the read replica after promoting it to a stand-alone, Aurora Master DB instance. Going this route did not allow us to go directly to an Aurora Serverless read-replica so we decided to go for plan C.
Read the AWS documentation on “Migrating Data from a MySQL DB Instance to an Amazon Aurora MySQL DB Cluster by Using an Aurora Read Replica”
Plan C was to use mysqldump from our application web server instance and create a .sql file that we could import into an Aurora Serverless instance. We chose this option as the best alternative since we were OK with minimal downtime.
mysqldump -h <RDS Endpoint> -u <username> -p db_schema_name > db_schema_name.sql
Create a new DB cluster to support Aurora MySQL Serverless with the same platform version of MySQL. Follow the documentation provided by AWS HERE.
After successfully creating backup files for each database schema you want to migrate to your new MySQL Serverless cluster, you can proceed to restore the SQL backups using the following command.
mysql -h <Aurora Serverless Cluster Endpoint> -u <username> -p <database destination> < backupfile.sql
Update your application to point to the new database cluster endpoint for Aurora Serverless, and start your web service/application. That’s it! Your new serverless backend for your application should be up and running.
Monitoring & Metrics
Once you let your system run for some time and you can build up a monitoring history, you can validate the usage and serverless capacity by analyzing “Serverless Database Capacity”, “CPU Utilization”, and “DB Connections”.
As you can see, our database cluster has been at 2 capacity units for the past week with varied CPU utilization and DB connections. As we analyze the data over the course of a month or so, it might make sense for use to go to a static Aurora RDS instance with a smaller instance than we previously had with standard RDS. Migrating to an Aurora RDS static instance is as simple as creating a database snapshot and restoring it to a static Aurora RDS instance. The only downtime would be the time it takes to change settings in your application to point to the new cluster.
The release of AWS Aurora Serverless has helped alleviate the constraint of scaling on demand and the need to have redundant RDS instances running in other availability zones as a replica. Not all use cases would apply for a Serverless Aurora instance and might be better suited for dev/test environments where database capacity would be utilized less than a production environment.
In terms of price comparisons, AWS does a good job of providing examples how ACU units and changes in IO/storage can affect overall charges. Scroll down to pricing examples HERE.
If your organization would like to explore how you can save on direct and indirect costs managing databases for development and production environments, contact us for a free consultation.