By Jason Gregson & Artem Shchodro
Introduction
Here at DoiT we see databases come in all sorts of shapes and sizes and we support our customers with both the technical support, but also understanding what databases to use where.
DoiT recognises that databases are a hugely complex and important element to any business which is why we have taken the approach to have a distinction between Databases and everything else - ie Core. Whilst there is quite a lot of overlap between the Core and Database engineering, there are some very specific disciplines around Databases that are not covered as a part of Core Engineering and therefore all discussions around databases and Database Migrations Service are handled by the Database specialist team.
There is no one size fits all and the same is true for databases. As the customer grows, the requirements change. The business can change the model or the way that data is collected, so what was true at the start, may not be the case later on in the future. The same goes for where the databases reside, on premises or in the cloud.
As most applications need to store data in some form of database, databases become the crown jewels in the customer platform. Navigating the changing landscape is time consuming and there are real technical challenges to head off. Amazon Web Services (AWS) provides several services/tools to help you migrate your databases as well as the applications that use the databases, to the cloud. You can change their database management system (DBMS) or even their architecture from SQL to NoSQL, depending on the requirements and the aspirations of the business.
It is fairly simple to migrate data from one server to another when both are using the same database engine, known as homogeneous migration. For example Microsoft SQL Server from an on premises to the same Microsoft SQL Server running in Amazon Relational Database Service (Amazon RDS) is more about making sure that all the data/schema’s are replicated and then ensuring the connectivity and security are correct to allow the application can connect to the “new” database. The real complexity is when there are changes, even what seems like small changes like version number can unearth a whole host of elements that need to be addressed. Then moving to a different engine, or heterogeneous migration, is harder and may require changes in your approach and application.
This article covers a high level overview of two database tools: AWS Schema Conversion Tool (AWS SCT) and AWS Database Migration Service (AWS DMS).
Why should I migrate my database to AWS?
Today’s applications come with all sorts of challenges that vary in complexity, such as large data volumes, high transactional throughput requirements, complex data types and business logic as well as proprietary vendor features. Migrating these workloads to the cloud is a goal for many Amazon Web services customers for a number of reasons, including reducing costs by providing easier automation and elasticity while lowering license costs with less vendor lock-in on some of the more proprietary DBMS’ (Microsoft SQL/Oracle/SAP). This also has the ability to unlock the innovation potential by decoupling the data systems from the application in a modern cloud centric way. Migration can also help modernize data infrastructure with cloud-only features like the Amazon Aurora storage engine. These are not steps to be taken without a lot of due diligence and technical expertise and this is where DoiT can be your first port of call.
Here are some common reasons to migrate a database
Database modernisation
Moving data from legacy database engines to modern database engines, enabling business agility to compete in the digital age.
Database migration
Moving data from one platform to another, in the context of enterprise applications.
Database replication
Frequently copying data from a database in one computer/server to a database in another to share the same level of information with all user
What are the benefits of migrating a database with AWS?
Relational databases are a mature technology that is widely understood, enabling you to routinely manage high levels of availability with on-premises servers. Although NoSQL databases are newer, we can usually maintain high availability on-premises; however, any database downtime can severely impact both application downtime and a company's ability to serve its customers in a timely manner. This has a detrimental reputational effect and is difficult to recover from, so by ensuring that the business and technical requirements are fully covered off in our conversations, we can ensure that we minimize these and other risks around the migration/s.
AWS database technologies, such as Amazon Relational Database Service (Amazon RDS) and Amazon DynamoDB, provide low-cost, scalable, highly available database solutions that are simple to set up and use. For many companies, migrating their databases to the cloud (particularly if switching to a different engine) would lower downtime and cost compared to on-premises servers.
Simple to use
AWS Database Migration Service (AWS DMS) is easy to use. You do not need to install drivers or applications, and it typically does not require changes to the source database.
Minimal downtime
AWS DMS helps you migrate your databases to AWS with virtually no downtime.
Supports widely used database engines
AWS DMS can migrate your data to and from most widely used commercial and open-source database engines.
Low cost
AWS DMS can migrate your data to and from most widely used commercial and open-source database engines.
Fast and easy to set up
You can set up a migration task within minutes in the AWS Management Console. This lets you define parameters used by Amazon DMS to execute the migration.
Reliable
AWS DMS is highly resilient and self-healing. It continually monitors source and target databases, network connectivity, and the replication instance.
How do I migrate my database to AWS?
AWS DMS and AWS Schema Conversion Tool (AWS SCT) can help you migrate your databases to or from AWS quickly and securely. Using AWS DMS, your source database remains fully operational during migration, minimizing downtime to applications that rely on the database.
You can migrate to the same database engine, or switch database engines to modernize your database platform. You can also replicate data to sync source and target databases.
For example:
You can migrate from an on-premises Oracle instance to an Amazon RDS for Oracle instance.
Or...
You can migrate an Oracle instance, either on-premises or in the cloud, to a PostgreSQL database like Amazon RDS for PostgreSQL or Amazon Aurora PostgreSQL.
What tools can help with my migration?
AWS DMS
At its most basic level, AWS DMS is a server in the AWS Cloud that runs the replication software. You create a source and target connection to tell AWS DMS where to extract from and load to. Then you schedule a task that runs on this replication partner server to receive a copy of your data.
AWS DMS creates the tables and associated primary keys if they don't exist on the target. You can pre-create the target tables manually, if you prefer. Or you can use AWS SCT to create some or all of the target tables, indexes, views, triggers, and so on.
AWS SCT
If you want to switch database engines, AWS SCT can convert your existing database schema to the target platform. This includes tables, indexes, views, and stored procedures, plus your application code. If the schema from your source database can't be converted automatically, AWS SCT provides guidance on how you can create an equivalent schema in your target database engine.It is important to understand what the issues are with regards to not be able to convert automatically as these will have a impact on both the resulting database but also how the current application will interact with the database.Again this is where DoiT’s experience comes in and we would be able to help understand the elements that are in play and what would need to be considered in the changes required.
How do I manage a migration project?
How do AWS DMS and AWS SCT fit into your migration project? AWS suggests to structure a migration into 12 steps. As not all of the steps are required to every migration project— DoiT International assigns a migration specialist from the Database Engineering department to every customer project to assess your particular use case. Based on the initial assessments and the characteristics of your application, database, infrastructure, available resources, and skill sets DoiT International will build a migration plan together with you.
This element of the migration will start to uncover elements that are not part of the physical migration, but more focused on the business objectives required - What does good look like? For example “We are currently using Database X and are going to grow by x1000 in the next 12 months”. This has all sorts of connotations that need to be understood, and maybe a homogeneous migration is not the best approach to solving the huge scaling requirements from the business.
What's next?
If you represent the engineering elements of the business who want to learn how to use the tooling for the next migration project, you can sign up for the free introduction course from AWS to learn more. This training covers: schema migration, data migration, and training and support. It will give you a deeper understanding of what is involved and demystify the technical elements for the migration itself. This however does not give you the experience to understand the nuances of a Database Migration, but that is where DoiT would be able to help and give you the support you need to be successful.
If you want to access the possible migration - . DoiT International helps customers to migrate their workload to the cloud and also provides ongoing support. Learn more about our approach to migrations and how DoiT International can help you.
Q: How much will all this cost?
The best part of this is that DoiT has a unique pricing model for helping our customers. This includes the approaches to Database Migrations to AWS. We do the work for zero added cost; there are no additional costs associated with migrating your workloads to the cloud.
Q: Where is the catch?
There is none. You need to pay for your AWS bill through DoiT as a reseller – and that’s it. We are an addition to the services and support you get directly from Amazon; we take nothing away from the relationship except for the payment of AWS billing, which is done directly through DoiT.Given the zero cost, you can appreciate that DoiT will not be carrying out the database migration itself, but we will act in an advisory capacity to ensure that you are getting the right advice at the right time with regards to the various stages of the migration. You will need to complete the work, or you can use a third party to execute it.In addition, you will get all the usual benefits of being a DoiT customer including:
Unlimited support (24x7 - Follow the sun)
Zero cost training
Zero cost access to DoiT Cloud Management platform (CMP)
Zero cost Well Architected Framework reviews (WAfR)
DoiT’s Flexsave (flexible savings)
DoiT’s Spot0 (flexible Spot pricing)
Jason
Jason Gregson is a Global Head of AWS Programs and Operations for DoiT international. After many years as an Solution Architect including 5 years at Amazon he is passionate about helping customers to build modern applications on latest AWS services. A pragmatic technologist and engineer at heart, he loves community-driven learning and sharing of technology to ensure that everyone knows the Art of the possible. Follow him on Twitter at @JasonGregson.
Artem
Artem Shchodro is a Senior Cloud Architect at DoiT international. He is an ex-Amazonian helping customers get the best value from the cloud. Artem is pragmatic and straightforward. Blending the business value with technology. Don’t follow him on Twitter.
By Jason Gregson & Artem Shchodro
Introduction
Here at DoiT we see databases come in all sorts of shapes and sizes and we support our customers with both the technical support, but also understanding what databases to use where.
DoiT recognises that databases are a hugely complex and important element to any business which is why we have taken the approach to have a distinction between Databases and everything else - ie Core. Whilst there is quite a lot of overlap between the Core and Database engineering, there are some very specific disciplines around Databases that are not covered as a part of Core Engineering and therefore all discussions around databases and Database Migrations Service are handled by the Database specialist team.
There is no one size fits all and the same is true for databases. As the customer grows, the requirements change. The business can change the model or the way that data is collected, so what was true at the start, may not be the case later on in the future. The same goes for where the databases reside, on premises or in the cloud.
As most applications need to store data in some form of database, databases become the crown jewels in the customer platform. Navigating the changing landscape is time consuming and there are real technical challenges to head off. Amazon Web Services (AWS) provides several services/tools to help you migrate your databases as well as the applications that use the databases, to the cloud. You can change their database management system (DBMS) or even their architecture from SQL to NoSQL, depending on the requirements and the aspirations of the business.
It is fairly simple to migrate data from one server to another when both are using the same database engine, known as homogeneous migration. For example Microsoft SQL Server from an on premises to the same Microsoft SQL Server running in Amazon Relational Database Service (Amazon RDS) is more about making sure that all the data/schema’s are replicated and then ensuring the connectivity and security are correct to allow the application can connect to the “new” database. The real complexity is when there are changes, even what seems like small changes like version number can unearth a whole host of elements that need to be addressed. Then moving to a different engine, or heterogeneous migration, is harder and may require changes in your approach and application.
This article covers a high level overview of two database tools: AWS Schema Conversion Tool (AWS SCT) and AWS Database Migration Service (AWS DMS).
Why should I migrate my database to AWS?
Today’s applications come with all sorts of challenges that vary in complexity, such as large data volumes, high transactional throughput requirements, complex data types and business logic as well as proprietary vendor features. Migrating these workloads to the cloud is a goal for many Amazon Web services customers for a number of reasons, including reducing costs by providing easier automation and elasticity while lowering license costs with less vendor lock-in on some of the more proprietary DBMS’ (Microsoft SQL/Oracle/SAP). This also has the ability to unlock the innovation potential by decoupling the data systems from the application in a modern cloud centric way. Migration can also help modernize data infrastructure with cloud-only features like the Amazon Aurora storage engine. These are not steps to be taken without a lot of due diligence and technical expertise and this is where DoiT can be your first port of call.
Here are some common reasons to migrate a database
Database modernisation
Moving data from legacy database engines to modern database engines, enabling business agility to compete in the digital age.
Database migration
Moving data from one platform to another, in the context of enterprise applications.
Database replication
Frequently copying data from a database in one computer/server to a database in another to share the same level of information with all user
What are the benefits of migrating a database with AWS?
Relational databases are a mature technology that is widely understood, enabling you to routinely manage high levels of availability with on-premises servers. Although NoSQL databases are newer, we can usually maintain high availability on-premises; however, any database downtime can severely impact both application downtime and a company's ability to serve its customers in a timely manner. This has a detrimental reputational effect and is difficult to recover from, so by ensuring that the business and technical requirements are fully covered off in our conversations, we can ensure that we minimize these and other risks around the migration/s.
AWS database technologies, such as Amazon Relational Database Service (Amazon RDS) and Amazon DynamoDB, provide low-cost, scalable, highly available database solutions that are simple to set up and use. For many companies, migrating their databases to the cloud (particularly if switching to a different engine) would lower downtime and cost compared to on-premises servers.
Simple to use
AWS Database Migration Service (AWS DMS) is easy to use. You do not need to install drivers or applications, and it typically does not require changes to the source database.
Minimal downtime
AWS DMS helps you migrate your databases to AWS with virtually no downtime.
Supports widely used database engines
AWS DMS can migrate your data to and from most widely used commercial and open-source database engines.
Low cost
AWS DMS can migrate your data to and from most widely used commercial and open-source database engines.
Fast and easy to set up
You can set up a migration task within minutes in the AWS Management Console. This lets you define parameters used by Amazon DMS to execute the migration.
Reliable
AWS DMS is highly resilient and self-healing. It continually monitors source and target databases, network connectivity, and the replication instance.
How do I migrate my database to AWS?
AWS DMS and AWS Schema Conversion Tool (AWS SCT) can help you migrate your databases to or from AWS quickly and securely. Using AWS DMS, your source database remains fully operational during migration, minimizing downtime to applications that rely on the database.
You can migrate to the same database engine, or switch database engines to modernize your database platform. You can also replicate data to sync source and target databases.
For example:
You can migrate from an on-premises Oracle instance to an Amazon RDS for Oracle instance.
Or...
You can migrate an Oracle instance, either on-premises or in the cloud, to a PostgreSQL database like Amazon RDS for PostgreSQL or Amazon Aurora PostgreSQL.
What tools can help with my migration?
AWS DMS
At its most basic level, AWS DMS is a server in the AWS Cloud that runs the replication software. You create a source and target connection to tell AWS DMS where to extract from and load to. Then you schedule a task that runs on this replication partner server to receive a copy of your data.
AWS DMS creates the tables and associated primary keys if they don't exist on the target. You can pre-create the target tables manually, if you prefer. Or you can use AWS SCT to create some or all of the target tables, indexes, views, triggers, and so on.
AWS SCT
If you want to switch database engines, AWS SCT can convert your existing database schema to the target platform. This includes tables, indexes, views, and stored procedures, plus your application code. If the schema from your source database can't be converted automatically, AWS SCT provides guidance on how you can create an equivalent schema in your target database engine.It is important to understand what the issues are with regards to not be able to convert automatically as these will have a impact on both the resulting database but also how the current application will interact with the database.Again this is where DoiT’s experience comes in and we would be able to help understand the elements that are in play and what would need to be considered in the changes required.
How do I manage a migration project?
How do AWS DMS and AWS SCT fit into your migration project? AWS suggests to structure a migration into 12 steps. As not all of the steps are required to every migration project— DoiT International assigns a migration specialist from the Database Engineering department to every customer project to assess your particular use case. Based on the initial assessments and the characteristics of your application, database, infrastructure, available resources, and skill sets DoiT International will build a migration plan together with you.
This element of the migration will start to uncover elements that are not part of the physical migration, but more focused on the business objectives required - What does good look like? For example “We are currently using Database X and are going to grow by x1000 in the next 12 months”. This has all sorts of connotations that need to be understood, and maybe a homogeneous migration is not the best approach to solving the huge scaling requirements from the business.
What's next?
If you represent the engineering elements of the business who want to learn how to use the tooling for the next migration project, you can sign up for the free introduction course from AWS to learn more. This training covers: schema migration, data migration, and training and support. It will give you a deeper understanding of what is involved and demystify the technical elements for the migration itself. This however does not give you the experience to understand the nuances of a Database Migration, but that is where DoiT would be able to help and give you the support you need to be successful.
If you want to access the possible migration - . DoiT International helps customers to migrate their workload to the cloud and also provides ongoing support. Learn more about our approach to migrations and how DoiT International can help you.
Q: How much will all this cost?
The best part of this is that DoiT has a unique pricing model for helping our customers. This includes the approaches to Database Migrations to AWS. We do the work for zero added cost; there are no additional costs associated with migrating your workloads to the cloud.
Q: Where is the catch?
There is none. You need to pay for your AWS bill through DoiT as a reseller – and that’s it. We are an addition to the services and support you get directly from Amazon; we take nothing away from the relationship except for the payment of AWS billing, which is done directly through DoiT.Given the zero cost, you can appreciate that DoiT will not be carrying out the database migration itself, but we will act in an advisory capacity to ensure that you are getting the right advice at the right time with regards to the various stages of the migration. You will need to complete the work, or you can use a third party to execute it.In addition, you will get all the usual benefits of being a DoiT customer including:
Unlimited support (24x7 - Follow the sun)
Zero cost training
Zero cost access to DoiT Cloud Management platform (CMP)
Zero cost Well Architected Framework reviews (WAfR)
DoiT’s Flexsave (flexible savings)
DoiT’s Spot0 (flexible Spot pricing)
Jason
Jason Gregson is a Global Head of AWS Programs and Operations for DoiT international. After many years as an Solution Architect including 5 years at Amazon he is passionate about helping customers to build modern applications on latest AWS services. A pragmatic technologist and engineer at heart, he loves community-driven learning and sharing of technology to ensure that everyone knows the Art of the possible. Follow him on Twitter at @JasonGregson.
Artem
Artem Shchodro is a Senior Cloud Architect at DoiT international. He is an ex-Amazonian helping customers get the best value from the cloud. Artem is pragmatic and straightforward. Blending the business value with technology. Don’t follow him on Twitter.
