3.4.1.1. Data Migration Strategies (Database Migration Service, Snow Family, DataSync, Storage Gateway)

3.4.1.1. Data Migration Strategies (Database Migration Service, Snow Family, DataSync, Storage Gateway)

💡 First Principle: The optimal data migration strategy is determined by balancing data volume, network bandwidth, downtime tolerance, and data type to ensure an efficient, reliable, and secure transfer to AWS.

Scenario: A large enterprise needs to migrate 500 TB of file shares from their on-premises NAS to "Amazon S3". They have a "Direct Connect" link. The migration needs to be online, automated, and include incremental transfers to keep data synchronized during the migration phase.

Moving large volumes of data from on-premises to AWS is a critical part of many migrations.

• "AWS Database Migration Service (DMS)": A service that migrates relational databases, data warehouses, "NoSQL" databases, and other data stores to AWS.
  • Practical Relevance: Supports homogeneous ("SQL Server" to "SQL Server") and heterogeneous ("Oracle" to "PostgreSQL") migrations. Enables continuous data replication ("CDC") for minimal downtime migrations.
  • Schema conversion and cutover pattern (heterogeneous): DMS moves data but does not convert code. Use the "AWS Schema Conversion Tool (SCT)" (or DMS Schema Conversion) to convert the schema and code objects (e.g., Oracle PL/SQL procedures to PL/pgSQL) and review what it cannot convert automatically. Then run DMS full load plus "CDC" so the source stays live during preparation, use DMS data validation to compare source and target rows, and in the cutover window stop writes, let CDC drain, run a final validation and repoint the application. Leaving the source database intact makes rollback a connection-string change.
• "AWS Snow Family" ("Snowball Edge"): Physical devices for transferring terabytes to petabytes of data into and out of AWS, bypassing internet limitations. (Closed to new customers on November 7, 2025; AWS suggests "DataSync" for online transfer, AWS Data Transfer Terminal for physical uploads, and "AWS Outposts" for edge compute. Snow Family is still on the SAP-C02 in-scope list.)
  • Practical Relevance: Ideal for very large datasets (tens of TBs to PBs) or environments with limited network bandwidth. "Snowball Edge" can also run "EC2 instances" and "Lambda functions" for edge computing.
• "AWS DataSync": An online data transfer service that simplifies, automates, and accelerates moving data between on-premises storage and AWS storage services ("S3", "EFS", "FSx").
  • Practical Relevance: Ideal for large datasets over the internet or "Direct Connect". Automates data transfer, handles encryption, and supports incremental transfers.
• "AWS Storage Gateway": A hybrid cloud storage solution. Connects on-premises environments to cloud storage.
  • Practical Relevance: Provides local caching for frequently accessed data while storing the primary data in "S3" or "EBS". Useful for hybrid file shares, cloud backups, and disaster recovery.

Visual: Data Migration Strategies

Database Migration Tooling and Transfer Details:

• Which database tool: For a heterogeneous move use the "AWS SCT" + "DMS" pair described above (SCT converts the schema and code, DMS moves the data); the same pair takes an Oracle data warehouse to "Redshift". For a homogeneous move (SQL Server to RDS for SQL Server) native backup and restore through S3 is the simplest, most reliable method when a cutover window of hours is acceptable, and "DMS" with "CDC" is the choice when near-zero downtime is required. "MGN" replicates servers, "DataSync" and "Snow Family" move files and objects; none of them migrates database engines.
• DataSync controls: A task can be limited with a bandwidth limit so it does not saturate a shared "Direct Connect" link, verifies data integrity, encrypts in transit, runs on a schedule for incremental syncs, and moves NFS/SMB data to "S3", "EFS" or "FSx" (including FSx for Windows File Server). "Storage Gateway" is for ongoing hybrid access with local caching, not bulk migration.
• When to ship devices: If the link is slow, unreliable or absent and the data would take weeks to move (tens to hundreds of TB), order "Snowball Edge" devices (a fleet for larger sets) instead of pushing it online; "S3 Transfer Acceleration" helps long-distance internet paths, not scarce bandwidth. (SAP-C02 questions still use "Snowball Edge" for this case; see the availability note above.)

⚠️ Common Pitfall: Attempting a large-scale data transfer over an unreliable or low-bandwidth internet connection. For multi-terabyte or petabyte transfers, an online method can take weeks or months and is prone to failure. Offline ("Snow Family") or accelerated ("DataSync" over "Direct Connect") methods are often required.

Key Trade-Offs:

• Online vs. Offline Transfer: Online transfers ("DataSync", "DMS") can offer minimal downtime but are limited by network bandwidth. Offline transfers ("Snow Family") are ideal for massive volumes regardless of bandwidth but involve shipping time and a longer cutover process.

Reflection Question: Which AWS service would be the most suitable for migrating 500 TB of file shares from an on-premises NAS to "Amazon S3" given the requirements for an online, automated, and incremental transfer with a "Direct Connect" link? How does this choice balance data volume, network bandwidth, and downtime tolerance?