3.2.4. Mirroring: Database vs. Metadata
💡 First Principle: Mirroring replicates external data into Fabric, but what gets replicated depends on the source. Database mirroring copies actual data rows; metadata mirroring copies only the catalog information (table names, schemas). Understanding this distinction is critical for choosing the right approach.
Scenario: Your organization uses Azure SQL Database for operational systems and Databricks for data science. You want both available in Fabric without duplicating data unnecessarily.
Mirroring Types
| Type | What's Replicated | Use Case |
|---|---|---|
| Database Mirroring | Actual data (CDC-based) | Azure SQL, Cosmos DB, Snowflake |
| Metadata Mirroring | Catalog information only | Azure Databricks Unity Catalog |
Mirroring Selection
- Azure Cosmos DB: Database mirroring (replicates documents)
- Azure SQL Database: Database mirroring (replicates tables)
- Azure Databricks: Metadata mirroring (replicates catalog, not data)
Database Mirroring: How It Works
Database mirroring uses Change Data Capture (CDC) to continuously replicate changes:
Source Database → CDC captures INSERT/UPDATE/DELETE → Fabric syncs changes → OneLake Delta tables
| Aspect | Behavior |
|---|---|
| Initial sync | Full snapshot of selected tables |
| Ongoing sync | Near real-time CDC replication |
| Data format | Converted to Delta tables in OneLake |
| Latency | Typically seconds to minutes |
| Schema changes | Propagated automatically |
| Writability | Mirrored tables are read-only; to reshape, partition or OPTIMIZE them, materialize into a lakehouse or warehouse table (CTAS, notebook) |
- Change capture for Azure SQL Database: the source engine scans its own transaction log and publishes changes to a OneLake landing zone. This is a change feed, so you don't enable Change Tracking or CDC yourself; check progress with
sys.dm_change_feed_log_scan_sessions. The server's managed identity must keep Read/Write on the mirrored item, or no new data is mirrored. - Setup: enable the Azure SQL server's managed identity and grant permissions → create a Mirrored Azure SQL Database item with the connection (Entra ID or SQL auth) → mirror all tables or selected ones → monitor replication. Query through the autogenerated SQL analytics endpoint, Spark or Direct Lake.
- Read-only replica: the replicator owns all writes. You can create views, but you can't INSERT/UPDATE, OPTIMIZE or VACUUM mirrored tables.
- Mirroring vs. a scheduled Copy pipeline: mirroring replicates continuously with no schedule or pipeline to maintain; a Copy pipeline moves a batch on each run and can filter or reshape data on the way.
Metadata Mirroring: How It Works
Metadata mirroring syncs the catalog without copying data:
Databricks Unity Catalog → Fabric syncs table/view definitions → Fabric reads the Delta files in place via shortcuts
| Aspect | Behavior |
|---|---|
| What syncs | Table names, schemas, locations |
| What doesn't sync | Actual data rows |
| Query execution | Fabric engines (SQL analytics endpoint, Spark, Direct Lake) read the data through OneLake shortcuts |
| Benefit | Unified catalog without data duplication |
When to Use Which
| Scenario | Mirroring Type | Why |
|---|---|---|
| Need data in Fabric for fast queries | Database mirroring | Data is local |
| Want unified catalog, data stays in place | Metadata mirroring | Avoids duplication |
| Source is Azure SQL/Cosmos | Database mirroring | Only option available |
| Source is Databricks | Metadata mirroring | Only option available |
⚠️ Exam Trap: Databricks uses metadata mirroring, not database mirroring. The data stays in Databricks storage; only the Unity Catalog metadata is synchronized to Fabric. Questions that mention "mirroring Databricks data" are testing whether you understand this distinction.
⚠️ Common Pitfall: Expecting instant freshness with metadata mirroring. Only the catalog is synced; the data is read in place through shortcuts, and changes made in Databricks can take seconds to minutes to appear in Fabric.
Reflection Question: Your company wants to use data from both Azure Cosmos DB (operational data) and Azure Databricks (ML features). What mirroring type would you use for each?
