Published: 22 Nov 2023 | Author: James Beresford

One of the latest announcements for Fabric is mirroring. Combined with the existing Shortcuts feature, this is another step towards making Fabric an all encompassing cloud data virtualisation platform. Let's have a quick run through these topics.

Data Virtualisation

One of the challenges with trying to build out a complete data estate is that data is rarely all in one location. The larger the organisation, the more complex the landscape will be, with multiple clouds, warehouses and lakes in play.

The Data Virtualisation paradigm is one where the analytical layer doesn't really care where the data is located. By using its own repository of metadata (data about the data) it can perform analytics on data regardless of where it is physically located.

This is of course a huge simplification of the concept, but it is clearly where Microsoft is trying to take the Fabric platform.

Mirroring

The Mirroring feature is conceptually similar to the replication capability that's been available in SQL server (and most other enterprise database platforms) for some time.

For supported systems - currently Azure Cosmos DB, Azure SQL DB and Snowflake, and coming are SQL Server, Azure PostgreSQL, Azure MySQL & MongoDB - you effectively get a live feed of data from these systems to OneLake. This is leveraging the Change Data Capture (CDC) capabilities of the source system, so we can assume some source side configuration is required.

Shortcuts

Shortcuts are links to other cloud storage (Currently AWS S3 Buckets or ADLSGen2, more to come) that allow Fabric to extract data from these as if they were part of OneLake.

A key difference between this and Mirroring will be performance - depending on the location of the data it may have some network lag to accomodate.

Summary

Fabric is gearing up to provide a very flexible platform for performing analytics by simplifying the process of accessing source data. Some of these solutions will not be a substitute for a properly managed data lake, but they will facilitate the analysis required to build and model data at a much lower cost. The mirroring and shortcut features are great accelerators for performing analytics.

Don't forget to join us at the Official Microsoft Sydney Fabric Meetup to keep on top of all things Fabric!

Cheers, James

CEO, Talos

Subscribe

Get the latest Talos Newsletter delivered directly to your inbox

TECHNOLOGY PARTNERS

Our partners including Microsoft, UiPath, Databricks & Profisee enable us to deliver business outcomes using best of breed technologies and solutions.

Automation & Analytics Technologies for Business

Our Solutions

Enterprise PowerBI
Enterprise PowerBI
Self Service Analytics

Enable self service analytics to meet the needs of the whole organisation with our proven methodologies.

Automation Initiation
Automation Initiation
Your automation journey

Using our EPIC methodology guiding you to deliver outcomes quickly and cost effectively.

Modern Data Platform
Modern Data Platform
Build Data components

Build, test and implement Data Platform components - secure, efficient, flexible and cost effective.

CASSIE
CASSIE
Customer Assistance Expert

Specialising in all customer-related processes, she has been trained to quickly learn specific customer processes.

Click here for more