We are excited to announce the Public Preview of Cross-Platform View Sharing. Available today, it allows data providers to share views across different platforms, clouds, and regions, promoting an open and interoperable data ecosystem.
View sharing has been useful; other vendors do it as well. But until now, it’s mostly been limited to the same platform. You could share views within one platform but not across multiple platforms and clouds. Databricks solves this problem with cross-platform view sharing and lets you share views seamlessly across different environments. This is a game changer because it expands data providers’ reach and avoids vendor lock-in for data consumers, making collaboration easier and faster.
Cross-platform sharing aligns with Databricks’ open sharing vision by enabling secure and seamless data exchange across different platforms, clouds, and regions
Understanding View Sharing
To understand view sharing, let’s first understand views. In Databricks, views are read-only representations of data created from tables or other views. They store query text but not the data itself. Views are part of the Unity Catalog
View sharing allows users to share views using the Delta Sharing protocol. Delta Sharing is the industry’s first open protocol for secure data sharing, making it simple to share data with other organizations regardless of which data platforms they use. View sharing promotes reusability and reduces redundancy, as multiple users can access and utilize the same views for analysis.
Previously, when a view was shared between Databricks accounts, consumers could query it using only Databricks Serverless SQL. Databricks Serverless SQL works across all three major cloud providers: AWS, Azure, and Google Cloud Platform (GCP), so views could be shared across clouds.
Now, with cross-platform view sharing, data consumers can leverage any type of Databricks cluster or even utilize open Delta Sharing clients to access and query shared views. Open Delta Sharing clients are tools or platforms that support the Delta Sharing protocol, allowing users to access shared views without needing to use Databricks. These clients include popular systems like Apache Spark™, Pandas, Power BI, Tableau, and others. This makes it possible for users across platforms i.e., who are not on Databricks, to still access and query the shared views via Delta Sharing.
Let’s take a look at this demo to see Cross-Platform View Sharing in action
Use Cases
Databricks to Databricks (D2D) Sharing
In this scenario, two Databricks customers can share views seamlessly within the Databricks ecosystem. Why is this important? Organizations collaborate with partners who may be on different clouds and in different regions and want to share views with clients/partners across clouds and regions. By leveraging Delta Sharing technology, they can seamlessly and securely share views, without making duplicate copies of the data.
Databricks to Open (D2O) Sharing
In this scenario, Databricks customers can share views with external recipients who are not using Databricks. Cross-platform view sharing supports open connectors (such as Apache Spark™, Pandas, Power BI, Tableau, etc.), allowing recipients to access shared views via the Delta Sharing protocol. This capability is particularly beneficial for business analysts and line of Business Users who require simplified access to data without needing to interact directly with complex data platforms
Databricks Marketplace data providers benefit from cross-platform view sharing by significantly expanding their market reach and monetization opportunities. This capability allows them to share views with a wider audience, including clients not using Databricks, thereby increasing their potential customer base. Data Consumers are not limited to querying views from the Databricks Platform, avoiding lockin with Databricks.
Cross-Platform View Sharing is a game-changer for our customers. Bringing zero-copy data sharing to complex enterprises at scale requires flexibility. The ability to share views across platforms enables us to provide the security and performance benefits of Delta Sharing to more customers, helping them unlock value from their customer data faster
— Derek Slager, CTO and Co-founder of Amperity
What’s ahead
In the coming months, readers can expect Databricks to introduce several advanced data-sharing features. The upcoming features include Sharing for Lakehouse Federation, which allows data providers to share data directly from various platforms (e.g., Amazon Redshift, Azure Synapse, Google BigQuery, Snowflake) without the need for replication.
Additionally, D2O OAuth Support will enhance security by enabling recipients to authenticate using OAuth tokens from their trusted Identity Providers (IdPs). Furthermore, the sharing of materialized views and Delta Live Tables will allow for efficient distribution of pre-computed query results and streaming data, providing fresh data with better performance and lower costs.
Getting Started
Cross Platform View Sharing is available in Public Preview today to AWS, GCP and Azure customers. Learn how you can use the Delta Sharing open-sharing protocol to share data from your Unity Catalog-enabled Databricks workspace with any user on any computing platform, anywhere
Source link
lol
