![[The AI Show Episode 142]: ChatGPT’s New Image Generator, Studio Ghibli Craze and Backlash, Gemini 2.5, OpenAI Academy, 4o Updates, Vibe Marketing & xAI Acquires X](https://www.marketingaiinstitute.com/hubfs/ep%20142%20cover.png)
![From drop-out to software architect with Jason Lengstorf [Podcast #167]](https://cdn.hashnode.com/res/hashnode/image/upload/v1743796461357/f3d19cd7-e6f5-4d7c-8bfc-eb974bc8da68.png?#)
.png?#)
.jpeg?#)
.jpg?#)
(1).jpg?width=1920&height=1920&fit=bounds&quality=80&format=jpg&auto=webp#)
_NicoElNino_Alamy.png?#)
_Igor_Mojzes_Alamy.jpg?#)
.webp?#)
.webp?#)
![Blackmagic Design Unveils DaVinci Resolve 20 With Over 100 New Features and AI Tools [Video]](https://www.iclarified.com/images/news/96951/96951/96951-640.jpg)
![Apple Considers Delaying Smart Home Hub Until 2026 [Gurman]](https://www.iclarified.com/images/news/96946/96946/96946-640.jpg)
Fabric & Databricks Interoperability (2): Configuring Hub Storage
Introduction This article provides a detailed guide on how to configure settings for the following use cases: Using tables created in Databricks within Fabric Using tables created in Fabric within Databricks :::note info This article is part of a four-part series: 1. Overview & Purpose of Interoperability 2. Detailed Configuration of Hub Storage (This Article) 3. Using Tables Created in Fabric within Databricks 4. Using Tables Created in Databricks within Fabric ::: Preparing Azure Data Lake Gen2 (ADLS2) as the Hub ① Deploy a storage account in Azure Portal as the hub :::note warn Enable hierarchical namespace. ::: ② Create a container named 'hub' and a directory named 'ext' Connecting Fabric to Hub Storage ① Create a Lakehouse :::note warn Enable Lakehouse schema (public preview). ::: ② Specify the hub storage in the new schema shortcut of the Lakehouse From [Tables] in the Lakehouse, click the three-dot menu and select [New Schema Shortcut]. Select [Azure Data Lake Gen2]. Enter the details for creating a new connection. :::note info How to check ADLS2 access URL: You can confirm it from the storage account's [Endpoints] section under 'Data Lake Storage'. ::: Enable the 'ext' directory and click [Next]. Click [Create]. The 'ext' directory is created as an external schema shortcut. Connecting Databricks to Hub Storage ① Create an access connector for Azure Databricks in the Azure Portal Follow the steps under "Step 1: Create an Access Connector for Azure Databricks" in the "Use Azure Managed Identity to Access Storage in Unity Catalog" guide, using a system-assigned managed identity. ② Grant the connector access to the hub storage from the Azure Portal Follow "Step 2: Grant Managed Identity Access to the Storage Account" in the same guide. ③ Create storage credentials in Databricks Log in to Databricks and navigate to [Catalog] > [+] > [Add Storage Credentials]. Add new storage credentials. | | Input Value | |:-:|:-:| |Storage Credentials or Service Credentials| Storage Credentials | |Credential Name| Any name | |Access Connector ID| Resource ID of the connector created in step ① (can be confirmed in Azure Portal) | After creation, click the newly created credential name. Click [Permissions] > [Grant]. Grant [ALL PRIVILEGES] to necessary users. :::note info Reference for steps and required permissions: Create Storage Credentials for Connecting to Azure Data Lake Storage Gen2 ::: ④ Add an external location in Databricks Log in to Databricks and navigate to [Catalog] > [+] > [Add External Location]. Create a new external location. | | Input Value | |:-:|:-:| |External Location Name| Any name | |URL| abfss://directory-name (hub) @ storage-account-name.dfs.windows.net | |Storage Credentials| Select the credentials created in step ③ | :::note info How to determine the URL: Refer to the storage account's [Endpoints] section used in step ② of "Connecting Fabric to Hub Storage". ::: :::note info Reference for steps and required permissions: Create an External Location to Connect Cloud Storage to Azure Databricks ::: Conclusion Now everything is set up! Next, let's proceed with the actual interoperability of tables. ▽ Next Article FabricとDatabricksの相互運用性③：Fabric で作成したテーブルをDatabricksで利用する（Databrickで閲覧・分析・編集可能） #BI - Qiita はじめにFabricで作成したテーブルをDatabricksでもシームレスに参照・編集が可能できるのか？このような疑問を持つ方も少なくないと思います。そこで今回はFabric で作成したテー… qiita.com ▽ Previous Article FabricとDatabricksの相互運用性①：hubストレージの目的 -Databricks で作成したテーブルをFabricで利用する、Fabricで作成したテーブルをDatabricksで利用する- #Azure - Qiita はじめにそれぞれの特徴がありつつも、大枠ではできることが似ているMicrosoft FabricとDatabricks。Azure Databricks Unity カタログのミラーリングを通… qiita.com
Introduction
This article provides a detailed guide on how to configure settings for the following use cases:
- Using tables created in Databricks within Fabric
- Using tables created in Fabric within Databricks
:::note info
This article is part of a four-part series:
1. Overview & Purpose of Interoperability
2. Detailed Configuration of Hub Storage (This Article)
3. Using Tables Created in Fabric within Databricks
4. Using Tables Created in Databricks within Fabric
:::
Preparing Azure Data Lake Gen2 (ADLS2) as the Hub
① Deploy a storage account in Azure Portal as the hub
:::note warn
Enable hierarchical namespace.
:::
② Create a container named 'hub' and a directory named 'ext'
Connecting Fabric to Hub Storage
① Create a Lakehouse
:::note warn
Enable Lakehouse schema (public preview).
:::
② Specify the hub storage in the new schema shortcut of the Lakehouse
From [Tables] in the Lakehouse, click the three-dot menu and select [New Schema Shortcut].
Select [Azure Data Lake Gen2].
Enter the details for creating a new connection.
:::note info
How to check ADLS2 access URL:
You can confirm it from the storage account's [Endpoints] section under 'Data Lake Storage'.
:::
Enable the 'ext' directory and click [Next].
Click [Create].
The 'ext' directory is created as an external schema shortcut.
Connecting Databricks to Hub Storage
① Create an access connector for Azure Databricks in the Azure Portal
Follow the steps under "Step 1: Create an Access Connector for Azure Databricks" in the "Use Azure Managed Identity to Access Storage in Unity Catalog" guide, using a system-assigned managed identity.
② Grant the connector access to the hub storage from the Azure Portal
Follow "Step 2: Grant Managed Identity Access to the Storage Account" in the same guide.
③ Create storage credentials in Databricks
Log in to Databricks and navigate to [Catalog] > [+] > [Add Storage Credentials].
Add new storage credentials.
| | Input Value |
|:-:|:-:|
|Storage Credentials or Service Credentials| Storage Credentials |
|Credential Name| Any name |
|Access Connector ID| Resource ID of the connector created in step ① (can be confirmed in Azure Portal) |
After creation, click the newly created credential name.
Click [Permissions] > [Grant].
Grant [ALL PRIVILEGES] to necessary users.
:::note info
Reference for steps and required permissions:
Create Storage Credentials for Connecting to Azure Data Lake Storage Gen2
:::
④ Add an external location in Databricks
Log in to Databricks and navigate to [Catalog] > [+] > [Add External Location].
Create a new external location.
| | Input Value |
|:-:|:-:|
|External Location Name| Any name |
|URL| abfss://directory-name (hub) @ storage-account-name.dfs.windows.net |
|Storage Credentials| Select the credentials created in step ③ |
:::note info
How to determine the URL:
Refer to the storage account's [Endpoints] section used in step ② of "Connecting Fabric to Hub Storage".
:::
:::note info
Reference for steps and required permissions:
Create an External Location to Connect Cloud Storage to Azure Databricks
:::
Conclusion
Now everything is set up!
Next, let's proceed with the actual interoperability of tables.
▽ Next Article
FabricとDatabricksの相互運用性③：Fabric で作成したテーブルをDatabricksで利用する（Databrickで閲覧・分析・編集可能） #BI - Qiita
はじめにFabricで作成したテーブルをDatabricksでもシームレスに参照・編集が可能できるのか？このような疑問を持つ方も少なくないと思います。そこで今回はFabric で作成したテー…
qiita.com
▽ Previous Article
FabricとDatabricksの相互運用性①：hubストレージの目的 -Databricks で作成したテーブルをFabricで利用する、Fabricで作成したテーブルをDatabricksで利用する- #Azure - Qiita
はじめにそれぞれの特徴がありつつも、大枠ではできることが似ているMicrosoft FabricとDatabricks。Azure Databricks Unity カタログのミラーリングを通…
qiita.com
