European Fabric Community Conference 2024 Announcements

By:   |   Updated: 2024-10-18   |   Comments   |   Related: > Microsoft Fabric


Problem

The inaugural European Microsoft Fabric Community Conference 2024 was held in Stockholm, Sweden from September 24 to 27, 2024. During this conference, 120 sessions, 3 keynotes, and 10 workshops were given by expert speakers, both from the community and Microsoft. As is typical for such big conferences hosted by Microsoft, many interesting announcements were made about the future of the Fabric data platform. This tip provides you with an overview.

Solution

The European Microsoft Fabric Community Conference 2024 (FabCon) conference is – just like its North American counterpart – a paid conference. Unlike Ignite and Microsoft Build, there are no recordings of the sessions publicly available. If you were not an attendee of the conference, the only information available is from official blog posts and announcements or from data community blog posts. In this tip, we will give you a comprehensive overview of Fabric news, but the list is not exhaustive and will not contain all the details. If possible, we will refer you to external sources for extra reading.

Microsoft Fabric Announcements

Here are some of the announcements from the conference.

Copilot & AI

It wouldn't be a Microsoft conference in this day and age if there wasn't some mention of AI and Copilot integration into the Fabric platform. An interesting update is the support of Copilot conversations in the Real-time Intelligence workflow, which is now in public preview. This feature allows you to translate natural language into KQL statements, for example, shown in the following screenshot from the Microsoft blog post:

copilot and KQL

(Source: Unlock faster insights with the new support for Copilot conversations in Real-Time Intelligence (Public Preview))

The YouTube video, AI-powered development with Copilot in Fabric, gives a nice demo of what is possible with Copilot (and git integration) in Fabric today. You can also read the following blog post for more information: European Fabric Community Conference 2024: Building an AI-powered data platform.

Also, some very important news is that there will be a 50% price reduction for Copilot in Fabric starting November 1, 2024. At the same time, prebuilt AI services and the AI Skill feature (both in public preview at the time of writing) will start billing from the same date. You can find more information in the blog post: Important billing update coming to Copilot and AI in Fabric.

Data Factory

Here is an overview of the announcements made:

incremental refresh for dataflows gen2

(Source: Recap of Data Factory Announcements at Fabric Community Conference Europe)

  • Fabric pipelines now have the new Azure Data Factory (ADF) item. With this feature, you can bring your existing ADF environment into Fabric. You can trigger, execute, and monitor your ADF pipelines directly from within Fabric. Andy Leonard has blogged about this new feature in his post Mount an existing Azure Data Factory in Fabric Data Factory. It's now also possible to invoke remote pipelines from ADF or Synapse from the Fabric Invoke Pipeline activity.
  • To make Fabric pipelines more user-friendly and reduce the learning curve for the Copy activity, the Copy Job item was introduced. With this new item, you can quickly generate full copy or incremental pipelines. You can learn more about it in the documentation.
  • At Microsoft Build 2024, the feature Data Functions was announced. You can now execute a Data Function as an activity in a pipeline. This feature is in private preview. You can find more information in the blog post Transform, Validate and Enrich Data with Python User Data Functions in Your Data Pipelines.
  • When you execute Fabric Notebooks from a pipeline, it's now possible to share sessions between the notebooks by using the session tags feature. In combination with the new high concurrency mode for Notebooks, you can achieve big performance improvements.

Warehouse Improvements

For the warehouse service in Fabric, several announcements were made as well:

  • The availability of VARCHAR(MAX) is in private preview. Previously, the warehouse supported only VARCHAR(n), with n being a number between 1 and 8,000. With the MAX size specification, you will be able to store up to 1MB per cell (according to this blog post). This might be useful when dealing with large strings - such as JSON files - or with dynamic SQL. In other versions of SQL Server, you can store up to 2GB. The official documentation doesn't seem to mention the shorter limit for the Fabric warehouse.
  • Although it was announced prior to FabCon, it is worth mentioning here: TRUNCATE TABLE is now also available for the warehouse.
  • The warehouse has also improved support for JSON. You can now use the FOR JSON clause and the same JSON scalar functions as in Azure SQL DB. You can find more information in the blog post Announcing improved JSON support in Fabric DW.
  • The editor for the warehouse and the SQL Analytics Endpoint is improved. The ribbon has been altered, and you can now have multiple tabs with different types of content, such as a SQL script, a data preview, and a semantic model, as shown in this screenshot below. More info can be found here.
tabs with different content
  • For people who like working with notebooks: Fabric now supports a T-SQL notebook for developing your warehouse. This feature is in public preview.
t-sql notebook
  • Nested common table expressions (CTEs) are in public preview. You can learn more about them here and here. Statements like these should be possible in Fabric (at the time of writing this feature was however not available in my region):
WITH cte_outer AS
(
   WITH cte_inner AS
   (
       SELECT 1 AS MyIntColumn
   )
   SELECT * FROM cte_inner
)
SELECT * FROM cte_outer;

Data Engineering Improvements

Other Announcements

  • Mirroring for Snowflake is now in Generally Available. Several announcements were made for Mirroring for Azure SQL DB, such as REST API support, the ability to mirror tables without a primary key, and connectivity to databases behind firewall or private endpoints. More info can be found in the post, Mirroring Azure SQLDB – new features and what's coming up?
  • There is a new Terraform provider for Fabric, allowing you to write Infrastructure as Code (IaC) for your Fabric solution.
  • There will be a Fabric SKU calculator, which is now in private preview. This will help to estimate the size of your Fabric capacities.
  • An important announcement for everyone who wants to automate processes using the REST APIs: service principal support is now available. Be sure to check the list to see which REST API endpoints are already supported.
  • You can now set Data Activator alerts on KQL querysets. Read more about it here. For other Data Activator improvements, check out the blog post Announcing Updates to Data Activator in Public Preview.
  • Similar to Azure, you can create tags in Fabric. By adding tags as metadata to assets, it will be easier to categorize and organize your Fabric environment. Learn more about them in the blog post Tag your data to enrich item curation and discovery.
Next Steps


sql server categories

sql server webinars

subscribe to mssqltips

sql server tutorials

sql server white papers

next tip



About the author
MSSQLTips author Koen Verbeeck Koen Verbeeck is a seasoned business intelligence consultant at AE. He has over a decade of experience with the Microsoft Data Platform in numerous industries. He holds several certifications and is a prolific writer contributing content about SSIS, ADF, SSAS, SSRS, MDS, Power BI, Snowflake and Azure services. He has spoken at PASS, SQLBits, dataMinds Connect and delivers webinars on MSSQLTips.com. Koen has been awarded the Microsoft MVP data platform award for many years.

This author pledges the content of this article is based on professional experience and not AI generated.

View all my tips


Article Last Updated: 2024-10-18

Comments For This Article

















get free sql tips
agree to terms