Qlik and dbt Labs Make Big Data Integration Acquisitions

Data never stands still, which makes data integration a perennial challenge. That’s ultimately what drove Qlik and dbt Labs today to announce strategic data integration acquisitions, including Qlik’s purchase of Upsolver, a provider of a no-code data ingestion environment for lakehouses and Apache Iceberg, and dbt Labs’ acquisition of SDF Labs and its real-time SQL validation tool.

While Qlik is known mainly for its business intelligence, analytics, and ML/AI tools, the company also has a significant stable of data integration tools that provide an array of data management, data quality, and data governance capabilities. The acquisition of Upsolver, which was founded by Ori Rafael and Yoni Eini in 2014, bolsters that data integration toolset.

Upsolver devoted itself to automating the data engineering work that typically goes into building data pipelines that load data lakes and lakehouses. The Sunnyvale, California developed a no-code platform that it claims can eliminate much of the tedium of hand-coding SQL-based data transformation routines, such as converting unstructured or semi-structured data into database tables.

In recent years, Upsolver has adapted its tools to write data in the Apache Iceberg table format. Iceberg, as we have discussed, is at the center of a resurgence in the data lakehouse design pattern, as it frees organizations to use a multitude of data processing engines against their tables, without the close coupling to the processing engines that was previously required to achieve accuracy and performance.

Qlik says that the acquisition of Upsolver and its real-time ingestion and optimization technologies will enable it to offer a single scalable platform for delivering analytics and AI insights on open, trusted, and governed data.

Upsolver provides a “many to many” data integration architecture (Image courtesy Upsolver)

“With Upsolver’s real-time streaming capabilities and advanced Iceberg optimization, we’re building on Qlik’s proven enterprise solutions to drive even greater value for our customers,” James Fisher, Qlik’s chief strategy officer, said in a press release.

The acquisition demonstrates the growing importance of open, real-time data architectures, according to Stewart Bond, the vice president of data intelligence and integration software research at IDC.

“This move positions Qlik to deliver solutions that meet the dual demands for scalability and interoperability, helping organizations unlock the full potential of data-intensive workloads such as AI and machine learning while maintaining control over costs and governance,” Bond said in a press release.

Upsolver had raised $42 million in venture capital through a Series B round in 2021. Qlik, which surpassed $100 million annual recurring revenue about 18 months ago, did not disclose the purchase price.

dbt Labs Nabs SDF Labs

Meanwhile, dbt Labs also made a strategic acquisition aimed at bolstering the data integration capabilities of its extremely popular data transformation environment.

The dbt folks from the Fishtown neighborhood of Philadelpha bought SDF Labs, which isn’t even a year old. SDF Labs, which is based in Seattle, Washington, came out of stealth last June with a Rust-based toolset and framework that’s designed to compile and understand the SQL that users write, regardless of platform. It can be used as an alternative to dbt, or used with it.

On its website, SDF Labs says its technology is “a multi-dialect SQL compiler, transformation framework, and analytical database engine packaged into a single CLI. Unlike other data transformation tools like DBT, SDF extracts SQL compilers from their clouds, understanding proprietary dialects of SQL (like Snowflake) so deeply that it can ultimately execute them.”

SDF is a development tool and runtime data integration framework (Image courtesy SDF Labs)

dbt Labs plans to incorporate SDF directly into its offering, which it says will deliver “orders of magnitude improvements to dbt performance and enhancing the developer experience with new levels of efficiency, data velocity and data quality.”

In particular, dbt likes how SDF “validates the SQL code a user is writing, immediately as it’s being written,” the company says in its press release. “This real-time feedback allows developers to embrace modern development accelerants like code completion and content assist as well as pinpoint errors and ensure data quality far earlier in the development process. This expedites data velocity, boosts data quality, and makes organizations much more efficient in their analytics practices.”

SDF Labs was founded by former Meta and Microsoft engineers, including Michael Levin, Elias DeFaria, Lukas Schulte, and Wolfram Schulte. The company had raised $9 million in seed funding before the acquisition by dbt Labs. Dbt Labs did not disclose the purchase price.

“Bringing SDF and dbt together is going to completely transform the dbt user experience with unprecedented levels of speed, accuracy, and velocity,” said Lukas Schulte, the SDF Labs CEO. “The SDF Team and I are so excited to magnify the impact that our technology can have by powering the data control plane that sets the standard for the future of data analytics.”

dbt Labs CEO Tristan Handy generally isn’t an excitable guy, but the acquisition of SDF Labs may have him dancing in the aisles.

“SDF will be a massive upgrade to the very heart of the dbt user experience moving forward,” he wrote in a blog post. “It will enable faster dbt project compilation (~2 orders of magnitude), amazing developer experience (think: type-ahead in your IDE of choice), the highest-fidelity lineage on the market, and much more.”

SDF also brings a built-in query engine (Image courtesy SDF Labs)

In particular, SDF brings SQL understanding into dbt, which is something that dbt had always left to the user and the database, as it treated all the SQL that it generated as text. With the integration of SDF, however, dbt will, for the first time, be elevated to understanding objects and types, and syntax and semantics, Handy writes.

“A complete understanding of SQL allows the SDF engine to faithfully emulate cloud data warehouses in their behavior and provide that feedback before execution and catch breaking changes as part of development rather than after deployment,” he writes.

“Best of all,” Handy continues, “integration is easy. SDF has adopted dbt’s syntax, configuration, libraries, and Jinja natively, as part of the SDF runtime. As a result, for most dbt projects there will be no code changes required to take full advantage of SDF’s capabilities!”

The acquisitions of Upsolver and SDF Labs by Qlik and dbt Labs, respectively, indicate that data integration is entering a period of accelerated development and change. Out are proprietary databases and tools and brittle data pipelines that lock you in and break. In are open formats, lakehouses, customer choice of query engines, and tools that allow you to easily point your data sources to new data destinations, without the loads of work that used to be required.

And of course, that’s great news for customers.

Related Items:

Why Data Lakehouses Are Poised for Major Growth in 2025

How Apache Iceberg Won the Open Table Wars

Tristan Handy’s Audacious Vision of the Future of Data Engineering