Data consolidation: Evolving themes on business semantics – SiliconANGLE

A hot topic in the data management space is the argument against data silos. Factors such as generative artificial intelligence and decentralized data infrastructures have intensified the need to embrace the data consolidation ethos.

What is the imperative to de-silo data operations? And what challenges and opportunities exist as companies move toward interconnected data infrastructures?

“You’re building intelligent applications, and by definition, these applications have to look at data,” said Molham Aref (pictured), chief executive officer of RelationalAI Inc. “The old architecture is where you’re moving a few records of data over to some Java to do something, to book you an airline seat, that doesn’t work anymore. You’re now doing things that require you to predict and you need to look at lots of data. And so, the new architectures will move the semantics and the business logic to the data. And my view is that business logic is going to have to change to fit in the modern data stack.”

Aref spoke with theCUBE Research’s Dave Vellante and George Gilber at the Supercloud 7: Get Ready for the Next Data Platform event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the future of data consolidation as one that integrates and harmonizes data with business logic, creating a seamless and intelligent ecosystem.

Data consolidation: Key players and battles

The last decade has seen significant developments in the data stack. Snowflake Inc. emerged as a dominant force in the database engine space, while Databricks Inc. took the lead in managing data pipelines. The field of data formats is another competition frontier, one that has seen the emergence of Iceberg as a dominant player, according to Aref.

“I think the next layer is about semantics,” he said. “Where do you put application logic and semantics in this new world? And where do you put it in a world where people are asking to rethink applications in terms of intelligent applications?”

One inalienable theme in the various contests across the data stack is openness. The next frontier is the catalog wars, where the aim is to unbundle catalogs from the respective technologies, allowing for greater flexibility and interoperability. This unbundling is a critical step toward the next layer of evolution: semantics.

“Let’s say, I have information about people in my database,” Aref said. “And let’s say there’s an entry for Dave, an entry for George, an entry for Molham. I might want to compute our ages from our birthdays. Now, I can just add an extra column and key in our ages, but that’s not good, because as time passes, I’ll have to keep updating our ages as we all get older. Instead of just storing the raw data, I might put in a bit of business logic that says, ‘A person’s age is equal to today minus their date of birth.’ And anytime someone needs to look up any of our ages, okay, I will just apply the semantics, the business logic.”

In traditional systems, application logic was often written in languages such as Java or COBOL, detached from the data itself. The modern approach expresses business logic relationally, ensuring it coexists seamlessly with the data. This transition to data consolidation is essential for applications that are capable of making predictions and optimizations based on vast amounts of data, according to Aref.

The legacy challenge and path forward

The current state of enterprise data management is a patchwork of thousands of applications, each with its data model and logic. This fragmented architecture is a product of technological constraints and business needs from decades past. However, with advancements in scalability, algorithms and data structures, there is an opportunity to revisit and overhaul these systems, according to Aref.

“Because the relational databases at the time were not powerful enough and scalable enough and their pricing models were such that they were too expensive to run the business logic, as a workaround, we pulled that logic out, we implemented by hand procedurally, because we didn’t have the relational language that could express it declaratively,” he said. “We had to live with this two-brain architecture, where the database had no idea what the application logic was doing. And the application logic didn’t understand databases.”

The goal is to move toward a common model where data and business logic are integrated into a unified framework. This approach simplifies the architecture, reduces redundancy and facilitates better decision-making. Subsequently, a gradual refactoring of legacy applications will occur, bringing forth a modern, data-centric framework, without the need for a disruptive big bang rewrite, Aref added.

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE Research’s coverage of the Supercloud 7: Get Ready for the Next Data Platform event:

Photo: SiliconANGLE