Mastering data governance in modern analytics – SiliconANGLE

The field of data and analytics is rapidly growing and evolving, requiring creativity, skill and a deep understanding of emerging technologies, particularly in data governance.

Advanced strategies for scaling, data governance and the balance between art and science in data processing are at the forefront of current discussions. These innovations not only enhance the efficiency and accuracy of data-driven decisions, but also present unique challenges in maintaining data quality and governance across diverse and unstructured data formats, according to Krishna Valluru (pictured), vice president and team leader of advanced strategies and research tech at Fidelity Investments.

“Scale is the biggest problem. How do you scale? You can experiment, you can do certain value. If you have a business use case, you can deliver the use case,” Valluru said. “However, the scaling is a biggest challenge for us. That is what we are focusing on. So, first we start looking into how should we have a proper governance, proper quality. Then, can we scale or not?”

Valluru spoke with theCUBE’s Dave Vellante and Sanjeev Mohan at the CDOIQ Symposium, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the challenges of scaling unstructured data, the role of artificial intelligence in data processing, ensuring data quality and governance, and the impact of emerging technologies such as large language models and quantum computing on data management.

Balancing art and science in data governance

As data volumes continue to grow exponentially, managing unstructured data and ensuring its quality for business applications is becoming increasingly complex, Valluru pointed out.

“Some of the data that you can classify them as public, so you have to understand, what is public data. Any company CEOs, any board members are public,” Valluru said. “So … we have to make sure that governance is in place. We can scale pretty fast, but we don’t want to impact the governance or the policies. We should not violate them.”

Emerging technologies also play a significant role in the future of data management. The introduction of large language models and generative AI has significantly altered the landscape, Valluru explained.

“It is good to get excited to see how it is, and then slowly slow down … number one. Number two is that what we look for is … at the enterprise we have some standards. You have to … make sure everything is stable,” Valluru said. “It should be in a generally available state. Until then, we cannot use it. But at the same time, you can see how, where other technologies are emerging. You can identify them a little bit faster so that either you can use them in an operational and not at the end user level.”

The intersection of AI and data management also opens the door to quantum computing. While still in its early stages, quantum computing promises to revolutionize how data is processed and analyzed alongside AI, Valluru added.

“I think quantum can do good calculations at some point. I think they can be independent. I think they can go in tandem [with AI],” he said. “So, it is not like either/or — or they can go in tandem, or some of the output from gen AI can be used in quantum, some of the quantum outputs can be used in gen AI. Some like math. It can do math pretty fast.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE Research’s coverage of the CDOIQ Symposium:

Photo: SiliconANGLE