(Ryzhi/Shutterstock)
Finding data to train AI models has become a major headache for enterprises, as they have largely exhausted the most useful publicly available datasets.
While organizations possess valuable proprietary data that could enhance their AI training efforts, they are often hesitant to leverage it due to privacy concerns, compliance issues, and the costs associated with generating synthetic data. These challenges create barriers to unlocking valuable insights and often hinder the organization’s capacity to innovate in a competitive market.
MOSTLY AI, a synthetic data solutions provider, has launched a new synthetic text generator to address major AI training challenges for enterprises. This innovative platform allows organizations to derive greater insights from their proprietary datasets while mitigating data concerns.
Using the synthetic text generator, organizations can safely tap into their proprietary data from emails, chatbot conversations, and customer service transcripts to train and fine-tune their large language models (LLMs). The platform can integrate proprietary data while ensuring that personally identifiable information (PII) and diversity gaps are excluded.
Initially, the company focused on helping enterprises create synthetic data in the form of structured tabular datasets. Having established this foundation, MOSTLY AI is now expanding its platform to include the generation of synthetic text data.
“Today, AI training is hitting a plateau as models exhaust public data sources and yield diminishing returns,” said Tobias Hann, CEO of MOSTLY AI. “To harness high-quality, proprietary data, which offers far greater value and potential than the residual public data currently being used, global enterprises must take the leap and leverage both structured and unstructured synthetic data to safely train and deploy forthcoming generative AI solutions.”
MOSTLY AI cites a recent survey by Gartner that reveals that 75% of companies will be using GenAI to create synthetic customer data by 2026, up from less than 5% in 2023. To facilitate this adoption, MOSTY AI aims to enable developers to generate synthetic text from their proprietary data for AI training purposes.
The new platform addresses another major enterprise challenge for training AI models – lack of data diversity. While developers can manually generate synthetic data, this can be labor-intensive, especially if they want to curate high-quality data for effective AI model performance.
MOSTLY AI’s new platform tackles the challenge of lack of data diversity by enabling the generation of synthetic text that is tailored to specific use cases, reflecting a wider range of scenarios and perspectives.
The synthetic text generator makes it easier for companies to integrate proprietary text data with structured datasets. By automating compatibility, it enables organizations to effectively utilize all relevant information for AI training, creating a comprehensive and statistically accurate view of their data assets. This approach not only supports the development of tailored GenAI solutions but also ensures that enterprises can meet their compliance requirements.
“Bringing almost a decade of deep technical expertise, MOSTLY AI delivers superior quality and reliability and is backed by a highly experienced team and industry-leading technological excellence,” said Christoph Hornung, Partner at Molten Ventures, an investor in MOSTLY AI.
“With the platform’s expansion into synthetic text, MOSTLY AI is well-positioned to support any enterprise with its sensitive data and LLM needs.”
According to MOSTLY AI, the synthetic text generated delivers 35% better performance compared to text generated by GPT-4o-mini. However, these results should be considered preliminary, as the two models serve different purposes and are optimized for distinct tasks, which means their performance metrics aren’t directly comparable.
Despite this, the introduction of synthetic text functionality marks a pivotal step forward for enterprises. The platform helps enterprises strengthen their AI training efforts by overcoming key challenges and can also support a wide range of GenAI and data analytics applications.
Related Items
Why A Bad LLM Is Worse Than No LLM At All
Vectara Spies RAG As Solution to LLM Fibs and Shannon Theorem Limitations
Source link
lol
