SingleStore adds Iceberg support and speedier vector search – SiliconANGLE

SingleStore Inc., developer of a real-time database platform that performs both transactional and analytical operations in the same engine, today announced native integration with the Apache Iceberg open-source table format and advanced search and scaling capabilities.

Apache Iceberg is rapidly emerging as one of the most popular formats for storing diverse datasets. It provides transactional consistency between multiple applications, schema evolution, the ability to query historical data and the ability to roll back to prior versions, among other features.

SingleStore said it’s addressing the often complex task of moving data into Iceberg-based data lakes, a process known as extract/transform/load. It said its approach provides for low-latency ingestion, bidirectional data flow and real-time operation.

The company, which was previously known as MemSQL, stores data in a distributed, hybrid architecture that combines in-memory row storage and disk-based column storage. Iceberg integration is immediately available in a public preview. A general availability date hasn’t been announced.

SingleStore also announced vector search enhancements that improve performance by about 40% using hierarchical navigable small world, an advanced algorithm used for “approximate nearest neighbor” search in large datasets. Improvements to the inverted file flat index, a type of data structure used in vector search engines for efficient similarity search, make SingleStore perform between 47 and 100 times faster than the pgvector PostegreSQL extension for vector operations, the company claimed.

This release also includes improved capabilities for full-text search, including improved relevance scoring, phonetic similarity, fuzzy matching and keyword-proximity-based ranking. That reduces the need for specialty databases to build generative artificial intelligence and real-time applications.

New autoscaling features now in public preview make performance more preditable by scaling compute resources automatically. The company said users can define thresholds for processor and memory usage for autoscaling to avoid consumption or billing surprises.

SingleStore is also introducing a fully managed private cloud offering that offers the management features and scalability of the database-as-a-service platform. The offering is now available in private preview on the Amazon Web Services Inc. cloud, where customers can run SingleStore in their own tenants to comply with data residency and governance policies.

SingleStore has raised more than $460 million since its 2011 founding. The private company said it surpassed $100 million in annual recurring revenue at the end of last year.

Image: Pixabay