stp2y - CybAI news

25 May

Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)

stp2y0 CommentsAI

Low-rank adaptation (LoRA) is among the most widely used and effective techniques for efficiently training custom LLMs. For those interested in open-source LLMs, it's an essential technique worth familiarizing oneself with.Last month, I shared an article with several LoRA experiments, based on the open-source Lit-GPT repository that I co-maintain with my colleagues at Lightning AI. This Ahead of AI article aims to discuss the primary lessons I derived from my experiments. Additionally, I'll address some of the frequently asked questions related to the topic. If you are interested in finetuning custom LLMs, I hope these insights will save you some…

25 May

Bringing generative artificial intelligence to space

stp2y0 CommentsChat-GPT

TAMPA, Fla. — Amazon Web Services is busy positioning its cloud infrastructure business to capitalize on the promise of generative artificial intelligence for transforming space and other industries. More than 60% of the company’s space and aerospace customers are already using some form of AI in their businesses, according to AWS director of aerospace and satellite Clint Crosier, up from single digits around three years ago. Crosier predicts similar growth over the next few years in space for generative AI, which uses deep-learning models to answer questions or create content based on patterns detected in massive datasets, marking a major…

25 May

Data Machina #247

stp2y0 CommentsViral News

The New Breed of Open Mixture-of-Experts (MoE) Models. In a push to beat the closed-box AI models from the AI Titans, many startups and research orgs have embarked in releasing open MoE-based models. These new breed of MoE-based models introduce many clever architectural tricks, and seek to balance training cost efficiency, output quality, inference performance and much more. For an excellent introduction to MoEs, checkout this long post by the Hugging Face team: Mixture of Experts ExplainedWe’re starting to see several open MoE-based models achieving near-SOTA or SOTA performance as compared to e.g. OpenAI GPT-4 and Google Gemini 1.5 Pro.…

25 May

German AI defence giant Helsing eyes $4B valuation in Series C funding

stp2y0 CommentsNews

German AI defense company Helsing is reportedly in talks to secure €370 million in Series C funding, potentially valuing the company at a staggering $4 billion. The round is said to be led by US VC giant General Catalyst, according to Forbes. In September last year, the company led Helsing in securing €209 million in Series B funding, solidifying its status as Europe's most well-funded defencetech startup and among the leading AI companies in the field. The Series C would bring its funds to almost €700 million. Founded in 2021, Helsing develops AI-based capabilities to "protect our democracies." The company has been active in…

25 May

Big tech has distracted world from existential risk of AI, says top scientist

stp2y0 CommentsChat-GPT

Big tech has succeeded in distracting the world from the existential risk to humanity that artificial intelligence still poses, a leading scientist and AI campaigner has warned.Speaking with the Guardian at the AI Summit in Seoul, South Korea, Max Tegmark said the shift in focus from the extinction of life to a broader conception of safety of artificial intelligence risked an unacceptable delay in imposing strict regulation on the creators of the most powerful programs.“In 1942, Enrico Fermi built the first ever reactor with a self-sustaining nuclear chain reaction under a Chicago football field,” Tegmark, who trained as a physicist,…

25 May

TMLR Paper “Conformal Prediction under Ambiguous Ground Truth” • David Stutz

stp2y0 CommentsGenAI

Conformal prediction uses a held-out, labeled set of examples to calibrate a classifier to yield confidence sets that include the true label with user-specified probability. But what happens if even experts disagree on the ground truth labels. Commonly, this is resolved by taking the majority voted label from multiple expert. However, in difficult and ambiguous tasks, the majority voted label might be misleading and a bad representation of the underlying true posterior distribution. In this paper, we introduce Monte Carlo conformal prediction which allows to perform conformal calibration directly against expert opinions or aggregate statistics thereof. Abstract Conformal Prediction (CP)…

25 May

Microsoft’s CTO on How to Build Enduring Businesses in the AI Era

stp2y0 CommentsRAG models

This week I flew out to Seattle to attend Microsoft's developer conference, Build. Read my full report and Evan Armstrong's analysis.Kevin Scott is not your average CTO. He says things like “chill” and “shit,” and he wears button-down shirts with funky patterns. Oh yeah, and he builds supercomputers. In his spare time, he likes to do woodworking.Scott was the first person at Microsoft to recognize the importance of AI and instigated its OpenAI partnership, which has become the single most important strategic move at Microsoft since the dot-com era. He's hopeful that in the next two to three years, a model…

25 May

The Role of AI in Big Data Quality Management

stp2y0 CommentsViral News

In the realm of big data quality management, the convergence of AI technologies has opened up avenues for unparalleled levels of data accuracy and reliability. By harnessing the power of artificial intelligence, organizations can now automate the process of detecting and correcting errors in massive datasets with unprecedented speed and efficiency. Through advanced machine learning algorithms, AI systems can continuously learn from data patterns, enhancing their ability to identify inconsistencies and anomalies that might have otherwise gone unnoticed by human analysts. AI-driven big data quality management solutions offer a proactive approach to maintaining data integrity by predicting potential issues before…

25 May

Watermarking AI-generated text and video with SynthID

stp2y0 CommentsNews

Company Published 14 May 2024 Announcing our novel watermarking method for AI-generated text and video, and how we’re bringing SynthID to key Google productsGenerative AI tools — and the large language model technologies behind them — have captured the public imagination. From helping with work tasks to enhancing creativity, these tools are quickly becoming part of products that are used by millions of people in their daily lives.These technologies can be hugely beneficial but as they become increasingly popular to use, the risk increases of people causing accidental or intentional harms, like spreading misinformation and phishing, if AI-generated content isn’t…

25 May

On the Stepwise Nature of Self-Supervised Learning

stp2y0 CommentsAIneural tangent kernel, representations, self-supervised, ssl, stepwise, training

Figure 1: stepwise behavior in self-supervised learning. When training common SSL algorithms, we find that the loss descends in a stepwise fashion (top left) and the learned embeddings iteratively increase in dimensionality (bottom left). Direct visualization of embeddings (right; top three PCA directions shown) confirms that embeddings are initially collapsed to a point, which then expands to a 1D manifold, a 2D manifold, and beyond concurrently with steps in the loss. It is widely believed that deep learning’s stunning success is due in part to its ability to discover and extract useful representations of complex data. Self-supervised learning (SSL) has…