CybAI news

Source: AuthorWhat are LLM Evaluation Metrics?Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP) due to their ability to understand and generate human language with remarkable proficiency. These models like GPT-4, BERT, T5, etc. are being utilized in a variety of applications ranging from automated text generation and translation to conversational agents, and content recommendation systems. Evaluating these LLMs has become critical to ensure their effectiveness and reliability.Evaluation with the help of various evaluation metrics helps developers understand how a model is performing in different contexts and tasks. Moreover, effective evaluation plays a pivotal role in…

arXiv:2408.03475v1 Announce Type: new Abstract: An emerging topic in large language models (LLMs) is their application to time series forecasting, characterizing mainstream and patternable characteristics of time series. A relevant but rarely explored and more challenging question is whether LLMs can detect and explain time series anomalies, a critical task across various real-world applications. In this paper, we investigate the capabilities of LLMs, specifically GPT-4 and LLaMA3, in detecting and explaining anomalies in time series. Our studies reveal that: 1) LLMs cannot be directly used for time series anomaly detection. 2) By designing prompt strategies such as in-context learning and…

arXiv:2408.03521v1 Announce Type: new Abstract: Shadow detection is a fundamental and challenging task in many computer vision applications. Intuitively, most shadows come from the occlusion of light by the object itself, resulting in the object and its shadow being contiguous (referred to as the adjacent shadow in this paper). In this case, when the color of the object is similar to that of the shadow, existing methods struggle to achieve accurate detection. To address this problem, we present SwinShadow, a transformer-based architecture that fully utilizes the powerful shifted window mechanism for detecting adjacent shadows. The mechanism operates in two steps.…

arXiv:2408.03630v1 Announce Type: new Abstract: Automatic extraction of procedural graphs from documents creates a low-cost way for users to easily understand a complex procedure by skimming visual graphs. Despite the progress in recent studies, it remains unanswered: whether the existing studies have well solved this task (Q1) and whether the emerging large language models (LLMs) can bring new opportunities to this task (Q2). To this end, we propose a new benchmark PAGED, equipped with a large high-quality dataset and standard evaluations. It investigates five state-of-the-art baselines, revealing that they fail to extract optimal procedural graphs well because of their heavy…