stp2y

[Submitted on 11 Jul 2024] View a PDF of the paper titled Automatic Pruning of Fine-tuning Datasets for Transformer-based Language Models, by Mohammadreza Tayaranian and 4 other authors View PDF Abstract:Transformer-based language models have shown state-of-the-art performance on a variety of natural language understanding tasks. To achieve this performance, these models are first pre-trained on general corpus and then fine-tuned on downstream tasks. Previous work studied the effect of pruning the training set of the downstream tasks on the performance of the model on its evaluation set. In this work, we propose an automatic dataset pruning method for the training…

[Submitted on 11 Jul 2024] View a PDF of the paper titled Inflationary Flows: Calibrated Bayesian Inference with Diffusion-Based Models, by Daniela de Albuquerque and John Pearson View PDF HTML (experimental) Abstract:Beyond estimating parameters of interest from data, one of the key goals of statistical inference is to properly quantify uncertainty in these estimates. In Bayesian inference, this uncertainty is provided by the posterior distribution, the computation of which typically involves an intractable high-dimensional integral. Among available approximation methods, sampling-based approaches come with strong theoretical guarantees but scale poorly to large problems, while variational approaches scale well but offer few…

arXiv:2407.08865v1 Announce Type: new Abstract: Shadow removal aims at restoring the image content within shadow regions, pursuing a uniform distribution of illumination that is consistent between shadow and non-shadow regions. {Comparing to other image restoration tasks, there are two unique challenges in shadow removal:} 1) The patterns of shadows are arbitrary, varied, and often have highly complex trace structures, making ``trace-less'' image recovery difficult. 2) The degradation caused by shadows is spatially non-uniform, resulting in inconsistencies in illumination and color between shadow and non-shadow areas. Recent developments in this field are primarily driven by deep learning-based solutions, employing a variety…

arXiv:2407.08842v1 Announce Type: new Abstract: Pre-trained large language models (LLMs) can now be easily adapted for specific business purposes using custom prompts or fine tuning. These customizations are often iteratively re-engineered to improve some aspect of performance, but after each change businesses want to ensure that there has been no negative impact on the system's behavior around such critical issues as bias. Prior methods of benchmarking bias use techniques such as word masking and multiple choice questions to assess bias at scale, but these do not capture all of the nuanced types of bias that can occur in free response…

Generative AI + Time-Series Forecasting? Many world-class organisations are starting to invest in new GenAI+TS forecasting methods that involve for example: developing new specialised VAEs, using Vision-Language Models, pre-training the model with trillions of TS data points, or incorporating text embedding and tokenisation into the TS forecasting method. Checkout these 6 very recent, interesting papers that show the impressive, rapid evolution in this area.Re-programming LLMs for time-series modelling. This a great post about how researchers are trying to align the information gap between time series and natural language from every perspective of training a LLM. Re-programming a LLM for time…