[Submitted on 9 Sep 2024]
View a PDF of the paper titled DiffusionPen: Towards Controlling the Style of Handwritten Text Generation, by Konstantina Nikolaidou and George Retsinas and Giorgos Sfikas and Marcus Liwicki
Abstract:Handwritten Text Generation (HTG) conditioned on text and style is a challenging task due to the variability of inter-user characteristics and the unlimited combinations of characters that form new words unseen during training. Diffusion Models have recently shown promising results in HTG but still remain under-explored. We present DiffusionPen (DiffPen), a 5-shot style handwritten text generation approach based on Latent Diffusion Models. By utilizing a hybrid style extractor that combines metric learning and classification, our approach manages to capture both textual and stylistic characteristics of seen and unseen words and styles, generating realistic handwritten samples. Moreover, we explore several variation strategies of the data with multi-style mixtures and noisy embeddings, enhancing the robustness and diversity of the generated data. Extensive experiments using IAM offline handwriting database show that our method outperforms existing methods qualitatively and quantitatively, and its additional generated data can improve the performance of Handwriting Text Recognition (HTR) systems. The code is available at: this https URL.
Submission history
From: Konstantina Nikolaidou [view email]
[v1]
Mon, 9 Sep 2024 20:58:25 UTC (41,123 KB)
Source link
lol
