Automated LaTeX Code Generation from Handwritten Math Expressions Using Vision Transformer

[Submitted on 5 Dec 2024]

View a PDF of the paper titled Automated LaTeX Code Generation from Handwritten Math Expressions Using Vision Transformer, by Jayaprakash Sundararaj and 2 other authors

View PDF
HTML (experimental)

Abstract:Converting mathematical expressions into LaTeX is challenging. In this paper, we explore using newer transformer based architectures for addressing the problem of converting handwritten/digital mathematical expression images into equivalent LaTeX code. We use the current state of the art CNN encoder and RNN decoder as a baseline for our experiments. We also investigate improvements to CNN-RNN architecture by replacing the CNN encoder with the ResNet50 model. Our experiments show that transformer architectures achieve a higher overall accuracy and BLEU scores along with lower Levenschtein scores compared to the baseline CNN/RNN architecture with room to achieve even better results with appropriate fine-tuning of model parameters.