Soutenance de thèse : Thibault DOUZON

Language Models for Document Understanding

Doctorant : Thibault DOUZON

Laboratoire INSA : LIRIS

Ecole doctorale : ED512 Informatique Et Mathématiques de Lyon

First used for natural language related tasks, language models can understand documents better than any previous statistical model, provided enough data for training and pre-training. This thesis proposes several architectures and training procedures to better model visually-rich documents. Its main findings are the data-afficiency of pre- trained transformers compared to recurrent neural networks, the importance of pre- training tasks for downstream performance, the introduction of pre-training tasks specific to business documents and alternative architectures to transformers for multi-page documents.