Abstract

This research focuses on building a custom pipeline centered around a financial large language model that runs well on resource-constrained devices. Some of the most important aspects of our implementation are the use of a synthetic dataset and the text organization method. While the reranker has a teacher-student architecture focused on the financial domain, the generative model is finetuned from a baseline model.
In the subsequent sections of this study, we will thoroughly examine the external services that facilitated the development of our solution, alongside a comprehensive analysis of every significant component within the entire pipeline.

Summary

Language models like ChatGPT and Gemini have advanced, but they are costly and challenging to use in resource-constrained settings. Our custom pipeline prioritizes accessibility and leverages adapted models to provide useful financial information. Techniques such as fine-tuning and quantization enhance performance on mobile devices. Results show a significant improvement in efficiency and accuracy, with potential for further enhancement.