Research|
Parameter-Efficient Domain Adaptation via Dual-Adapter Training and Merging: Methods and Evaluation for LLM-Based Financial Analysis Tasks
This thesis addresses the challenge of adapting LLMs to financial analysis through a novel dual-adapter training and merging framework. We focus on Axyon AI’s production system, Alyx, which requires processing diverse financial tasks—stock briefs, news classification, sector analysis, and report generation—each following distinct formatting conventions and reasoning patterns. The key challenge arises from severe sample imbalance in available training data: 1,388 production-critical premarket samples (3.6%) versus 36,782 auxiliary ex-premarket samples (96.4%). Naive combined fine-tuning on this imbalanced distribution causes catastrophic underfitting on minority tasks, degrading performance by 15.2% despite training on 27.5× more data.