A Unified Multimodal State Representation Framework for Deep Reinforcement Learning in Financial Trading

Abstract

Financial markets are inherently non-stationary, noisy, and driven by complex interactions between price dynamics, news sentiment, and macroeconomic shifts. Traditional reinforcement learning (RL) trading systems predominantly rely on raw price data or technical indicators, often overlooking rich exogenous information. This paper presents a unified multimodal state representation framework designed for deep reinforcement learning trading agents under realistic transaction cost constraints. The proposed pipeline systematically integrates high-frequency market data from Binance (BTCUSD) and OANDA (EURUSD) with offline-engineered features into a standardized 30-minute bar-end state matrix. Specifically, the framework combines price-derived indicators, latent price embeddings and directional probabilities extracted via a 1D Convolutional Neural Network (CNN), event-based media attention and sentiment features from the Global Database of Events, Language, and Tone (GDELT), and resampled macroeconomic indicators (such as the VIX and central bank interest rates). To prevent look-ahead bias and model distortion, all non-price variables are rigorously lagged, winsorized, and scaled using robust normalization fitted exclusively on historical training partitions. By decoupling feature engineering from online RL optimization, this modular architecture ensures absolute reproducibility and provides a robust, low-dimensional state foundation for policy optimization algorithms like Proximal Policy Optimization (PPO).