How Machine Learning Enhances Market Volatility Forecasting Accuracy

Sponsored by

Machine learning has many applications in finance, such as asset pricing, risk management, portfolio optimization, and fraud detection. In this issue, I discuss the use of machine learning in forecasting volatility.

In this issue:

• Latest Posts

• Using Machine Learning to Predict Market Volatilit …

  • Findings

• Machine Learning Models for Predicting Implied Vol …

  • Findings

• Closing Thoughts

• Featured Resource

• Educational Video

  • Machine Learning for Realised Volatility Forecasti …

• Volatility Weekly Recap

• Around the Quantosphere

Latest Posts

• Predicting Corrections and Economic Slowdowns (11 min)

• Rethinking Leveraged ETFs and Their Options (12 min)

• Using Skewness and Kurtosis to Enhance Trading and Risk Management (10 min)

• Gold Ratios as Stock Market Predictors (11 min)

• Volatility of Volatility: Insights from VVIX (11 min)

Using Machine Learning to Predict Market Volatility 

The unpredictability of the markets is a well-known fact. Despite this, many traders and portfolio managers continue to try to predict market volatility and manage their risks accordingly. Usually, econometric models such as GARCH are used to forecast market volatility.

In recent years, machine learning has been shown to be capable of predicting market volatility with accuracy. Reference [1] explored how machine learning can be used in this context.

Findings

• Machine learning models can accurately forecast stock return volatility using a small set of key predictors: realized volatility, idiosyncratic volatility, bid-ask spread, and returns.

• These predictors align with existing empirical findings, reinforcing the traditional risk-return trade-off in finance.

• ML methods effectively capture both the magnitude and direction of predictor impacts, along with their interactions, without requiring pre-specified model assumptions.

• Large current-period volatility values strongly predict higher future volatility; small values have a muted or negative impact.

• LSTM models outperform feedforward neural networks and regression trees by leveraging temporal patterns in historical data.

• An LSTM using only volatility and return history over one year performs comparably to more complex models with additional predictors.

• LSTM models function as distribution-free alternatives to traditional econometric models like GARCH.

• Optimal lag length remains critical in LSTM performance and must be selected through model training.

• The study reports an average predicted realized volatility of 44.1%, closely matching the actual value of 43.8%. -Out-of-sample R² values achieved are significantly higher than those typically reported in related volatility forecasting literature.

In short, the paper aimed to demonstrate the potential of machine learning for modeling market volatility. In particular, the authors have shown how the LSTM model can be used to predict market volatility and manage risks. The results suggest that this is a promising alternative approach to traditional econometric models like GARCH.

Reference

[1] Filipovic, Damir and Khalilzadeh, Amir,  Machine Learning for Predicting Stock Return Volatility (2021). Swiss Finance Institute Research Paper No. 21-95

Machine Learning Models for Predicting Implied Volatility Surfaces

The Implied Volatility Surface (IVS) represents the variation of implied volatility across different strike prices and maturities for options on the same underlying asset. It provides a three-dimensional view where implied volatility is plotted against strike price (moneyness) and time to expiration, capturing market sentiment about expected future volatility.

Reference [2] examines five methods for forecasting the Implied Volatility Surface of short-dated options. These methods are applied to forecast the level, slope, and curvature of the IVS.

Findings

• The study evaluates five methods—OLS, AR(1), Elastic Net, Random Forest, and Neural Network—to forecast the implied volatility surface (IVS) of weekly S&P 500 options.

• Forecasts focus on three IVS characteristics: level, slope, and curvature. -Random Forest consistently outperforms all other models across these three IVS dimensions.

• Non-learning-based models (OLS, AR(1)) perform comparably to some machine learning methods, highlighting their continued relevance.

• Neural Networks forecast the IVS level reasonably well but perform poorly in predicting slope and curvature.

• Elastic Net, a linear machine learning model, is consistently outperformed by the non-linear models (Random Forest and Neural Network) for the level characteristic.

• The study emphasizes the importance of model selection based on the specific IVS characteristic being forecasted.

• Performance evaluation is supported using the cumulative sum of squared error difference (CSSED) and permutation variable importance (VI) metrics.

• The research highlights the utility of Random Forest in capturing complex, non-linear patterns in IVS dynamics.

• Accurate IVS forecasting is valuable for derivative pricing, hedging, and risk management strategies.

This research highlights the potential of machine learning in forecasting the implied volatility surface, a key element in options pricing and risk management. Among the five methods studied, Random Forest stands out as the most consistent and accurate across multiple IVS features.

Reference

[2] Tim van de Noort, Forecasting the Characteristics of the Implied Volatility Surface for Weekly Options: How do Machine Learning Methods Perform? Erasmus University, 2024

Closing Thoughts

These studies highlight the growing effectiveness of machine learning in financial forecasting, particularly for market volatility and implied volatility surfaces. Models like LSTM and Random Forest demonstrate clear advantages over traditional methods by capturing complex patterns and dependencies. As financial markets evolve, leveraging such tools offers a promising path for enhancing predictive accuracy and risk management.

Featured Resource

Anton Vorobets just released a new quant course on Portfolio Construction and Risk Management. The course focuses on applied aspects of quantitative and quantamental investment management, drawing on perspectives from sophisticated multi-asset investors.

To learn more about the course, visit his website at

https://antonvorobets.substack.com/p/lecture-1-intro-and-python-setup

Educational Video

Machine Learning for Realised Volatility Forecasting

In this video, Eghbal Rahimkina and Ser-Huang Poon explore how machine learning can improve realized volatility forecasting. They compare traditional models like HAR with ML approaches using data from limit order books and news sentiment. Their results show that ML models, especially LSTMs, perform better on normal trading days but struggle during market jumps. The presentation also covers data cleaning, model setup, and evaluation methods, offering a clear view of how advanced techniques can enhance financial forecasting.

Volatility Weekly Recap

The figure below shows the term structures for the VIX futures (in colour) and the spot VIX (in grey).

Stocks posted a strong performance on Monday, and the momentum continued through the end of the week. On Friday, stocks reached a new all-time high, gaining additional support after the Administration announced a trade deal with China. The S&P 500 rose 1.72%, and the Nasdaq gained 1.62%. Oil and gold both declined during the week, while crypto surged alongside equities. Bitcoin posted a solid 6% gain.

On the volatility front, short-term spot VIX rose slightly but remained in contango. Front-month VIX futures stayed relatively flat, while mid-term futures saw a modest increase. VXX lost -1.45% due to contango, while VIXM gained 0.36% due to the rise in the mid-term VIX futures. The roll yield remains strongly positive.

Around the Quantosphere

• The End Of The Quant? How AI Is Democratizing Financial Analysis (forbes)

• Jane Street barred from Indian markets as regulator freezes $566 million over Nifty 50 manipulation claims (cnbc)

• Inside the high-frequency trading floor, where grads are paid $250,000 (AFR)

• How to get a capital markets job in an investment bank (Financial Careers)

• Implied Vols Collapse As Stocks Hit New Record High (Cboe)

• Hedge fund returns halfway through 2025: How big names like Citadel, Balyasny, and more have managed this year's markets. (Bundle)

• Hedge Fund Strategy Built on Catastrophes Taps Hot New Trend (Yahoo Finance)

• Hedge funds dive into private credit. (Financial Times)

Disclaimer

This newsletter is not investment advice. It is provided solely for entertainment and educational purposes. Always consult a financial professional before making any investment decisions.

We are not responsible for any outcomes arising from the use of the content and codes provided in the outbound links. By continuing to read this newsletter, you acknowledge and agree to this disclaimer.