It has been quite interestingly found that a large proportion of the studies that
employed textual data focus on stock market and foreign exchange rate (FOREX) prediction, due to the following three main reasons:
- Lack of accessibility for many assets
- The nature of other financial products
- Transparency of stock and currency markets

University of Michigan - Ann Arbor

Utilizing textual data to improve modeling of the financial market dynamics has long been the tradition of trading practice. The growing volume of financial reports, press releases, and news articles also galvanizes the wish to run this analysis automatically to keep a competitive business advantage. As a result, articles that leverage NLP techniques to predict financial markets are fast accumulating, gradually establishing the research field of natural language based financial forecasting (NLFF), or from the application perspective, stock market prediction.

Applications of NLP in Financial Markets

https://dspace.mit.edu/bitstream/handle/1721.1/116314/10462_2017_9588_ReferencePDF.pdf?sequence=2&isAllowed=y

A Survey of Natural Language Based Financial Forecasting

Stock Market and FOREX Prediction

To represent textual financial data as features that can be easily processed by a computer, most of the early NLFF papers have employed bag-of-words, which represent the semantics of a piece of text by the set of words and the frequency of their appearance. At the document level, semantics is discomposed to multiple topics and corresponding relevance coefficients. These techniques enable the analysis of a large volume of financial articles as a whole.

Semantic Modeling in NLFF

Sentiment analysis is an important perspective for NLFF due to the interactive nature of financial activities. According to the five-eras vision of the future web, market sentiment will become a prominent factor that influences trading and information flow as well as shaping products and services.  Existing approaches to affective computing fall into three categories: knowledge-based
techniques, statistical methods, and hybrid approaches.

Semantic Analysis in NLFF

Statistical methods extract conjunctions between words, usually depending on a large annotated corpus. Also, these meaningful narratives can be fed into deep neural networks to produce vector representations. Based on the capability to extract semantics and sentiments from natural language, the problem of financial forecasting can be modeled at a more abstract level. Time series analysis is a classic technique which gives more weight to endogenous factors.

Event Extraction in NLFF

A fundamental question that might be often discussed is: Where does the excess return of financial forecasting come from?

The most acknowledged answer lays on the negation of the efficient market hypothesis (EMH)  in a real world case. The idea is that if all the participants in a market are “informationally efficient”, all deals would be conducted at a fair value. The excess return should come from the passionate or noise traders, which further offends the hypothesis of rational man. To reconcile this problem, behavioral economics has come up with theories that are compatible with the interactive nature of the market and participants, such as the adaptive market hypothesis (AMH). Then excess return can be ascribed to information asymmetry.

Efficient Market Hypothesis (EMH)

From the perspective of Artificial Intelligence, three sources of information have been most heavily exploited: historical time series data such that used by technical analysis and semantic features and sentiment information extracted from the financial news as valued by fundamental analysis.

Sources of Information in NLFF

The intriguing task of financial forecasting attracts researchers with both computer science and finance backgrounds. The ways they formalize this task are diversified, and together form a spectrum of perspectives:
- The connectionist perspective
- The portfolio management perspective
- The energy system perspective
- The social network perspective

Spectrum of Perspectives in NLFF

Linear regressions and SVM are classic methods that dominate prediction models in the past decades. It is still an open question as to what category of algorithms is especially appropriate for NLFF, however, mainstream algorithms can be placed into four categories: regressions, probabilistic inferences, and neural networks, or a hybrid of them.

Algorithms in NLFF

The main findings from previous studies in NLFF are as follows:
- The illusion of growth
- The predictability of financial news
- The “20-minute” theory
- The Monday effect
- The reversal effect

Main Findings in NLFF

Three promising future directions of NLFF in the near term are as follows:
- Domain specific resources building
- Online predictive model:
- Comprehensive evaluation measurements

Learn Before

Related