Random walk stock market prices

Posted: Ivan-22 Date: 11.06.2017

StuartReid On February 8, A Non-Random Walk Down Wall Street is a collection of papers which challenge the prevailing random walk hypothesis. This series of articles has the following goals: This series is also inspired by many of the thoughtful comments I received after I published my post about the random walk hypothesis, Hacking The Random Walk Hypothesis. So please keep the comments coming. This article's featured image is from the most recent Wall Street inspired film to hit the box office: If you haven't seen it yet, do yourself a big favour and go watch it.

A Recipe for the Financial Crisis. Stock Markets Do Not Follow Random Walks: Evidence from a Simple Specification Test.

This post is broken up into the following sections: I always appreciate the input. This hypothesis is a logical consequent of the weak form of the efficient market hypothesis which states that: To a statistician the assertion that future prices cannot be predicted by analyzing prices from the past goes by a different name: The Theory of Speculation. In his paper he proposed using Brownian motion, a Markov and Martingale process, to model stock options.

Is this just fantasy? Is this the real life? Personally my mind rebels against the theory because it is too elegant; too simple. I like complexity; I like chaos. Back to the top.

Secondly, you define which statistical properties you would therefore expect to see in asset prices. If the asset prices don't exhibit the expected properties, then the assets don't evolve according to the model of the random walk hypothesis you assumed they did to begin with. It's not good enough to simply state that market returns aren't random, you need to also specify what type of random they aren't.

Under this hypothesis, variance is a linear function of time discussed in more detail in the next section. Under this hypothesis, variance is a non-linear function of time discussed in the next section. It just so happens that the weaker the form of the random walk hypothesis, the harder it is to disprove and the more powerful your statistical tests need to be.

My favourite comic strip, Dilbert, gets it. In other words, we will be testing the second variant of the random walk hypothesis, RW2. This log-price process satisfies the Markov property and is given by the following recurrence relation: It is a simplification.

Nevertheless, the desired effect of stochastic volatility namely, fatter tailed distributions and a higher volatility, is clearly present as can be seen from the two density plots below and the following two time series plots. Comparison of the distribution of random disturbances sampled from the homoskedastic model blue against the heteroskedastic model red.

As can be seen the heteroskedastic distribution has fatter tails. Comparison of the two sequences of random disturbances sampled from the homoskedastic model blue against the heteroskedastic model red. As can be seen, the heteroskedastic sequence has many more large disturbances.

Stock Market Prices Do Not Follow Random Walks

At this point you may be wondering what all the fuss is about. Then results will be shown. These processes are later used to test the calibration and the variance ratio test. Using this code is quite simple. And I would get something like this out in the plots. This can be computed in R using the following function,.

Use a subset of the observations: A significant improvement on this estimator can be obtained by using overlapping samples. And as with the previous estimators we see that as our sample size increases the estimates improve. This observation is the heart of the variance ratio test. Now let's test to see whether or not this is true.

We also see that this statistic is more sensitive than the differences statistic. This is a good characteristic. I've tried my best to explain it. Thus any rejection of the random walk hypothesis using their test would not be due to the stochastic nature of volatility or long-run drifts, but rather due to the presence of autocorrelation in the increments of X. The reason why they wanted to focus on testing for the presence of autocorrelation in the increments of X is because this indicates whether or not X satisfies the Markov property and, as per the logic laid out at the start of this article, whether or not future prices could at least in theory be partially forecasted using historical prices.

To achieve this objective we start with a rather simple observation, namely: This is because the variance of the sum of uncorrelated increments must still equal the sum of the variances. In light of this there are two options: Lo and MacKinlay opted for the latter approach, assumed that the model of heteroskedasticity has a finite variance , and developed their heteroskedastic-consistent variance ratio test accordingly.

In particular the variance ratio test they defined is not applicable to models of heteroskedasticity from the Pareto-Levy family. The assumptions made by Lo and MacKinlay for the null hypothesis are shown in the box below,. The second moment is finite, this is assumed because if it wasn't then the variance ratio is no longer well defined i.

I know how heavy that looks, but the intuition is simple. Our null hypothesis states that: And assuming this is true then we are correct in stating that: If the above explanation which is simplified when compared to the full derivation done by Lo and MacKinlay did not make much sense, then I would like to direct you to their original paper.

The two p-values computed using the Shapiro Wilk test were 0. These both indicate that the distributions are normal. T hey both look reasonable to me. And below we have exactly the same simulation results except that they are with stochastic volatility applied to the log price process.

The type of stochastic volatility is the one defined under the model specification section. Again, they both look reasonable to me. What can we tell from the above results?

The first, and most condemning set of results, was obtained on fifty stock market indices from around the world. The indices considered cover the Americas South and North , Europe, Africa, and the Middle East, as well as the South Pacific and Asia.

A large portion of the indices fall outside of the expected distribution and are therefore deemed to be not random. In order of significance the least likely to be random stock market indices were: The full set of results is available as a CSV file. We can also say that they they almost surely don't evolve according to a random walk. This will be discussed in more detail in the conclusion.

Some stocks were removed because data was not available on Yahoo! The results here differ from the results on the stock market indices in one major way: This observation is discussed in some detail in the conclusion. I can hear you asking, "so which stock is the least random? If you are wondering why so many REITs Real Estate Investment Trusts are in this list, so am I.

I have listed a few hypotheses in the conclusion section but I welcome any of your thoughts as well. In this article we have taken our first step down a Non-Random Walk Down Wall Street. Evidence from a simple specification test. In this article we implemented Lo and MacKinlay's test in R all of the code is publicly available , tested that the test works as expected on model data, and then applied that test to two real world data sets: For the global stock market indices all the available historical data was used and for the the individual stocks the past ten years worth of daily prices were used.

Assuming these assets evolved according to a random walk with drift and stochastic volatility we would have expected to see: Stock markets do not follow random walks.

This statement is as true today as it was 28 years ago in when Lo and MacKinlay concluded the same result on weekly returns data for a number of stocks and a broad based market index in the United States. During our investigation two unanticipated, but interesting, results were obtained: For these two observations I offer up the following two, untested , hypotheses: And that marks another nail in the coffin for the random walk hypothesis.

That said, I really don't think this should come as a surprise. For decades academics and practitioners have been discovering anomalies in price and returns data which simply shouldn't exist under the random walk hypothesis. The truth of the matter is that they are both great programming languages and, ultimately, what tool you use should be determined by your use case, not just your comfort zone.

An Introduction to Burton Malkiel's A Random Walk Down Wall Street - A Macat Economics Analysis

I just read the AMH. One question I would have is now that more and more trading becomes algorithmic and algorithms will have a different behavior.

How do we make sure that the evolutionary perspective will continue to hold. It would be interesting to see what autocorellation looks like in the "algorithmic era". I don't know how to define that, but it would be recent. I'm not just referring to HFT but the gradual trend towards machine learning of course. I know these systems are man made and emotional responses can lead to pulling the plug on a strategy, etc.

However, I still have a notion that as more and more hedge funds rely on, and trust these models that they may deviate from human behavioral patterns. That's not to say they won't have inefficiencies and behave as "fish out of water" occasionally or frequently , but that landscape looks decidedly different in my minds eye.

I prefer the AMH to EMH for two reasons.

random walk stock market prices

Firstly, AMH doesn't shy away from real-world complexity and, secondly, AMH diagnoses so-called "inefficiencies" as emergent properties. Autocorrelation is an example of a small but consistent emergent property that I suspect will exist so long as there are a sufficient number of agents in the system which are acting upon past prices.

This is regardless of whether they are using historical prices in trading using past prices to forecast future prices Most efficient market proponents forget that finance isn't just about buying and selling stocks any more. The real estate bubble formed in was a nice example of a complex emergent property of the global economy and the stock market which arose from the seemingly unrelated simple behaviours of many agents To be more specified the behaviours which resulted in the bubble were 1 government deregulation, 2 the FED keeping interest rates low post the dotcom bubble, 3 the popularity of securitization and it's products MBSs, CDOs, etc.

So the bubble wasn't random and it wasn't a massive conspiracy of the banks either, it was an emergent property and the subsequent crash was inevitable, not unpredictable.

That having been said, it was very difficult to see and that's why so few people managed to short the bubble successfully. I'm not sure what the future will look like. But I can tell you that three things concern me 1 as you mentioned, the rise of algorithmic trading see the flash crash , 2 The proliferation of passive investment strategies, index trackers, and index-linked products zero diversity and 3 the venture capital bubble in the USA see the 's.

Which is not to say that randomness couldn't also be an emergent property of the stock market for finite periods of time. All I am saying is that if we waited another 28 years and re-ran the analysis the markets would be as random as they are now or perhaps even more so and the reasons would be the same: More quantitative evidence of what people have long known; that markets are not perfectly random in the Gaussian sense.

So, there is the famous Keynes quote: Let me ask the obvious. How does one use this research to build a trading methodology, or perhaps even identify individual trades? Hi Stryder, thanks for the comment.

First off, it was stated throughout the article this test extends to many more models of market randomness than the original Gaussian i. Secondly, it is a conflict of interest for me to blog about trading strategies because I work for a hedge fund where I develop proprietary quantitative trading strategies everyday.

Hi D, thanks for the comment! The above results are actually generated on daily data from Yahoo! The original test was done on weekly data probably because it was done way back in but in my analysis I used daily data.

Hacking the Random Walk Hypothesis

Anyway, I ran the results again on discrete weekly returns just for you and the conclusion remains the same: I think it would be valuable to write a follow-up post where I rerun the analysis from this post and some of the results from the previous post on rolling weekly, monthly, and annual returns.

I'd just need to check the independence assumptions. You've taken a complex topic and presented it in a very accessible way, not to mention demonstrated how Lo and MacKinlay's work can be applied in practice.

Thanks also for sharing your code - very useful indeed. I subscribe to the theory that markets can't be efficient simply because of the breadth of literature identifying past and current pricing anomalies. However, I intuited not backed up by anything other than anecdotal evidence and my own limited experience that the more developed a market, the more efficient it would likely be.

It follows that, liquidity constraints aside, emerging markets present significant trading opportunities. I was surprised that you found that the Dow Jones Index made the list of the indices least likely to be random.

Having said that, all the other indices listed were emerging markets. I would imagine that institutional traders would limit their participation in emerging markets due to capacity constraints, but I wonder if these markets are profitable stomping grounds for retailers and smaller portfolios. I haven't traded emerging markets, but its something else to explore. Finally, in response to Stryder's request for trading ideas: For example, in deciding which markets to trade, you could exclude any that fail the non-randomness test.

It may be interesting to apply these tests to markets using a rolling window of data in order to draw conclusions about the potential temporal characteristics of randomness, however you'd more than likely run into issues related to not having enough data for any window size worth considering.

I completely agree with your thoughts. I think there is something to be said for efficiency and bigger markets have much deeper liquidity and should be more efficient and, therefore, more random. As such, I was also surprised to see the DJIA pop up in the list of statistically significant results on the indices side. South Africa is an emerging market and from my own anecdotal experiences I would agree with your assertion that emerging markets might offer more trading opportunities but that those opportunities are smaller due to the size of the market and the lack of liquidity.

That said, I don't think that this logic extends to individual assets. Individual assets are non-random everywhere ;-. Just ask the big guns: Hey Ryan, thanks for the comment.

You are right, well spotted! It's actually the code which is the wrong way around so I've updated the GitHub gists. Thank you for explaining us a new statistical toll and providing us some good pieces of code and graphics. This definitely gives us some strong empirical evidence against the Markov property of financial time series. I would like to mention that some interpretations of the EMH postulate the market processes should be Martingales, which are not equivalent to Markovian processes, even though a large class of financial models belong to both classes the Brownian motion being our greatest example.

Some general martingales would be part of your third kind of random walk, thus they could be a possible alternative for us to reject the Null Hypothesis. It would be interesting to go one step further and test this weaker class of random walk. In your first definition of a random walk with stochastic volatility, I think you have made a mistake when you defined sigma squared as a Gaussian random variable.

In the following parts, you implemented what I consider a fair model, where sigma, not sigma squared, is normally distributed. Hi Guilherme, thanks for the comment. I would love to take credit but the statistical test isn't new. The credit for the idea is due solely to Lo and MacKinlay My objective is simply to make it a bit more accessible. A Markov process is a process without memory, whereas a Martingale process is one where the average value is constant over time. So I would say that a Martingale is a stricter form of the random walk hypothesis.

It is also applicable to models of volatility and interest rates more-so than equity what was considered here. But perhaps I am wrong in that interpretation.

Nevertheless, along with pareto and fractal models of stock market price evolution, this would be an interesting topic to explore. Hopefully I will find some time to cover those topics in the future: It took me a while to get the maths in the original paper, but when I did, I just had to write it up.

Thank you for the post. Really interesting topic and results. Wanted to add my 2 cents to the discussion. Results produced in your post say that the observed data is not compatible with random walk models in consideration.

These models can still be of use in some applications to quant finance. It depends on the problem you are solving with them. In real asset results section, it is incorrect to say that there are some stocks are least likely to be random basing your judgments on the magnitude of z-scores. Magnitude of z-scores does not reflect the size of an effect or the importance of a result. It is not correct to say that e. Public Storage is less likely to be random than Aflac Incorporated.

High z-scores are weak evidence against H0. Conversely, large effects may produce low z-scores. Let alone the selection bias in investment universe presented in this section. As mentioned, the fact that you see the data incompatible with random walk model as you defined it says nothing about the model.

random walk stock market prices

You might define a thousand more random walk models and there are dozens of them exist in quant world , and test them against data. In that case, you would get some data compatible with some of them, and some data - not really. Some models of random walk may have so many degrees of freedom and be so "sophisticated" that any data would be easily explained by them. And you see that we have a problem of induction here. It is interesting how biased we all are in terms of drawing conclusions from limited information.

We take time series of unknown properties. We take a model. We test it with data. Sometimes data is compatible with the model, sometimes it is not. We tend to draw overgeneralized conclusions about the model, while in reality statistical method gives us information about the data. We tend not to see the problem of induction.

In the end, these subtle moments in judgement make the difference. Instead, what we see that some data tested is not compatible with typical basic random walk models used by modern alchemists quants. In the end, I would like to share it with you and your readers the recent ASA's statement on statistical significance misuse and bad science http: You are quite right in your initial assessment: This refutation of the random walk hypothesis and any other refutation is model specific and I have not tested every possible model of a random walk.

You are also correct that the choice of p-value threshold is rather arbitrary in the scientific community and in this work as well. And regarding your statement that "magnitude of z-scores does not reflect the size of an effect or the importance of a result" - you're right this is true. The Z scores are more interesting than they are useful which is why I didn't recommend using them. Having conceded those points I think that the conclusion, "stock market prices do not follow random walks" still stands for four very simple reasons which I hope you will appreciate:.

Firstly, testing every possible model specification is impossible because it is easy to respond to each and every refutation of the random walk hypothesis with an even more convoluted and more nuanced random walk model which does pass even the most sophisticated randomness tests that's what has been happening for 30 years. But as with all disputed theories there always comes a point where the adapted theory becomes more complex than the simple, unmistakable conclusion that the theory is, quite simply, wrong.

We have reached that point with the random walk hypothesis: This is exactly what happened in the late 's when scientists kept coming up with exciting new theories like Lorentz Contraction to defend their misguided theory of absolute space.

So whilst it would be easy for me to make the conclusion that A: Ultimately A and B are empirically equivalent but, theory B has fewer assumptions.

All we are assuming with B is that there is some degree of predictability in stock market prices - what's so crazy about that? Anyway, as history has taught us it is the simplest theories which usually triumph [1].

That's Occam's razor for you. The second, third, and fourth reasons are quite trivial and make for less a less exciting debate but they must be said:.

The second reason is that this test extends to a large class of random walk models and, as discussed in the article, is really looking for statistically significant autocorrelations in the returns structure. If such autocorrelations exist then this represents a violation of all forms of the random walk hypothesis because the walks are therefore quite literally "not random" i.

The third reason is that the original and the replicated results are statistically significant to many different thresholds: In other words, all markets contain assets which are not strictly unpredictable. The evidence speaks for itself. The inclusion of more assets will not magically shrink the Z-scores of the assets I have already run the analysis on so the conclusion still stands and can only be bolstered by the inclusion of more assets into the universe. This isn't a trading strategy.

Speaking practically I don't have an unbiased dataset but if you do I welcome you to prove me wrong: Anyway, getting back to the philosophy of science I must admit that I disagree with your comment on a much deeper level because I despise the nihilistic approach to statistics and the science of uncertain systems which many people seem to have adopted in recent years.

Granted, statistical significance testing has flaws - what method doesn't? And is never coming to a conclusion and therein not furthering the scientific body of knowledge really better than potentially believing the wrong conclusion for some period of time?

Even when we recognize that this is the very nature of science to be wrong? I don't think so. Which is why I have no problem using statistical significance testing which has worked for all the great scientific minds who came before me.

I will, however, do my very best to make sure that all the assumptions are transparent. Furthermore, please note that despite my conclusions I have never once argued that the falseness of the random walk hypothesis undermines it's usefulness to quants. I use random walks every day, I just know not to believe them.

As they say, all models are wrong but some models are useful. The random walk hypothesis is one such more: Lastly, I just wanted to point out that your comment, whilst great for provoking critical thoughts for my readers, offers no practical advice whatsoever and is, therefore, not actionable.

So if you did have any suggestions for further analysis or perhaps a better methodology for performing the above analysis I would really love to hear it. Thanks again for your comment Pavel. Dear Stuart, How can i use the NNR neural network regression in an everyday lifestyle of trading in FX?

Am totally new t it and i'll need you to asisst me with it knowledge please. Sign me up for updates from this blog! Quantocracy is the best quantitative finance blog aggregator with links to new analysis posted every day. NMRQL is the quantitative hedge fund I'm a part of.

We use machine learning to try and beat the market. Turing Finance June 16, Hide Navigation Code Experiments Stochastic Models R4nd0m GitHub About This Site Turing Finance Stuart Reid Contact Suggestion Box. Algorithmic Trading Computational Finance Computational Investing. View the code on Gist. Tags A Non Random Walk Down Wall Street Computational Finance Lo and MacKinlay Random Walk Hypothesis Randomness Tests RStats Stochastic Process Variance Ratio vrtest. Share on Facebook Tweet This Share on Google Plus Pin This Email This.

Previous Story How to be a Quant Next Story Lossless Compression Algorithms and Market Efficiency? February 8, Reply. Was indeed worth, Gordon, to wait some extra time for your next TuringFinance post.

February 9, Reply. Looking forward to the follow on work. February 10, Reply. Thanks again Stuart, I very much enjoyed your post. Thanks Kris, love your blog as well! Hi Stuart, Excellent article. February 15, Reply. Hi Stuart, Excellent work! March 12, Reply. Those maths are so beautiful, I wanna return to math. Thanks for the post. March 24, Reply. Hello Stuart, Thank you for the post. Hey Pavel, Thanks for the insightful comment -- worth much more to me than just 2 cent: Having conceded those points I think that the conclusion, "stock market prices do not follow random walks" still stands for four very simple reasons which I hope you will appreciate: The second, third, and fourth reasons are quite trivial and make for less a less exciting debate but they must be said: May 21, Reply.

Turing Finance Mailing List For email updates when I post a new article.

Review: A Random Walk Down Wall Street - The Simple Dollar

It's free and I won't send you any spam. Friends of Turing Finance Quantocracy is the best quantitative finance blog aggregator with links to new analysis posted every day. Turing Finance on Twitter My Tweets. Turing Finance on Facebook. Most Popular Posts 10 misconceptions about Neural Networks Regression analysis using Python Hacking the Random Walk Hypothesis Random walks down Wall Street, Stochastic Processes in Python Stock Market Prices Do Not Follow Random Walks Algorithmic Trading System Architecture How to be a Quant.

Most Recent Posts Testing the Random Walk Hypothesis with R, Part One The Promise of Computing Lossless Compression Algorithms and Market Efficiency? Stock Market Prices Do Not Follow Random Walks How to be a Quant Hacking the Random Walk Hypothesis A Quant's view of CFA Level I.

Blogroll Alex Chinco's Blog Colah's Blog Compounding My Interests Ernie Chan Explained Visually Flirting with Models Gekko Quant Gestaltu Off The Convex Path Philosophical Economics Quandl Quant at Risk Quant Dare QuantInsti Quantocracy Quantopian Quantpedia Quants Portal QuantStart QuantStrat TradeR Robot Wealth Sebastian Rashka The Alpha Architect The Financial Hacker.

Popular Tags Agent-based Computational Economics Agent Based Model Algorithmic Trading Algorithmic Trading Systems Ant Colony Optimization Artificial Intelligence Computational Economics Computational Finance Computational Investing Correlation Analysis Economic Forecasting Neural Network Efficient Market Hypothesis Genetic Programming Machine Learning Neural network particle swarm optimization Portfolio optimization Quantitative Methods Randomness Tests Random Walk Hypothesis Regression Analysis RStats Self Organizing Map Stochastic Process Swarm Intelligence.

Disclaimer This is a personal blog. As such the opinions expressed here are my own and do not necessarily represent those of my employer. Furthermore, all information on this blog is for educational purposes and is not intended to provide financial advice.

inserted by FC2 system