In January of this year, I visited one of the leading organizations that practices ‘evidence-based investing’, Dimensional Fund Advisors (DFA).  This visit came about after 11 months of research on this new (for me) way of investing.  After learning as much as I could on the topic, I was convinced that it adds value for the long-term investor and began discussing and implementing their strategies with clients.

A few months later, I heard something on the radio that struck me.  Here was an advisor who had his own methodology, as many advisors do.  It was not the same discipline that I discovered, but he used the same term—evidence.  I realized that for the public, how would they know which evidence was really relevant, and which to rely on?  Even though in my research there was overwhelmingly legitimate evidence to justify adjusting portfolios, how would anyone else know the difference between this evidence and anyone else’s so-called evidence, really?  How do we all know which evidence to take seriously and which are simply false leads?

Evidence-Based Investing: A Never-Ending Story

First, it’s worth noting that academic research is never fully final, nor does it allow for absolutes in our application of it. As University of Chicago professor of finance and Nobel laureate Eugene F. Fama has said, “You should use market data to understand markets better, not to say this or that hypothesis is literally true or false. No model is ever strictly true. The real criterion should be: Do I know more about markets when I’m finished than I did when I started?” [Source]

With this in mind, here are nonetheless a number of important qualities to seek when assessing the validity of academic evidence:

No Agenda – Rather than beginning with a point to prove, ideal academic inquiry is conducted with no agenda other than to explore intriguing phenomena and report the results. It is then up to us practitioners to apply the useful findings.

Robust Data Analysis – The analysis should be free from weaknesses such as data that is too short-term or too small of a sampling; survivorship bias (wherein returns from funds that went under during the analysis period are disregarded); apples-to-oranges benchmark comparisons; or plain, old-fashioned faulty math.

Repeatability – Results should be repeatable in additional studies across multiple environments and timeframes. This helps demonstrate that the results weren’t just random luck or “data mining.” As AQR fund manager and founding principal Clifford Asness describes, “If a researcher discovered an empirical result only because she tortured the data until it confessed, one would not expect it to work outside the torture zone.” [Source]

Peer Review – To ensure that all of the above and more is taking place as required, scholars are expected to publish their detailed data sets, methodologies and findings in a credible academic journal or similar forum, so their credentialed peers can review their work and either agree that the results appear to be valid or refute them if they are not.

The Alternative: Data Foolery

If our emphasis on deep and diligent peer review sounds like it doesn’t really apply to you and your tangible wealth, think again. Echoing the sentiment about lies, damned lies and statistics, when faulty conclusions are inappropriately applied, the results can send countless investors astray, with real dollars lost.

The Importance of Peer Review

Substantive, meaningful peer review remains an essential component in separating real academic evidence from sloppy work that all too often occupies headline-grabbing news. Peer review also enables scholars to reference and build on their colleagues’ best work, which enables collective insights into important subjects to deepen and expand over time.

That’s why the evidence that survives the gamut of academic peer review, and has withstood the test of time is the evidence that we are most interested in applying to a set of orderly (if never certain) principles to guide our practical investment strategies.

Our Role as Advisers: Separating Fact from Fiction

When I think back on that radio show and all of the ‘evidence’ that is available to the public, I realize that this is a major part of my job:  to continuously learn, and surround myself with those that have an interest in continuously scanning the work being presented to the public, and translating the results with an eye toward helping investors appropriately view the big picture that emerges.

 

Of all the ways we can go about investing, which ones are expected to best serve our clients’ personal interests and financial goals? Equally as important, which are more likely to detract from our efforts? The evidence-based answers to these vital questions explain why we pay so much attention to qualities such as portfolio structure, cost management, patient trading, and global market exposure. It’s also why we advise against trying to chase or flee current market trends or pick popular funds and stocks, despite what seemingly knowledgeable talking heads seem to forever be recommending.

 

This is what I’ve been referring to as “evidence-based investing”. This is the essential difference between building a manageable investment strategy that reflects your personalized goals versus succumbing to chaotic, nerve-wracking guess-work in an ever-noisy world of market mayhem.  We choose the evidence.