Quant Screening: Three Questions for Investment Managers

Evaluating investment managers is a challenging endeavor. Why else would asset owners expend so much time and resources, often with the aid of consultants, to conduct manager searches? Proper manager selection and evaluation requires thorough due diligence, but a relatively simple filter can serve as a helpful initial screen of potential investment managers.

There are three basic questions that asset owners should ask of any quantitative manager before initiating their due diligence process with that manager. If a manager does not provide adequate responses, they may not merit further consideration. Though our focus is quantitative managers, the same questions also work for fundamental managers, especially in regards to the quantitative screens or signals they use in their investment processes.

1. What are the drivers of your investment process?

Investment managers should be able to explain what factors they consider most important to their investment decision making and provide some conceptual justification for them. For example, their equity factors ought to be economically intuitive and understandable rather than opaque or synthetic. As a case in point, consider the definition of the Value factor. A single understandable metric like price-to-book has advantages over hybrids such as a “Value” factor comprised of some combination of price-to-book and price-to-earnings.

Why avoid such hybrid approaches? First, the evidence that price-to-earnings is a rewarded risk factor has far weaker empirical support relative to price-to-book. Second, even if we were to use both metrics, a hybrid that combines the two individual metrics in some way, say 50% price-to-book and 50% price-to-earnings, does not make any economic sense. That is, what is the return stream of the hybrid “factor” a return stream of? Third, combining different metrics may give us exposures that we do not want. Finally, even if we combine factors as above, we will have to apply some form of weighting scheme, whether static or dynamic. But then we have to provide a justification for our weighting scheme. If our only justification is that it worked well in a backtest, then we are succumbing to the most fundamental error in both investing and statistics: We are basing what is supposed to be a generalizable investment strategy on an overfitted metric.

Thus, using a clear set of factors that makes economic sense and can be defended on conceptual grounds is critical to evaluating whether a manager has a firm and well-constructed investment process or is making investment decisions based on a flimsier set of considerations.

An important additional component of equity factor strategies is controlling the potential negative interaction effect among the various equity factors. For example, the stocks in a Value strategy have at least some exposure to Momentum and Size, among other factors. If the exposure is large and negative, then the strategy could wash away the premia that is being harvested from the Value exposure. Thus, managers must have a procedure in place that allows for factor tilts but controls for these negative interaction effects. If not, then a given strategy will stray from its stated mandate. Managers should be able to explain how their process ensures their intended exposures in the presence of interaction effects.

Finally, an important aspect of gauging a manager’s answers to our first question is their consistency. What if different members of an investment team, say the head of research and the senior portfolio managers, have divergent views on what the most important factors are in their investment process? Then maybe their strategy is not fully developed. This “inconsistency risk” can plague both quantitative and fundamental managers but is perhaps more common among fundamental managers who often have less disciplined investment processes relative to their quantitative peers.

2. What evidence is there that your investment process will be effective?

A well-constructed investment process should be validated through a large body of empirical evidence and a comprehensive array of statistical tests. For example, a quantitative process should be supported by very large data sets, tests that use different subsamples, and various types of simulations. All these validation methods should be documented, ideally in peer-reviewed journals. For example, the investment team at Scientific Beta has collectively published dozens of papers over the years that articulate its views and back up its approach to equity factor investing with evidence.

Why is publishing papers in journals useful? Because it gives the wider investment community the opportunity to evaluate an investment team’s ideas. And because the evaluators share no business interests with the authors, their assessments are more objective. Publishing research helps establish the legitimacy of quantitative investment processes. Not only does it provide a view into a manager’s investment methodology, but it also aligns a manager’s research efforts with genuine scientific practice.

In science, answers to questions are derived from consensus. That is, different research teams operating independently come to similar conclusions. Because of this, their results reinforce each other. If a manager cannot explain or provide any support, empirical or otherwise, why their process works, asset owners should take it as a red flag.

Of course, some investment firms do not publish their research because they say they want to protect the proprietary elements of their investment process, their ”secret sauce.” But that is not convincing. After all, other firms do publish their research without fear of misappropriation. Either way, a firm’s methodologies should be supported by both proprietary manager research and research external to the firm.

3. What risk controls are part of your investment process?

Ensuring that a strategy is delivering what it is supposed to and is not exposing itself to undesirable risks is integral to effective investment processes. For example, in an equity factor strategy, the goal is often providing focused exposure to one or more factors. So, a Value strategy’s return should be primarily driven by exposure to the Value factor. If a factor strategy’s return stream comes from other factors or the idiosyncratic risk of individual stocks, then unwanted risk exposures are creeping in. Thus, lack of risk control may lead to unintended consequences.

Model misspecification is a potential risk in any investment strategy. Quantitative strategies, in particular, often determine the asset weights in their portfolio using some form of optimization. While any optimization may be constrained, it nevertheless could unduly expose a portfolio to concentration risk in specific securities, regions, or sectors, among other types of risk. After all, no model is perfect, and every model processes inputs differently. As such, managers need to have controls in place to guard against any given model tilting the portfolio towards undesirable or overly concentrated exposures. Using more than one model to determine asset weights is one way to do this.

In applying any model, selecting what inputs to use is an important consideration. Does a process rely primarily on more stable metrics, such as volatility, or on more erratic variables, expected returns, for example? Managers must provide this information to assure asset owners that their models are robust and stable.

Concluding Thoughts

To be sure, these three questions are just the beginning of the due diligence process. However, as an initial filter, they are great starting points for evaluating any manager. If the answers to any of these questions are unsatisfactory, the manager’s process may have fundamental flaws, and the manager may be unsuitable for further vetting.

If you liked this post, don’t forget to subscribe to the Enterprising Investor.

All posts are the opinion of the author(s). As such, they should not be construed as investment advice, nor do the opinions expressed necessarily reflect the views of CFA Institute or the author’s employer.

Image credit: ©Getty Images / Alex Liew

Professional Learning for CFA Institute Members

CFA Institute members are empowered to self-determine and self-report professional learning (PL) credits earned, including content on Enterprising Investor. Members can record credits easily using their online PL tracker.