|
|
How Natural Language Processing and Machine Learning Boost Fund Manager Selection Success

How Natural Language Processing and Machine Learning Boost Fund Manager Selection Success

Reiner Braun

Universität München
School of Management

Florencio López-de-Silanes

Université Côte d’Azur SKEMA
Business School and NBER

Ludovic Phalippou

University of Oxford,
Said Business School

Picture of Natalia Sigrist
Natalia Sigrist

Partner, Private Equity
Unigestion

Picture of Borja Fernández Tamayo
Borja Fernández Tamayo

Associate, Private Equity
Unigestion

Key points

Overview

Private equity (PE) has become an important component of investment portfolios across the globe. PE fund manager selection is one of, if not the, most important, yet challenging, decisions that investors in the PE asset class need to take. In this paper, we take forward our pioneering work on machine learning (ML) in private equity to examine the efficacy of combining ML algorithms and Natural Language Processing (NLP) to predict the performance of private equity funds.

The challenge of fund selection

Historically, investors have relied on their experience and, to a certain degree, on their gut feeling to tackle the challenge of fund manager selection within private equity.

At Unigestion, we believe this traditional approach can be enhanced with Artificial Intelligence-based techniques and result in superior returns. This belief is based on the potential of such techniques to remove human biases and AI’s ability to provide better understanding of the complex relationships between the factors influencing investment returns.

In 2019, we pioneered the use of Machine Learning (ML) algorithms to predict the performance of PE funds ex-ante using quantitative features related to investment strategy, market conditions, and the performance track record of PE funds[1].

Continuing with this line of research, we have partnered with the University of Oxford, SKEMA Business School, and the Technical University of Munich to broaden the previous work by examining the efficacy of combining ML algorithms and Natural Processing Language (NLP) techniques to predict the performance of PE funds[2]. The combination of these techniques has proved successful in predicting future stock price movements in public markets (Ke, Kelly, and Xiu, 2019). However, in the context of privately-held illiquid investment vehicles such as private equity funds, its application had been uncertain due to a number of factors:

  • The main disclosure document used by private equity fund managers to market their fund offering is the Private Placement Memorandum (PPM). While it describes the investment opportunity, provides backgrounds of the fund management team, and outlines the core terms of the fund, it is not subject to strict regulations. Therefore, fund managers have some flexibility regarding the content and presentation of information to potential investors. Completeness and transparency of the text cannot be guaranteed;
  • There is a large time span, usually 10-12 years, between the identification of NLP-based signals extracted from fundraising documents and the ultimate performance of a fund. This increases the importance of post-investment factors which cannot be taken into consideration ex-ante;
  • Given the private nature of these documents, the fund universe and the amount of available data to train algorithms are limited to proprietary databases, which results in relatively small samples.

At the same time, academic research has documented systematic differences in the way GPs source, select, invest, monitor, create value, and exit deals (Gompers, Kaplan, and Mukharlyamov, 2016). Moreover, there is evidence that some of these differences have informative power to explain fund performance (Biesinger, Bircan, and Ljungqvist, 2021).

These findings, coupled with the potential of the NLP techniques to identify reliable signals in texts, suggest that algorithmic performance predictions based on the analysis of PPMs can provide reliable insights to investors.

Combining NLP and ML to predict the probability of investment success

In order to extract informative signals from PPMs, we transform the text embedded in the “Investment Strategy” and “Investment Process” sections of PPMs using the Term-Frequency-Inverse Document Frequency vectorizer (TF-IDF)[1]. Then we feed three ML classifiers (Lasso, Random Forest, and Gradient Boosting) with the TF-IDF features to predict the probability that the ultimate fund’s Total Value to Paid-In ratio (TVPI) will exceed the median TVPI of funds raised in the same vintage and pursuing the same investment strategy (LBO or other private equity funds) reported by Preqin[2]. If this probability is higher than 0.5, the fund is labelled as successful.

We assess the performance of three ML classifiers with the ROC (Receiver Operating Characteristics) curve and the corresponding AUC (Area Under the Curve). This latter metric represents the probability that a randomly chosen successful fund (fund TVPI exceeds the median TVPI of its Preqin peers) is attributed a higher probability of being successful than a randomly chosen unsuccessful fund (fund TVPI is below the median TVPI of its Preqin peers). In these terms, an AUC of 0.5 is equivalent to flipping a coin. Thus, the closer the AUC is to 1, the better the model distinguishes between these two categories.

We leverage a dataset from the PPMs of 304 funds, with performance available as of June 2022, that were raised between 2003 and 2013 to train the three algorithms and test them on 72 funds raised between 2014 and 2016, so-called out-of-sample[3][4].

Figure 1 shows the AUC of the three algorithms. Gradient Boosting achieves the highest AUC (0.659) among the three algorithms. Overall, the AUC in the three analyses remains significantly above 0.5

Figure 1: Out-of-Sample AUC for the Three Algorithms

Source: Unigestion, based on Preqin data as of 30 June 2022

In order to mitigate concerns about “look-ahead bias”, we then restricted the training sample to funds raised in 2007 or earlier with performance information available as of December 2013. This reduced the training sample to 122 funds, while the test sample remained unchanged.

Figure 2 shows the ROC curve for the Gradient Boosting trained on the funds raised between 2003 and 2007 with performance information available as of December 2013 and compares it to the straight line, which corresponds to flipping a coin. The AUC resulting from back testing Gradient Boosting is 0.641.

Source: Unigestion, based on Preqin data as of 30 June 2022

Avoiding the availability bias while selecting fund investment opportunities

To compare our approach to fund selection against a simple “follow the crowd” approach, we benchmark the TVPI of the managers selected by the back tested Gradient Boosting classifier against those that are able to fundraise capital more successfully (i.e. perceived to be successful by investors) using the funds raised between 2014, 2015, and 2016. To proxy for fundraising success, we use the number of months needed to close a fund.

Figure 3 plots the size-weighted average TVPI of portfolios composed of the top two to nine funds per year selected by the Gradient Boosting classifier and by the measure of fundraising success. The red line depicts the size-weighted average TVPI of the 72 funds raised between 2014 and 2016. Across all portfolio sizes, size-weighted average TVPI of the Gradient Boosting is higher than the size-weighted average TVPI of the funds with the fastest fundraising speed.

For example, an investor committing capital to the top five funds per year selected by Gradient Boosting classifier would have achieved a 2.25x TVPI, whereas an investor putting capital into the five funds with the fastest fundraising would have generated a 2.09x TVPI. This way the quantitative model can help investors avoid the so-called FoMO (“Fear of missing out”) effect which can cause investors to make suboptimal choices.

Figure 3: Relative Performance of Algorithmically Selected Fund Portfolio TVPIs

Source: Unigestion, based on Preqin data as of 30 June 2022

Unlocking the “Black Box”

ML algorithms are frequently referred to as “black boxes” because the source of their predictions is difficult to interpret. However, recent developments in the ML field have proposed solutions to this challenge. We use one of these novel techniques – SHAP values developed by Lundberg and Lee (2017) – to determine which word combinations are more relevant to predicting GP quality.

We find that “operational (and) financial”, “network relationship” and “relationship (with the) management team” among other combinations of words, are positively associated with fund success. On the other hand, “investment criteria” and “company management” are negatively correlated with fund success. Figure 4 depicts the top-25 features in terms of variable importance in predicting fund success. Features in blue (red) are positively (negatively) correlated with fund success.

Figure 4: Most Relevant Combinations of Words to Make Predictions

Source: Unigestion, based on Preqin data as of 30 June 2022

We emphasise that ML algorithms use non-linear interactions among multiple word combinations to make predictions. As a result, we cannot state with a high degree of certainty whether a fund will perform well or badly because the description of its investment approach includes a specific combination of words. SHAP value merely helps us better interpret the overall model output. The beauty of ML lies in its ability to make sense of complex, non-linear relationships among various features and identify patterns humans cannot observe.

What next: combining NLP-based features and numerical features?

We believe that recent advancements in big data and AI will help private market investors reduce information asymmetries, democratise PE opportunities and create value through a more transparent and efficient investment evaluation and selection process.

The above results outline the potential benefits of using NLP-based methods in combination with ML algorithms in investment decision-making. While this study only relies on textual data to predict fund performance, Unigestion believes the combination of textual data with other numerical inputs can lead to better predictive capability and, consequently, better returns.

References

Biesinger, M., Bircan, C., & Ljungqvist, A. (2021). Value creation in private equity.

Braun, Fernández Tamayo, López-de-Silanes, Phalippou, and Sigrist (2023)

Gompers, Paul, Steven N. Kaplan, and Vladimir Mukharlyamov. “What do private equity firms say they do?.” Journal of Financial Economics 121.3 (2016): 449-476.

Ke, Z. T., Kelly, B. T., & Xiu, D. (2019). Predicting returns with text data (No. w26186). National Bureau of Economic Research.

Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.

[1] See Unigestion Perspective paper “A Quantitative Approach to Private Equity Fund Selection

[2] See Braun, Fernández Tamayo, López-de-Silanes, Phalippou, and Sigrist (2023)for the academic paper.

[3] TF-IDF vectorizer is a methodology used to represent words into numerical vectors. The TF-IDF of a certain word in a document accounts for the frequency of the word in the document as well as its frequency across all documents. Therefore, TF-IDF vectorizer captures the relative importance of words in a set of documents.

[4] Source: Preqin data as of 30 June 2022.

[5] Unreported results show that the predictive ability of the algorithms improve when we restrict the training sample to funds raised in 2003 onwards, instead of 1999 onwards, given that the vocabulary is more similar across years.

[6] A 80/20 training/test split is the Pareto Principle in Machine Learning. The training sample includes c.81% of the sample (i.e., 304 funds) and the test sample is composed of c.19% of the sample (i.e., 72 funds).

Important information

INFORMATION ONLY FOR YOU
This document has been prepared for your information only and must not be distributed, published, reproduced or disclosed (in whole or in part) by recipients to any other person without the prior written consent of Unigestion. It is neither directed to, nor intended for distribution or use by, any person or entity who is a citizen or resident of, or domiciled or located in, any locality, state, country or jurisdiction where such distribution, publication, availability or use would be contrary to law or regulation.

RELIANCE ON UNIGESTION
There is no guarantee that Unigestion will be successful in achieving any investment objectives. An investment strategy contains risks, including the risk of complete loss.

Except where otherwise specifically noted, the information contained herein, including performance data and assets under management, relates to the entire affiliated group of Unigestion entities over time. Such information is intended to provide you with background regarding the services, investment strategies and personnel of the Unigestion entities. No guarantee is made that all or any of the individuals involved in generating the performance on behalf of one or more Unigestion entities will be involved in managing any specific client account on behalf of another Unigestion entity.

NOT A RECOMMENDATION OR OFFER
This is a promotional statement of our investment philosophy and services only in relation to the subject matter of this presentation. It constitutes neither investment advice nor recommendation. This document represents no offer, solicitation or suggestion of suitability to subscribe in either the investment vehicles to which it refers or to any securities or financial instruments described herein. Any such offer to sell or solicitation of an offer to purchase shall be made only by formal offering documents, which include, among others, a confidential offering memorandum, limited partnership agreement (if applicable), investment management agreement (if applicable), operating agreement (if applicable), and related subscription documents (if applicable). Such documentation contains additional information material to any decision to invest. Please contact your professional adviser/consultant before making an investment decision.

Reference to specific securities should not be construed as a recommendation to buy or sell such securities and is included for illustration purposes only.

RISKS
Where possible we aim to disclose the material risks pertinent to this document. The views expressed in this document do not purport to be a complete description of the securities, markets and developments referred to in it. Unigestion maintains the right to delete or modify information without prior notice. The risk management practices and methods described herein are for illustrative purposes only and are subject to modification.

Investors shall conduct their own analysis of the risks (including any legal, regulatory, tax or other consequences) associated with an investment and should seek independent professional advice. Some of the investment strategies or financial instruments described or alluded to herein may be construed as high risk and not readily realisable investments, and may experience substantial & sudden losses including total loss of investment. These are not suitable for all types of investors. Unigestion has the ability in its sole discretion to change the strategies described herein.

PAST PERFORMANCE
Past performance is not a reliable indicator of future results, the value of investments, can fall as well as rise, and there is no guarantee that your initial investment will be returned.  Returns may increase or decrease as a result of currency fluctuations.

NO INDEPENDENT VERIFICATION OR REPRESENTATION
No separate verification has been made as to the accuracy or completeness of the information herein.  Data and graphical information herein are for information only and may have been derived from third party sources. Unigestion takes reasonable steps to verify, but does not guarantee, the accuracy and completeness of information from third party sources. As a result, no representation or warranty, expressed or implied, is or will be made by Unigestion in this respect and no responsibility or liability is or will be accepted. All information provided here is subject to change without notice. It should only be considered current as of the date of publication without regard to the date on which you may access the information. An investment with Unigestion, like all investments, contains risks, including total loss for the investor.

FORWARD-LOOKING STATEMENTS
This document may contain forward-looking statements, including observations about markets and industry and regulatory trends as of the original date of this document. Forward-looking statements may be identified by, among other things, the use of words such as “expects,” “anticipates,” “believes,” or “estimates,” or the negatives of these terms, and similar expressions. Forward-looking statements reflect Unigestion’s views as of such date with respect to possible future events and are subject to a number of risks and uncertainties, including, but not limited to, the impact of competitive products, market acceptance risks and other risks. Actual results could differ materially from those in the forward-looking statements as a result of factors beyond a strategy’s or Unigestion’s control. You are cautioned not to place undue reliance on such statements. No party has an obligation to update any of the forward-looking statements in this document.

TARGET RETURNS
Targeted returns reflect subjective determinations by Unigestion based on a variety of factors, including, among others, internal modeling, investment strategy, prior performance of similar products (if any), volatility measures, risk tolerance and market conditions. Target returns are based on Unigestion’s analytics including upside, base and downside scenarios and might include, but are not limited to, criteria and assumptions such as macro environment, enterprise value, turnover, EBITDA, debt, financial multiples and cash flows. Targeted returns are not intended to be actual performance and should not be relied upon as an indication of actual or future performance. 

USE OF INDICES
Information about any indices shown herein is provided to allow for comparison of the performance of the strategy to that of certain well-known and widely recognized indices. There is no representation that such index is an appropriate benchmark for such comparison. You cannot invest directly in an index and the indices represented do not take into account trading commissions and/or other brokerage or custodial costs. The volatility of the indices may be materially different from that of the strategy. In addition, the strategy’s holdings may differ substantially from the securities that comprise the indices shown.

ASSESSMENTS
Unigestion may, based on its internal analysis, make assessments of a company’s future potential as a market leader or other success.  There is no guarantee that this will be realised.

No prospectus has been filed with a Canadian securities regulatory authority to qualify the distribution of units of this fund and no such authority has expressed an opinion about these securities. Accordingly, their units may not be offered or distributed in Canada except to permitted clients who benefit from an exemption from the requirement to deliver a prospectus under securities legislation and where such offer or distribution would be prohibited by law. All investors must obtain and carefully read the applicable offering memorandum which contains additional information needed to evaluate the potential investment and provides important disclosures regarding risks, fees and expenses.

Legal Entities Disseminating This Document

United Kingdom
This material is disseminated in the United Kingdom by Unigestion (UK) Ltd., which is authorized and regulated by the Financial Conduct Authority (“FCA”).

This information is intended only for professional clients and eligible counterparties, as defined in MiFID directive and has therefore not been adapted to retail clients.

United States
In the United States, Unigestion is present and offers its services in the United States as Unigestion (US) Ltd, which is registered as an investment advisor with the U.S. Securities and Exchange Commission (“SEC”) and/or as Unigestion (UK) Ltd., which is registered as an investment advisor with the SEC.  All inquiries from investors present in the United States should be directed to clients@unigestion.com. This information is intended only for institutional clients that are qualified purchasers as defined by the SEC and has therefore not been adapted to retail clients.

European Union
This material is disseminated in the European Union by Unigestion Asset Management (France) SA which is authorized and regulated by the French “Autorité des Marchés Financiers” (“AMF”).

This information is intended only for professional clients and eligible counterparties, as defined in the MiFID directive and has therefore not been adapted to retail clients.

Canada
This material is disseminated in Canada by Unigestion Asset Management (Canada) Inc. which is registered as a portfolio manager and/or exempt market dealer in nine provinces across Canada and also as an investment fund manager in Ontario, Quebec and Newfoundland & Labrador. Its principal regulator is the Ontario Securities Commission (“OSC”).

This material may also be distributed by Unigestion SA which has an international advisor exemption in Quebec, Saskatchewan and Ontario. Unigestion SA’s assets are situated outside of Canada and, as such, there may be difficulty enforcing legal rights against it.

Switzerland
This material is disseminated in Switzerland by Unigestion SA which is authorized and regulated by the Swiss Financial Market Supervisory Authority (“FINMA”).

Document issued May 2024.

Related insight

Watch Fundamental Analyst, Fleura Shiyanova and Quantitative Analyst, Raphaël Zacharias debate the value of artificial intelligence in equities investing under the watchful eye of Xavier Marconnet, Head of Core AI Equities.

[…]

Unigestion, the global boutique asset manager focusing on investing in private equity’s emerging champions, has sold its successful investment in nurseries group Kindred to Livingbridge, a leading UK-focused mid-market private equity firm. It has also made a new investment in the company, investing alongside Livingbridge to support Kindred’s next stage of growth.

[…]

How can investors avoid missing out on attractive new investment opportunities when distributions from prior investments remain so lacklustre?  While exit activity in Q3 was flat vs. Q2, exit volume over the year-to-date remains more than 20% down vs. the same period in 2023.

[…]

The Unigestion Equities team showcases the latest progress they have made in leveraging AI to improve risk and alpha forecasts, and details how they have embedded these innovations into portfolio construction techniques.

[…]

Subscribe