The Challenges of Benchmarking Private Equity Performance

Data Biases

Alternative investments, including PE as well as hedge funds, are well known for having biases in their datasets, mainly because of their illiquid and private nature. In their paper, Fung and Hsieh (2009) “Perspectives: Measurement Biases in Hedge Fund Performance Data: An Update” elaborate on the survivorship bias, the backfill bias (when an index provider adds a new fund to their index and “fills in” prior returns of that fund) and warn investors to be “mindful of how much of the hedge fund industry one is observing before passing judgment on the performance statistics of the hedge fund industry as a whole”. We have encountered similar biases within PE.

Discretion in Reporting 

PE managers enjoy a large amount of discretion when it comes to reporting performance in databases.

PE managers can decide whether to report figures in a specific quarter or not and to report only TVPI or only IRR.

• PE managers can request performance data contributed by one of their LPs to be removed.
• PE managers can decide whether to report figures in a specific quarter or not and to report only TVPI or only IRR. As explained previously, a standalone TVPI or IRR measure may be highly misleading.
• PE managers are required to report performance figures on a “net to investors” basis. “Net to investors” means net of: i) management fees and other fund expenses, ii) realised carried interest and, importantly, iii) carried interest provisions (i.e. the carried interest that the manager would receive if the unrealised portfolio of the fund was liquidated at current valuation). This is however not always the case. We came across numerous examples where performance is reported as gross of one (or more) of these three items.

Data Inconsistency

The best way to verify whether a fund’s performance is accurately reported would be to compare the reported metrics to the Capital Account Statements or Investor Quarterly Reports of the funds. These documents are unfortunately not available unless one is an investor in the fund or has close ties with its fund manager.

One of the PE performance databases however allows users to access the underlying cash flows of a specific fund (if available). Whilst this does not guarantee the accuracy of the data, it is of great help to understand how TVPI and IRR derive from the quarterly cash flows and NAV development. In some instances, we could not reconcile reported performance metrics with quarterly cash flows. For example, “Fund V”, a 2018 fund, reports an IRR of 33% whilst calculating the IRR from the cash flows and residual NAV gives an IRR or 22%. This represents a difference of 11%. Surprisingly, both reported and calculated TVPIs are identical, at 1.35x.

Data Scarcity 

Clearly, no PE performance database can give access to a complete dataset, due to the private nature of the asset class. Still, it is surprising to see the limited performance data coverage as compared to the size of the overall PE universe.

Out of the 25’818 funds raised, only 2’890 (11%) have reported at least a TVPI or an IRR over the last four quarters

Figure 2 shows the number of funds raised from 2010 to 2021 versus the number of such funds reporting performance data in Database X as at May 2022. Out of the 25’818 funds raised, only 2’890 (11%) have reported at least a TVPI or an IRR over the last four quarters. The percentage grows from 11% to 19% if we only consider funds having raised more than USD 25 million.

A sample size of about one-fifth of the size of the whole universe can only be fully representative if the sample is the result of a random selection.

A sample size of about one-fifth of the size of the whole universe can only be fully representative if the sample is the result of a random selection. However, in our case, the sample cannot be the result of a random selection because it derives from voluntary reporting.

Figure 2: Number of Funds Raised vs. Number of Funds Reporting Performance

Source: Unigestion and Database X, as at 13.05.2022.

Data Scarcity

As mentioned in the Overview section, we have witnessed a trend whereby worst performing funds tend to be less represented than best performing funds in PE performance databases. This suggests some form of survivorship bias.

Worst performing funds tend to be less represented than best performing funds in PE performance databases. This suggests some form of survivorship bias.

In Figure 3, we show the result of an analysis based on one of our most diversified evergreen PE programmes, which has made over 100 fund investments over 20 years. 77% of the funds with a TVPI above 2.0x reported at least one performance metrics in Database X over the last five quarters (as at May 2022). Conversely, only 25% of the funds with a TVPI below 1.0x reported at least one of performance metrics[2]. Therefore, the average performance of funds which report performance metrics is likely higher than the average performance of the whole universe of funds.

The average performance of funds which report performance metrics is likely higher than the average performance of the whole universe of funds.

Figure 3: Proportion of Funds Reporting Performance Based on their TVPI

Source: Unigestion and Database X, as at 13.05.2022.

Which Metrics are Used to Measure PE Performance?

The most commonly used performance metrics in PE are the TVPI and the IRR. Nonetheless, these metrics are prone to misinterpretation and are easy to manipulate through timing of cash flows and/or credit line usage.

Beware of IRR Pitfalls

This IRR and multiple combination indicates that the rate of return experienced by investors is likely to be much lower than 46%

Various academics have researched the dangers of using IRR as a performance measure. One of the best known research papers on the subject was written by Phalippou (2008) “The Hazards of Using IRR to Measure Performance: The Case of Private Equity”. In the paper, Phalippou highlighted two of the best performing funds in Calpers’ portfolio, according to the Calpers Performance Report as at June 2007: (1) Media Communication Partners II, L.P. (vintage 1992) with a TVPI of 4.5x and an IRR of 39% and (2) Doughty Hanson Fund II, L.P. (vintage 1995) with a TVPI of 2.0x and an IRR of 46%. Phalippou rightly pointed out that, in a simplified scenario where the fund contributes all of its capital on day 1 and receives all proceeds through one distribution, the capital would need to be invested for 4½ years compounding at an IRR of 39% in order to generate a TVPI of 4.5x. However, the capital would only need to be invested for less than 2 years compounding at an IRR of 46% in order to generate a TVPI of 2.0x. Whilst it can happen that some deals are exited after a short holding period of 2 years, it is highly unlikely that the overall portfolio of a PE fund is exited in an average of 2 years. In the case of Doughty Hanson Fund II, the short holding period implied by the combination of its TVPI and IRR somewhat contradicts the fact that the fund is still up and running after 12 years of existence. As Phalippou puts it: “(…) this IRR and multiple combination indicates that the rate of return experienced by investors is likely to be much lower than 46%.”

Applying Phalippou’s approach, we picked a more recent “real life” example, the one already mentioned on page 4. Fund V (intentionally kept anonymous), a 2018 vintage fund, reported a TVPI of 1.35x and an IRR of 33% at 31.12.2021 in Database X. Based on the combination of TVPI and IRR, Fund V has an implied duration[3] of slightly more than 1 year. In other words, a little over a year is required to reach a TVPI of 1.35x with a rate of return of 33%. Looking at it another way, the rate of return of 33% compounded over three years (the life of this fund so far) would lead to a TVPI of 2.4x, much more than the reported 1.35x. For a fund with 3 years of existence (vintage 2018), the implied duration of slightly more than a year suggests at least one of the following:

1. the fund used a credit line;
2. the fund was raised in 2018 but only started investing in 2020 in deals rapidly valued upwards;
3. the fund received a large early distribution coming from either a quick exit, a dividend financed by debt, or a return of capital call inaccurately classified as a distribution

These three actions can artificially inflate the IRR, especially in the first years of a fund’s life.

Any one of these three actions can artificially inflate the IRR, especially in the first years of a fund’s life. Knowing that TVPI and IRR are usually considered together (with equal weight) in the construction of default quartile benchmarks in PE performance databases, an artificially inflated IRR can materially reduce the relevance of a given benchmark. Worse, if a fund only reports its IRR (and no TVPI), some databases will solely consider such IRR (with double weight) in their default quartile benchmarks, making the benchmarking exercise even less accurate.

The combination of TVPI and IRR does not tell the full story.

Thus, without further information, the combination of TVPI and IRR does not tell the full story and sometimes, simply does not make mathematical sense.

The Distorting Effect of Credit Lines

Credit lines were initially introduced to simplify the cash flow management of PE funds. A credit line allows new portfolio acquisitions to be closed and/or fund expenses to be paid without the need to issue multiple capital calls to investors upfront. This feature of “grouping” capital calls is generally appreciated by investors as it simplifies their operational burden. Propelled by the low interest rate environment post-Global Financial Crisis, PE managers have been incentivised to maximise credit line usage.

TVPI and IRR can be astronomical.

In our study, we came across a wide range of PE managers having issued very few capital calls in the first 18–24 months following the first closing of their funds, while reporting an ever-increasing NAV. Most probably, these managers finance their first acquisitions with credit lines rather than with capital from their investors. In such cases, TVPI and IRR can be astronomical. This is because, if unrealised gains exceed the cost of the credit line and limited investors’ capital has been drawn down, net gains have been generated using next to no capital. Hence, funds using large and long dated credit lines generate very different performance compared to funds not using credit lines, especially during the early years of their life. Obviously, credit line usage is not reported and the associated risk cannot be assessed: the default risk increases with credit line usage, as credit line providers are typically more senior than fund investors with respect to first recourse on distributions. In an ideal world, details on credit lines should be available in PE performance databases, in order for investors to make performance metrics more comparable.

In Figure 4, we show the cash flows of Fund Y, a 2015 vintage fund (intentionally kept anonymous), as reported in Database X as at May 2022. In Figure 5, we analyse the impact of a credit line of only 10% of fund size on the IRR of Fund Y, in three scenarios: i) no credit line, ii) 6-month credit line and iii) 18-month credit line. It appears that using an 18-month credit line improves the fund IRR by 1.9% as compared to using no credit line.

Using an 18-month credit line of only 10% of fund size improves the IRR by 1.9%.

IRR is therefore very sensitive to the use of a credit line. Hence, PE managers who are predominantly benchmarked on IRR are incentivised to use credit lines to optimise IRR, rather than relying on the merits of “old fashioned” portfolio performance.

Figure 4: Cash Flows of Fund Y – Aug 2015 to Sep 2021

Source: Unigestion and Database X, as at 13.05.2022.

Figure 5: Impact of Credit Line Maturity on IRR of Fund Y (vintage 2015)

Source: Unigestion and Database X, as at 13.05.2022.

The Distorting Effect of Recycling Distributions

“A relevant indicator of reinvestment level is the Distributed to Paid-In (DPI) ratio.

Another point to consider when analysing a fund TVPI is the reinvestment level, also known as the recycling level. This mechanism enables PE managers to reinvest all or part of the distribution proceeds received during the fund investment period into new investments, in lieu of having to call additional capital from investors. These reinvestments will likely produce gains and thus boost the TVPI by increasing the numerator (Total Value) while keeping the denominator (Paid-In) unchanged. It is difficult to judge the extent to which a fund has recycled capital without comparing the fund size to the total amount invested in underlying portfolio companies. This information is unfortunately not easily available. However, a relevant indicator of reinvestment level is the Distributed to Paid-In (DPI) ratio. If the DPI of a fund remains low after 4-5 years of fund existence, it is likely that the fund manager reinvested rather than distributed proceeds to investors.

The Importance of Taking PIC into Account

A TVPI provides more information on the performance of a PE fund than an IRR, provided it is associated with the level of paid-in capital (PIC)

From our point of view, a TVPI provides more information on the performance of a PE fund than an IRR, provided it is associated with the level of Paid-In Capital (PIC) from investors relative to commitment. On a standalone basis, a TVPI of 2.0x does not provide sufficient information about the return of a PE fund from an investor standpoint. However, combining TVPI with PIC provides a clearer picture of performance in terms of absolute gain, as illustrated in Figure 6.

Figure 6: Absolute Gain for a USD 10 Million Commitment

Source: Unigestion, based on a commitment of USD 10 million for both Fund A and Fund B.

In Figure 6, we highlight the relationship between TVPI and PIC, and the associated gain. By benchmarking on TVPI (as well as on IRR), Fund A beats Fund B hands down. However, when accounting for PIC and the related absolute gain, Fund B looks much more attractive. An investor should therefore prefer Fund B, if it aims at maximising return on committed capital.

An accurate benchmarking should at least consider the following metrics: TVPI, IRR, DPI and PIC

In summary, an accurate benchmarking should at least consider the following metrics: TVPI, IRR, DPI and PIC. Adding DPI and PIC is crucial to understand the potential usage of credit lines and reinvestment level. PIC also provides information on whether a fund called capital in an efficient manner. A fund which has for example called only 60-70% of committed capital after 10 years of existence would not optimally use the capital of its investors. Indeed, the investors would have likely recorded the entire committed amount as a liability and they would have likely paid management fees on the entire committed amount, not just on what was called. Finally, DPI also provides important information on how much “locked-in” value is contained within the TVPI.

Conclusion

While benchmarking is an important component in any PE due diligence and selection process, there are many caveats to consider.

First, in any benchmarking process, the constituents of peer groups should be analysed in details to verify whether such groups are constructed with relevant peers or not. In addition, investors should be aware of the methodology used to calculate quartile ranking and it should reflect the objectives of the investor.

Second, benchmarks may not be wholly representative due to i) data inconsistencies in the self-reported data, ii), data scarcity where only a small proportion of the fund universe reports performance metrics and iii) survivorship bias. Specifically, there is clear evidence that the performance metrics of the best performing funds are more diligently reported than those of worst performing funds.

Finally, TVPI and IRR, the most commonly used performance metrics, are easily distorted by the use of credit lines or the recycling of distributions proceeds. Therefore, DPI and PIC should also be considered as they provide important additional insights.

That being said, we do not mean to discard the benchmarking exercise through PE performance databases. We rather intend to raise awareness of the common pitfalls and reiterate the following in order to conduct more robust benchmarking exercises:

1. At the very least, seek to identify the constituents of the peer group proposed by the database and eliminate the obvious “odd ones out”.
2. Better, create your own bespoke peer group with funds that have truly comparable strategies and represent real investment alternatives for you.
3. Verify that the PE managers in the peer group systematically report performance, i.e. every quarter, for all their funds (Fund I, Fund II, Fund III, etc.).
4. Use reported cash flows to validate performance metrics.
5. Ensure that reported performance is truly “net to investors” by cross-checking with other sources (for example quarterly reports if available).
6. Aggregate and cross-check data from multiple databases.
7. Consider all performance metrics available: TVPI, IRR, DPI and PIC.
8. Calculate absolute gains to remove the distorting effect of credit lines on TVPI.
9. Identify the amount of recycled distributions and adjust the TVPI accordingly to remove the distorting effect of recycling distributions.
10. Complement performance benchmarking by considering a range of factors such as team size and experience, consistency of track record over time, quality of investment process and depth of ESG approach.
11. In summary, use benchmarks responsibly!

[1]  See analysis here. 

[2]  Source: Unigestion, based on data in Database X, as at May 2022.

[3] Implied duration is calculated by solving for ‘d’ where (1+IRR)^d = TVPI, interpreted as the number of years over which to compound the IRR in order to reach the TVPI. For example, the implied duration of a fund valued at a TVPI of 2.25x with an IRR of 50% is 2.0 years, i.e. (1+50%)² = 2.25.