The Need To Measure The Effect Of Merger Policy And How To Do It

ECONOMIC ANALYSIS GROUP
EAG Discussion Papers are the primary vehicle used to disseminate research from economists in the Economic Analysis Group (EAG) of the Antitrust Division. These papers are intended to inform interested individuals and institutions of EAG's research program and to stimulate comment and criticism on economic issues related to antitrust policy and regulation. The analysis and conclusions expressed herein are solely those of the authors and do not represent the views of the United States Department of Justice. Information on the EAG research program and discussion paper series may be obtained from Russell Pittman, Director of Economic Research, Economic Analysis Group, Antitrust Division, U.S. Department of Justice, BICN 10000, Washington, DC 20530, or by email at russell.pittman@usdoj.gov. Comments on specific papers may be addressed directly to the authors at the same mailing address or at their email address. Recent EAG Discussion Paper titles are listed at the end of this paper. To obtain a complete list of titles or to request single copies of individual papers, please write to Janet Ficco at the above mailing address or at janet.ficco@usdoj.gov or call (202) 3073779. Beginning with papers issued in 1999, copies of individual papers are also available from the Social Science Research Network at www.ssrn.com. Abstract In this article, I explain the inadequacy of our current state of knowledge regarding the effectiveness of antitrust policy towards mergers. I then discuss the types of data that one must collect in order to be able to perform an analysis of the effectiveness of antitrust policy. There are two types of data one requires in order to perform such an analysis. One is data on the relevant market pre and post merger. The second is data on the specific predictions of the government agencies about the market postmerger. A key point of this article is to stress how weak an analysis of only the first type of data is. The frequent call for retrospective studies typically envisions relying on just this type of data, but the limitations on the analysis are not well understood. As I explain below, retrospective studies that ask whether prices went up post merger are surprisingly poor guides for analyzing merger policy. It is only when the second type of data is combined with the first type that a reliable analysis of antitrust policy can be carried out. There is a need both to collect the necessary data and to analyze it correctly. I. Introduction  The Need for Measures The antitrust policies of the United States should be reviewed periodically to make sure that the policies are promoting not impeding competition. The recent Antitrust Modernization Commission performed just such a function and concluded that U.S. antitrust policy was basically sound, though the report makes a number of recommendations for improvement. That report relied largely on the qualitative judgment of learned practitioners and scholars. Although the qualitative judgment of such people is important, it is no substitute for quantitative studies and measures. The dearth of such studies and measures means that there is no reliable guide for determining whether our antitrust policy is too lax in some areas and too stringent in others. I will concentrate my discussion about measures of antitrust policy effectiveness on merger policy because there are numerous merger investigations each year, and therefore a quantitative study of merger policy is possible, while that is not true of nonmerger policy where at most a handful of cases are brought each year. I will focus on the mergers that the government chooses to investigate (e.g., those that receive a second request for information under the HartScottRodino Act) and assume that the others raise no competitive concerns. This is of course, a simplification, but not an unreasonable one, especially for an initial analysis of the problem. In this article, I explain the inadequacy of our current state of knowledge regarding the effectiveness of antitrust policy. I then discuss the types of data that one must collect in order to be able to perform an analysis of the effectiveness of antitrust policy. There are two types of data one requires in order to perform such an analysis. One is data on the relevant market pre and post merger. The second is data on the specific predictions of the government agencies about the market postmerger. A key point of this article is to stress how weak an analysis of only the first type of data is. The frequent call for retrospective studies typically envisions relying on just this type of data, but the limitations on the analysis are not well understood. As I explain below, retrospective studies that ask whether prices went up post merger are surprisingly poor guides for analyzing merger policy. It is only when the second type of data is combined with the first type that a reliable analysis of antitrust policy can be carried out. There is a need both to collect the necessary data and to analyze it correctly. II. Why a Comprehensive Study of Antitrust is Needed Several commentators feel passionately that antitrust is too lax (e.g., New York Times) while some claim just the opposite (e.g.,Wall Street Journal), but passion is no substitute for evidence. By evidence I mean numbers or studies relying on quantitative data. Imagine that the Federal Reserve Board was trying to control the rate of inflation but did not have access to price statistics. Instead it relied on the opinions of a few nonrandomly chosen shoppers about how fast they thought prices were rising. I suspect that the Fed would do a much poorer job of controlling inflation than it now does. Moreover, it is possible that in the absence of reliable quantitative information, monetary policy could be heavily influenced or could be perceived to be influenced by the ideological views of the people running the Federal Reserve Board. There are some data on antitrust but they mainly relate to the frequency of enforcement actions, such as the number of cases brought. Those numbers are often analyzed, yet knowing how many cases are brought tells one little about whether there are too few or too many cases brought, and whether the right cases are being brought. Unfortunately, the problem of figuring out what statistics to collect in order to determine whether antitrust policy is working well is a much harder problem than that facing the Fed in its price data collection efforts. We suggest below what statistics one should collect, and describe the type of analyses one could perform with such data. Surprisingly, the analysis is anything but straightforward. Simple tests, based on sensible intuition, turn out to be misleading, while slightly more refined tests work well. A fundamental question facing enforcement officials is whether their current merger policy is too lax or too stringent. This question is different from whether a particular merger enforcement decision was correct. It is rather asking whether overall the government is allowing too many or too few mergers. Specifically, is the government analysis of mergers systematically biased? The answer to this question requires one to identify the types of government analyses that are correct and those that are wrong and the circumstances that lead to the most errors. Because this question deals with overall policy, it can only be answered by systematically examining all (or a sample of) mergers. Determining whether in one particular case, the government turned out to be correct or not tells one very little about whether overall government policy should be altered. Indeed, even if the government policy is set exactly right, it would still be true that the government would make random errors in cases. Although it would be desirable to minimize such errors, it is not true that the presence of such errors indicates a systematic bias in policy. This last point, though perhaps obvious, often seems to get ignored when one hears the frequent calls for retrospective studies of past merger. Because it is an important point, I will highlight it and other key points by labeling them "Result". Result 0. A retrospective study of an individual merger tells the analyst nothing about whether there is a systematic bias in antitrust policy. At most, the analyst can learn whether a particular merger turned out to harm consumers. But, even that observation tells one little about whether the decision to allow the merger was a wise one based on the information available at the time of the merger. Even a merger that has a zero predicted price increase will turn out, for random reasons, to raise price about half the time. In the next section, I first discuss the types of measures that one might use to gauge the effectiveness of merger policy and the accuracy of the merger analysis that government agencies use. I then discuss biases that are likely to arise when analyzing such measures. Failure to use information about whether the mergers are challenged causes one to reach incorrect conclusions. This last point, which has to do with what economists call a selfselected sample, seems to have escaped notice and causes retrospective merger reviews to be quite imprecise guides to policy. Subsequent sections show how to apply the analysis to increasingly realistic settings. III. The Sample Selection Problem and How to Do the Analysis Correctly There are two types of data one needs to evaluate antitrust policy. The first is market data pre and post merger. The second is the enforcement agency's predictions of the merger. Any analysis of the data must account for the fact that the merger data one examines  and, to repeat, I only look at mergers that have received a "second request" for more information  already reflects a decision by the government agency about whether to challenge the merger. Virtually all of the data on mergers will represent mergers that the Department of Justice has decided not to challenge.^{(1)} Therefore, as is now well understood since the work of Nobel Laureate James Heckman and others, the analysis based on such a sample may yield misleading results unless one explicitly understands the implications of how the sample is chosen. Let me explain. Suppose that a merger is proposed and that if the merger goes through the expected price change, ΔP, from the merger is drawn from some underlying probability distribution, ceteris paribus. (For simplicity, normalize the initial price to 100 so that ΔP can be thought of as a percentage price change. If the government agency knew this ΔP, then it would allow the merger if ΔP 0 and would challenge the merger if ΔP > 0. This would be the optimal merger policy.^{(2)} Of course, the government could be a poor predictor of ΔP and may make a systematic error, S, in forming predictions. If ΔP_{DOJ} is the DOJ's prediction of ΔP, then
If S > 0, the DOJ is systematically biased. It always overpredicts ΔP and therefore is too stringent in challenging mergers. If S < 0, the DOJ is systematically biased in underpredicting ΔP, and is therefore too lax in allowing mergers. Consider the case where S = 0. The shaded part of Figure 1 indicates which mergers the government allows to go through unchallenged. Since all mergers in the shaded part have ΔP 0, an analysis of unchallenged mergers will reveal that on average ΔP is not zero, as some might expect, but negative! Result 1: If the government is unbiased (S = 0), retrospective studies of unchallenged mergers should be expected to indicate that on average post merger price falls. Similarly, if the government is too stringent (S > 0), an analysis of unchallenged mergers should be expected to indicate that postmerger prices fall, since the only unchallenged mergers are those with negative ΔP less than S. Therefore, one cannot conclude that merger policy is too stringent merely from observing that postmerger prices fall. Result 2: If the government is too lax (S <0), then it is still quite possible that E(ΔP)<0 where E(ΔP) is the expectation across all unchallenged mergers of ΔP conditional on a merger being unchallenged . Figure 1 The reason for Result 2 is easy to see in Figure 1. If the boundary between "allow" and "challenge" moves away from 0 and to the right (S < 0)^{(3)} , then it will still be the case that many unchallenged mergers will have ΔP < 0. Only when S gets sufficiently large negatively will E(ΔP) > 0. We therefore have: Result 3: For a sufficiently biased policy (S < 0) of laxity, E(ΔP)>0. The consequence of Results 13 is that retrospective studies of price change that focus on the average price change will not be a very good way of evaluating merger policy. It is correct that if one finds that ΔP is on average positive, then we know the government policy is too lax, but this is a very weak test. The reason is that we know from Result 2 that retrospective studies of price change can show negative price increases even if the government policy is too lax. A much better test of government bias would be to combine pre and post merger price data, ΔP, with the DOJ's predicted price changes, ΔP_{DOJ}, and then explicitly calculate S. Notice that from the way (1) is set up, the estimate of S as the average of ΔP_{DOJ} minus ΔP over all unchallenged mergers will precisely estimate S. We have: Notice that from the simple assumptions underlying equation (1), it follows that S is estimated correctly for each merger as ΔP_{DOJ }  ΔP. The contrast between the precision in Result 4 and imprecision in Results 1 or 2 emphasizes why combining pre and post merger data with data on the enforcement agency's assessment is necessary to avoid the imprecision of Results 1 or 2. (In the next section, we show that the same type of results survives in a more realistic setting in which the bias is regarded as a random variable.) According to Result 1, one cannot conclude that antitrust policy is too stringent merely by observing whether price falls postmerger. According to Result 2, retrospective merger studies may fail to detect a lax antitrust policy because retrospective studies may show no price increase post merger. But Result 4 shows that a lax antitrust policy will be detected by more careful studies that combine pre and post merger data with data on the DOJ predictions at the time of merger.^{(4)} IV. Evaluation of Antitrust Analyses The previous section explained the need to combine pre and post merger data with DOJ data on their evaluation of the merger. The discussion focused for simplicity only on price. But, of course, during the course of an investigation there are many types of analyses that are done. Each of them can be analyzed for their accuracy, as I now explain. In many merger investigations, considerations of entry, product repositioning, ability of buyers to vertically integrate, and predictions of price and market share from merger simulations are all used to guide the analysis. Yet we have few if any studies investigating the validity of any of these types of analysis. For example, suppose that in some of the unchallenged mergers, one finds that the reason for the government agency not challenging the merger is related to the likelihood of entry. We should test whether in fact entry turns out to be an important constraining effect on price. How often does entry occur in cases where it is alleged to be easy and therefore a tight constraint on price? When it does not occur, is that because price did not rise? Do government agencies too willingly accept claims that entry can constrain price? Similarly, in cases where the government relies on merger simulation, how well do the price predictions and market share predictions turn out? In cases where the government relies on product repositioning, does such repositioning in fact occur after the merger? How does the frequency of large buyers' using vertical integration as a means to protect themselves against price increases compare to the frequency of the government's reliance on vertical integration as a constraint on a merger's ability to raise price? Again, when it does not occur, is that because price did not rise? Without such studies, there is no way to judge and improve the analysis underlying most merger policy. In order to perform these types of studies, the DOJ at the end of each merger investigation should fill out a data sheet that summarizes each of their analyses, including price, entry, product and predictions, so that their predictions can be compared to actual industry behavior. Of course, one would have to account for how conditions post merger have changed (e.g., cost may have exogenously risen, demand conditions may have changed, product quality may have changed, etc.) and figure out how that would change the DOJ prediction, but that type of adjustment is routinely done in econometric studies. Such adjustments no doubt complicate the analysis, but are essential. V. Extension of Results — A More Realistic Model In this section, we show that our major results persist in a more realistic and complicated model of bias. We also discuss how to use a dataset on challenged mergers in addition to the dataset on unchallenged mergers.
Using the same notation as before, we previously defined S, the systematic bias, by the equation
Notice that in eq. (1), ΔP_{DOJ} and ΔP are both expected prices not the actual price in the future. In fact, there will typically be many new events that occur between the time of the merger review when the predictions are formed and the time when the actual price is observed. For any particular merger, ceteris paribus, the relation between the actual price change, ΔP*, and the predicted is
where E is a random variable with expectation 0, independent of ΔP. If one can observe ΔP* for many mergers, then it follows that an estimate of the average ΔP will be given by the average of ΔP* across all mergers since the average of E will, in expectation, equal 0. The upshot is that the addition of E in (2) creates no estimation complications and the procedure described in the earlier section where we ignored E is a valid one for calculating expected price changes.^{(5)} Eq. (1) has the unrealistic implication that the DOJ is off by exactly S in its expected price prediction in each merger. A more realistic model would allow for any systematic bias to be random across mergers, but to have a common average, S. For example, one can think of the DOJ being systematically biased upward in its price predictions on average, but on some mergers it is less so, while others it is more so. For example, one could think of the economist choosing one of many modeling techniques and that the randomness arises because the modeling techniques vary. We therefore, rewrite equation (1):
where is a random error independent of ΔP and S with expectation equal to 0. The consequence of this more realistic setup is that the simplicity of Figure 1 disappears (or is reduced) and a more sophisticated analysis is required. The reason the simplicity vanishes is because the set of unchallenged mergers will now be more complicated to determine than in Figure 1. For example, if s = 0, then under the previous assumptions, as Figure 1 shows, the set of all unchallenged mergers are those in the shaded area to the left of the ?P = 0 line. Now, however, it is possible that some merger where ΔP < 0, may be challenged if the error is sufficiently positive. The probability that a merger is challenged will still be monotonic in ΔP, but it will not be either 0 or 1 as in Figure 1. Similarly, a very bad merger (ΔP very high) has a chance of being unchallenged if is sufficiently negative. The net effect is that unlike before where
f_{ΔP} = the probability density of ΔP, and (ΔP) = probability a merger with actual predicted price increase of ΔP will go unchallenged. Still assuming for illustration purposes that S = 0, it is straightforward to calculate (ΔP) as the probability that ΔP_{DOJ} = 0 or that ΔP + 0 or  ΔP which can be written as where f_{}() is the probability density of . (ΔP) is monotonic in ΔP. By comparing eqs. (4) to (5), we notice that negative ΔP's in eq. (4) no longer receive a weight of 1, but instead the lower weight, (ΔP), and positive ΔP's no longer receive a weight of 0, but instead the positive weight (ΔP). This means that having randomness in will tend to increase any estimate of the post merger price increase. Indeed, depending on the distribution of , one could observe a post merger price increase even though S = 0. In other words, even if there is no systematic bias at all in the DOJ's predictions, retrospective studies could very well show that there are on average price increases for unchallenged mergers. This confirms the results from the earlier analysis that retrospective merger studies that focus only on the average of ΔP are quite weak in their implications for the evaluation of merger policy. The intuitive reason for this last result is that those mergers that are unchallenged will tend to be dominated by those where the DOJ was unusually low (negative ) in their price predictions and accordingly allows some mergers with high ΔP to get approved. If there are many such mergers with high ΔP, then the average ?P over unchallenged mergers will be positive. The following simple single numerical example illustrates the point. Suppose that s = 0, so that the DOJ is unbiased. Supposed that ΔP can take on one of two values with equal probability, $5 or $10. In the absence of , the DOJ would challenge the merger with ΔP = ^{$}10 and leave unchallenged the merger with ΔP = ^{$}5. Retrospective studies of unchallenged mergers will show that postmerger pricing is $5 below premerger levels. Now suppose that we introduce the error which takes on one of two values 11 or +$11 with equal probability. There are now two possibilities for each merger outcome. For the merger where ΔP = $5, the DOJ will predict a price change of either $16 or +$6, so it allows that merger to go through with probability ½. Similarly for the merger with ΔP = $10, the DOJ will predict a price change of either $1 or $21, so again it allows the merger to go unchallenged with probability ½. Hence, even when merger policy is unbiased (S = 0), retrospective studies of unchallenged mergers will now find that on average the price increase is ½ (5) + ½ $10 = $2.5! This example is meant to be illustrative only. However, it underscores the limitations of the inferences that one can draw about merger policy from retrospective studies. In the earlier analysis, we showed how a combination of pre and post merger data together with data from the DOJ analysis can provide a much better guide to assessing merger policy than retrospective studies alone. Does that remain true in the more sophisticated model? The answer is, yes, though with some caveats. For any proposed merger, it follows from (3) that
For mergers that are not challenged, we know that ΔP_{DOJ} 0, or ΔP + S + 0, or
This means that for unchallenged mergers the upper tail of is not observed, hence, it will not be true that E() = 0, but instead will be skewed toward being negative and hence E(/unchallenged merger) < 0. Therefore, if one estimates S by averaging ΔP_{DOJ}  ΔP/ over all unchallenged mergers, it follows from (6) and (7) that the estimate, , of S will have the property that E() (b) Challenged MergersAnother SelfSelected Sample Finally, we turn to another selected sample that we have so far ignored — namely those mergers that are challenged, go to court, and are allowed to proceed. To understand why this is the only other available data set for analysis consider Figure 2 which diagrams the major possible outcomes from merger investigations. If the DOJ predicts  perhaps after a "fix" to the terms of the merger  no price increase from the merger, (ΔP_{DOJ}0), then the merger is unchallenged and goes forward. This set of mergers provides data (labeled dataset 1 in Figure 2) that we have already discussed extensively. But in addition to unchallenged mergers, there are mergers that the DOJ challenges (ΔP_{DOJ} > 0). In those, several outcomes are possible, as Figure 2 illustrates. The parties could alter their proposed merger so that the new merger is unchallenged and thereby becomes part of dataset 1.^{(7)} The parties could abandon the merger, leading to dataset 2 which contains no information on completed mergers. Alternatively, the parties could go to court, and the court could enjoin the merger. This set of mergers, dataset 3, also contains no information on completed mergers. The final possibility is that the court sides with merging parties and allows the merger to go through. This set of mergers  that we have ignored so far  comprise dataset 4 which we now analyze. The set of mergers in dataset 4 results from unsuccessful court challenges is a selfselected sample, like dataset 1. It represents mergers that have the property that ΔP_{DOJ} > 0. The analysis of dataset 4 has some similarities to that for dataset 1, though there is now the complication of the court's decision. In the case where the DOJ bias is nonstochastic (i.e., eq. (1)) and under the assumption that the court is unbiased, the court will allow a merger to proceed only if ΔP_{DOJ} 0. Hence, we return to a similar type of result that we had previously in that the expected price change of a completed (challenged) merger pre and post merger should be negative. But this time, this finding is independent of S since the court is deciding which mergers go forward. Again, as before, S can be calculated assuming one also has data on the DOJ predictions. [Even if one does not have data on ΔP_{DOJ}, one does observe that the DOJ decided to sue (ΔP_{DOJ}> 0) and one also observes that the court has concluded that ΔP < 0. Even if one does not observe ΔP_{DOJ}, one can, with sufficient structure on the model, estimate S in a manner similar to that described in Fn 4.]^{(8)} If we now add the complication that the bias, S + $\eta $, is stochastic with ? being random with mean 0, we obtain from eq. (3) that the challenged mergers that comprise dataset 4, have the property that (S + $\eta $) will tend to be above average. The reason is that for a challenged merger ΔP_{DOJ }> 0 which implies ΔP + (S + $\eta $) > 0, or S + >  ΔP or that the expectation of ? will be positive (i.e., >  (S + P)), since it is truncated at the lower end. Intuitively, this occurs because the DOJ is likely to lose in court when it is overly stringent (S + is large). Therefore, if one tries to estimate S as = average of ΔP_{DOJ}  ΔP, one will obtain an estimate of S that is on average too high (S < ) and so is an upper bound. If is negative, one can say that antitrust policy is too lax (S < 0), but cannot reach such definitive statements if > 0 because either a positive or negative S is consistent with a positive . Just as before, it is possible to put a bit more structure on the problem to account for the truncation of ? ( See, Footnote 4), and then estimate S. Finally, there may have been so few litigated cases that estimating S may suffer from small sample estimation problems. Although I have discussed analyzing dataset 1 and 4, I note that there are other subsamples of the data that one might think of separately analyzing. I list a few suggestions below:
VII. Conclusion Without quantitative measures of the effectiveness of merger policy and of the accuracy of the government's analyses underlying merger policy, judgments about the appropriate antitrust policy will be based on qualitative information that can be subject to alternative interpretations. Merger policy can be an important force for either promoting or impairing competition. Merger policy is too important a policy to let it be set in the absence of detailed quantitative studies of its effects on price and other dimensions of competition. The government agencies should embark on such studies immediately and if they lack the authority to either collect the data or study it, they should seek it. Antitrust analysis of individual cases has gotten increasingly sophisticated. Evaluation of antitrust policy has not. There is a need to gather post merger industry data and a need to gather the predictions of DOJ merger analysis in order to evaluate whether U.S. policy and analysis can be improved. Strong opinions are not substitutes for quantitative analysis. Bibliography Antitrust Modernization Commission Report, 2007 Greene, William, Econometric Analysis, 2003 FOOTNOTES * University of Chicago; National Bureau of Economic Research. I currently serve as Deputy Assistant Attorney General, Antitrust Division, U.S. Department of Justice. I thank Thomas Barnett, Ken Heyer, Jeffrey Lien, Hill Wellford, and Gregory Werden for helpful discussions. These views do not necessarily reflect those of the United States Department of Justice. 1. We discuss in the next section how to use data on challenged mergers. For simplicity, I use the DOJ as the government agency responsible for mergers. What I say obviously applies also to the FTC. 2. With fixed cost of litigation, one might want to require a positive ΔP, but this is a detail for the point being made in the text. Indeed, the government can challenge a merger only if it "substantially" lessens competition. I am, for simplicity, assuming that the DOJ is using a consumer (not total) surplus standard. 3. If the government is lax (e.g., S = 5), then it will allow a merger where ΔP =$5. Hence, the boundary in Figure 1 between "challenge" and "allow" moves to the right to ΔP = $5. 4. In the absence of data on DOJ predictions, it might still be possible to estimate S. If one can observe ΔP for each unchallenged merger, then one can draw the distribution of ΔP. Under the assumptions in the text, the largest observed value of ΔP will equal  S. To see this, notice in Figure 1 that the line ΔP = S is the dividing line between the area labeled "challenge" and "allow" when S O. Because ΔP is an expectation not an actual value, the method just described needs to be adjusted slightly. I discuss this adjustment in Section 4. Estimating S as described in the text is likely to produce more accurate estimates of S since it utilizes more data. 5. The addition of a stochastic component, E, means that the procedure to estimate S described in Footnote 4 needs to be modified slightly. The actual distribution of ΔP* for unchallenged mergers is a mixture of the distribution of ΔP (truncated at ΔP = S) and the distribution of E. Under certain assumptions on the distributions, one can estimate the (truncated) distribution of ΔP and then estimate S as max ΔP. 6. If one is willing to define a distribution on , one could estimate S by maximum likelihood while simultaneously accounting for the truncation in . Other estimation techniques also exist. See, e.g., Greene (2003), Ch. 22 for how econometric techniques can be used to handle this problem. 7. The group of unchallenged mergers that have been "fixed" might be an interesting one to study separately. 8. A more complicated model for dataset 1 could analyze the decision of the merging parties to settle (fix the case or abandon it) based on what their estimates of winning in court are. This would provide additional information to estimate S. 