Evaluation Of The United States Department Of Justice

Table of Contents

1. Verizon's Demonstration of Its Ability to Provide Access to DSL Loops Merits Careful Examination
1. Principles That Guide the Department's Evaluation
2. Verizon's Provisioning of DSL-Capable Loops
1. Access to DSL-Capable Loops Generally
2. Access to Line Sharing for DSL
2. Conclusion

Verizon first filed an application for long distance authority in Massachusetts in September 2000. The Department of Justice ("Department") Evaluation of that application focused principally on Verizon's provision of unbundled digital subscriber line ("DSL") loops to its competitors, noting that Verizon's performance reports indicated a lack of parity with respect to certain aspects of its performance in providing access to DSL loops, and that based on the record at that time, the Department was not able to assess the validity of Verizon's objections to the prescribed performance measures or of its suggested alternative methods of measuring performance. The Department also concluded that Verizon had not yet demonstrated that suitable performance measures with unambiguous benchmarks were in place to deterbacksliding.1 Verizon withdrew its initial Massachusetts application on December 18, 2000, and filed its present application on January 16, 2001.

A number of changes have taken place since the Department filed its Evaluation of that initial Massachusetts application. Verizon and the competitive local exchange carriers ("CLECs") agreed to modify the carrier-to-carrier ("C2C") performance measures for DSL loop performance and to create a set of C2C performance measures for line sharing; the Massachusetts Department of Telecommunications and Energy ("MA DTE") adopted these changes to the C2C performance measures;2 Verizon submitted to the MA DTE changes to its performance assurance plan, proposing to add additional DSL-capable loop and line-sharing measurements and to make DSL a separate mode of entry;3 Verizon's separate data affiliate, Verizon Advanced Data, Inc. ("VADI"), is now fully operational in Massachusetts;4 Verizon has agreed to proceed with the development and deployment of a mechanism to provide CLECs with electronic access to loop make-up information;5 and Verizon conducted reinspections of line-sharing-related collocation work,6 modified its order processing software to allow some line-sharing orders to flow through to theprovisioning stage without manual intervention,7 and established a wholesale service center dedicated to DSL loops and line sharing.8

This Evaluation will briefly highlight some of the key disputed issues relating to access to DSL loops.9 The current application shows improvement in some aspects of Verizon's performance in providing access to DSL loops. There are still, however, factual disputes between Verizon and various commenters relating to the validity of the performance data and the quality of Verizon's performance. In some areas, most notably with respect to line-shared DSL loops, there is very little evidence reflecting the results of commercial experience with Verizon's provisioning process.

As explained below, Verizon's current application relies on a substantial number of statistical analyses of its performance. These performance reports measure Verizon's performance in a variety of ways. Each performance report is subject to important qualifications and/or exclusions of data -- in many cases the accuracy or validity of these reports is challenged by commenters -- and many of them indicate that Verizon has not provided nondiscriminatory access to DSL loops. While these additional submissions by Verizon make the current application stronger in some respects than the first Massachusetts application, the record still fails to provide a clear demonstration of nondiscriminatory performance. Accordingly, theDepartment remains unable to conclude on the current record that Verizon has adequately demonstrated its ability to provide nondiscriminatory access to DSL loops.

I. Verizon's Demonstration of Its Ability to Provide Access to DSL Loops Merits Careful Examination

1. Principles that Guide the Department's Evaluation

   In evaluating whether local telecommunications markets are "fully and irreversibly open to competition," the Department has placed considerable importance on an applicant's demonstration that it has achieved an acceptable level of performance, i.e., that it has established a benchmark to which it can be held thereafter, based on objective data reflecting actual commercial experience.10 The rapid and effective implementation of complex new access arrangements, in an environment in which incumbents lack commercial incentives to implement such arrangements, will depend to a significant degree on regulatory policies. Experience has shown that regulatory agencies are handicapped in deciding how best to resolve the many technical disputes that arise in this context, and in deciding how rapidly incumbents reasonably may be required to implement new access arrangements, by the absence of readily available and objective information on these issues.11

   The Telecommunications Act of 199612 mitigates these difficulties by conditioning long-distance entry on a Bell operating company's ("BOC") effective implementation of marketopening arrangements, thereby strengthening the BOC's incentives to implement new access arrangements rapidly and effectively. By measuring a BOC's implementation of access arrangements against objective performance benchmarks, regulatory agencies can to some extent reduce their involvement in complex disputes between incumbents and entrants concerning the technical means by which access should be provided, and focus instead on more tractable questions about the level of quality of wholesale performance that should be deemed acceptable and whether that level of quality has been achieved. Moreover, once it has been demonstrated through objective measures that a particular level of performance is feasible, it will be easier to impose contractual or regulatory sanctions if the performance is subsequently degraded. Thus, achieved performance benchmarks in a commercial environment serve not only as a demonstration that the market has been opened, but also as a tool to ensure that it will remain open.

   In emphasizing these principles, the Department recognizes that careful judgments are required for several different types of questions. First, there are typically many dimensions on which to measure wholesale performance and room for reasonable disagreement about which dimensions have the greatest competitive significance.13 Second, the level of performance that should be deemed acceptable or unacceptable on any single dimension of performance cannot be defined with scientific precision. As the Federal Communications Commission ("Commission") has recognized, a requirement of "parity" is not a requirement of absolute equality ofperformance,14 and any attempt to prescribe a required level of performance, like any line-drawing exercise, will be unavoidably arbitrary in distinguishing between performance just barely above the prescribed level and performance just barely below that level. Third, some judgment is required in assessing the amount of data and the duration of performance at an "acceptable" level that are needed to establish a suitable performance benchmark. For example, if performance data indicate little variability from month to month, one might conclude that a suitable benchmark had been achieved on the basis of a short period of acceptable performance. In another context, if performance data show substantial variability from month to month or a trend of declining performance, it would be prudent to require a longer period of acceptable performance. Finally, as the Commission has recognized, there should be some "play in the joints" of the section 271 process so that evolving obligations under section 251 do not operate, in practice, to keep approval of a section 271 application perpetually out of reach of the BOCs.15

   With these important considerations in mind, the Department urges the Commission, in this and other section 271 proceedings, to pay particular attention to the importance of demonstrated achievement of adequate benchmarks of wholesale performance, measured by objective performance data. In particular, participants in the section 271 process (including applicants, state commissions and commenters) should work towards developing a record that will show whether such benchmarks have been achieved. Components of such a showing would include three types of evidence: (1) whether reliable systems for gathering, processing, reporting and ensuring the integrity of performance data have been implemented; (2) whetherthose systems measure the competitively significant aspects of a BOC's wholesale performance; and (3) whether an "acceptable" level of performance has been defined and can be demonstrated through that performance measurement process. Attention to these principles will provide important competitive benefits, both in opening the markets for local telecommunications services and in facilitating proof that will permit additional long distance competition by the BOCs.

2. Verizon's Provisioning of DSL-Capable Loops
   1. Access to DSL-Capable Loops Generally

      Within the framework of these general principles, the Department turns to some of the specific questions raised by this particular application. In its current application, Verizon reports multiple versions of certain data to measure the adequacy of Verizon's performance in providing stand-alone DSL loops and line-sharing loops to its competitors. First, Verizon provides its official C2C DSL performance reports for September, October, and November 2000 using the performance measures that were prescribed by the MA DTE at that time.16 Similar performance data for December 2000 also has been added to the record.17 Second, Verizon recalculated some of these performance reports by excluding certain data from its September and October reports because it claims those data reflected performance problems caused by a strike of Verizon employees.18 Third, Verizon recalculated its performance under certain measures excluding data concerning performance problems that Verizon attributed to the failure of some CLECs tofollow prescribed policies and procedures.19 Some CLECs contest the appropriateness of these exclusions.20 Fourth, in December 2000, the MA DTE adopted revisions to some of the C2C performance measures and a separate set of C2C performance measures for Verizon's provision of access to line-sharing DSL loops.21 Verizon recalculated performance data for the September through November time period, retroactively applying the revised performance measures as well as the new line-sharing measures to its performance in that time period.22

      Review of these data reveals that while Verizon's performance has improved since the time of its first application for Massachusetts, a number of questions and concerns remain as to whether Verizon has adequately demonstrated its ability to provide nondiscriminatory access to DSL loops. For certain important measures, Verizon's reported performance still falls below prescribed standards. In addition, some of Verizon's restatements of these measures do not provide a reliable alternative means of determining that Verizon is providing its competitors with adequate access to DSL loops, and significant questions remain as to other restatements.

      The Department notes the following particular concerns:

      • Percent DSL orders completed within six days (PR-3-10). The C2C reports show significantly fewer lines completed within six days for CLECs than for Verizon or itsseparate data affiliate, VADI,23 although the percentage of orders completed within six days for both CLECs and VADI is increasing each month.24 Verizon restated its performance using the revised performance measure, under which Verizon may exclude three types of orders: orders that require manual loop qualification, orders for which the requested provisioning interval is greater than six days, and orders for which Verizon did not have an available loop.25 In addition, Verizon excluded strike-affected data.26 CLECs question the accuracy of some of the exclusions and the assumptions under which Verizon excluded data.27 Even with these exclusions, which collectively remove from the performance report more than 83 percent of CLEC orders, Verizon's performance, while better, still falls substantially short of completing 95 percent of CLEC orders within six days, the revised prescribed standard.28
      • Percent DSL orders completed on time (PR-4-14 to -4-18). These C2C performance measurements show that Verizon's on-time performance is improving, but remains below the 95 percent standard.29 Verizon's removal of strike-affected orders from thesemeasures for September and October 2000 improves Verizon's reported performance somewhat.30 Verizon's final data presentation of the revised on-time measure, which excludes orders for which Verizon cannot provide a loop and adjusts for strike-affected orders, shows on-time performance that exceeds the 95 percent standard only for November 2000.31 One CLEC questions the reliability of these data, alleging that some of the loops counted by Verizon as completed were never properly provisioned.32 In addition, Verizon's average installation interval performance, a companion measure, is inconsistent, ranging from better than retail to 41.7 percent worse than retail.33
      • Percent installation troubles on DSL lines within 30 days (PR-6-01). The C2C reports show a substantial lack of parity.34 The reported rate of installation troubles is lower in the revised performance measure, under which Verizon excludes troubles reported by CLECs that do not do joint acceptance testing of the loop at the time of installation, but the revised data still show a greater rate of installation troubles for CLECs than for Verizon retail or VADI.35 Moreover, the revised measure appears to be flawed. While trouble reports from carriers that do not conduct acceptance tests are excluded from the numerator of this measure, orders from such carriers are not excluded from the denominator.36 The result is to inappropriately lower the trouble report rate. When these orders are excluded from the denominator, the reported trouble rate is higher for October and November 2000 under the revised measure than as reported under the original C2C measure.37 Contending that CLECs have inappropriately accepted nonworking loops, Verizon also modified both the C2C and revised measure results by excluding reportedtroubles for such loops from its data.38 These modifications result in still lower reported rates of installation troubles, although none of these four sets of data shows parity performance. In addition, CLECs question whether Verizon may appropriately exclude some of these trouble tickets, a factual dispute that remains unresolved.39
      • Percentage missed DSL repair appointments (MR-3-01). The C2C data show that from September through November 2000, Verizon met 85 percent of DSL repair appointments for CLECs compared to approximately 86 percent for Verizon retail DSL customers.40 These aggregate data, however, blur the fact that Verizon's performance has been inconsistent. Verizon performed worse for CLECs than for itself in October and November 2000 and then better for CLECs than for itself in December 2000.41
      • Mean time to repair (MR-4-01 & MR-4-02). The C2C reports show that the mean time to repair CLEC lines has decreased significantly over the past few months, but there still was a significant disparity between Verizon's performance for itself or VADI and its performance for CLECs from September through November 2000. Although Verizon's performance for CLECs improved substantially in December 2000, narrowing the gap, Verizon has not established a consistent record of improved performance.42 Verizon's alternative analysis excludes from the data instances in which Verizon contends that CLECs have refused weekend repair appointments. This alternative analysis reduces the disparity in performance between CLECs and Verizon's retail business.43 Unlike some of the other disputed measures, however, the definition of this C2C measure has not been revised to account for refused appointments. Therefore, the C2C reports for this measure will likely continue to be disputed in the future, and thus may have limited value as a benchmark of performance. The Department also notes that excluding only declinedCLEC appointments, and not those declined by BOC customers, may create an analysis that is biased toward finding parity.44
      • Percentage out of service more than 24 hours (MR-4-08). This C2C performance measurement demonstrates a lack of parity, although the percentages are falling for both Verizon and the CLECs, and the gap between the two is shrinking.45
   2. Access to Line Sharing for DSL

      In this application, Verizon has provided DSL line-sharing data that is distinct from its performance reports concerning DSL stand-alone loops. The Commission required incumbent local exchange carriers ("ILECs") to implement line sharing by June 6, 2000.46 Implementation of line sharing required central office modifications, including the installation of splitters. During the spring of 2000, in anticipation of the June 6^th deadline, Verizon and the CLECs operating in the Verizon region, including Massachusetts, developed a plan to expedite the necessary implementation work.47 As Verizon completed this work, CLECs began submitting a few line-sharing orders.

      It subsequently became clear that the central office work had not been properly performed and that CLEC orders were not being properly completed.48 To remedy these problems, Verizon conducted reinspections of all line-sharing-related central office work beginning December 1, 2000.49 Verizon expected to finish these reinspections by the end of January 2001, and to finish fixing the problems identified at the inspections thereafter.50 The number of completed CLEC shared lines in service in Massachusetts so far is small, but growing.51

      Verizon argues that it fully complies with the line-sharing requirements established by the Commission.52 In support of this argument, Verizon reports on its line-sharing performance for September through November 2000 using recently adopted, consensus performance measures.53 The value of these performance reports, however, is substantially undermined by Verizon's statement that certain reports "may overstate" its performance because Verizon technicians marked some orders complete, even though splitter installation problems prevented line sharing over those loops.54 Since Verizon's application does not quantify the extent of thisproblem, it is impossible to reach any conclusions about the quality of Verizon's performance with respect to line-sharing loops based on these data.

      Further, some CLECs state that once they detected problems in a central office, they stopped submitting orders for that office.55 Verizon's reinspection process, begun in December 2000, appears to validate CLEC claims that Verizon was unable to process line-sharing orders in certain central offices during September, October and November 2000.56 Because Verizon's performance measuring system is not designed to measure problems that limit the ability of CLECs to submit orders, performance reports covering such time periods may not constitute reliable evidence of adequate performance.

      Verizon is making efforts to resolve its line-sharing implementation difficulties. As noted above, Verizon has installed new software that permits it to verify that the loop is indeed working for data;57 modified its collocation inspection procedure and reinspected CLEC DSL collocations;58 automated order processing for some line-sharing orders;59 and dedicated a wholesale service center to DSL orders.60