Evaluation Of The United States Department Of Justice

Communications with respect to this document should be addressed to:

Table of Contents

Index of Full Citations

Introduction and Summary

1. State Commission Section 271 Proceedings
2. Entry into the Local Telecommunications Markets
3. Access to BellSouth's Operations Support Systems
4. Reliability of BellSouth's Performance Data
5. Conclusion

The United States Department of Justice ("the Department"), pursuant to Section 271(d)(2)(A) of the Telecommunications Act of 1996⁽¹⁾ ("the 1996 Act"), submits this evaluation of the joint application filed by BellSouth Corporation, BellSouth Telecommunications, Inc., and BellSouth Long Distance, Inc. on February 14, 2002, to provide in-region, interLATA services in Georgia and Louisiana ("Supplemental Application").

BellSouth's Supplemental Application to the Federal Communications Commission ("FCC" or "Commission") is its second for the state of Georgia and its fourth for the state of Louisiana. BellSouth filed its initial joint application for long distance authority in Georgia and Louisiana in October 2001 ("Initial Application"). On November 6, 2001, the Department submitted to the FCC its evaluation of the Initial Application ("Georgia/Louisiana I Evaluation"), which focused primarily on the adequacy of BellSouth's operational support systems ("OSS") and the reliability of BellSouth's performance data.⁽²⁾ Based on concerns raised in these areas, the Department concluded that it could not fully support BellSouth's Initial Application.⁽³⁾ BellSouth subsequently withdrew its Initial Application from the Commission on December 20, 2001.

As the Department has explained, in-region, interLATA entry by a regionalBell Operating Company ("BOC") should be permitted only when the local markets in a state have been "fully and irreversibly" opened to competition.⁽⁴⁾ This standard seeks to measure whether the barriers to competition that Congress sought to eliminate with the 1996 Act have in fact been fully eliminated and whether there are objective criteria to ensure that competitive local exchange carriers ("CLECs") will continue to have nondiscriminatory access to the facilities and services they will need from the BOC in order to enter and compete in the local exchange market. In applying its standard, the Department considers whether all three entry paths contemplated by the 1996 Act -- facilities-based entry involving the construction of new networks, the use of the unbundled elements of the BOC's network ("UNEs"), and resale of the BOC's services -- are fully and irreversibly open to competitive entry to serve both business and residential customers.

This Evaluation focuses on changes in BellSouth's OSS and progress in the auditing of BellSouth's performance metrics since the filing of the Department's Georgia/Louisiana I Evaluation. The Department recognizes that additional improvements in BellSouth's OSS have been identified and will be implemented under the direction of the Georgia and Louisiana Public Service Commissions ("PSCs"), and that final completion of the metrics audit under the auspices of the Georgia PSC should further improve the accuracy and reliability of BellSouth's performance reports. Based on this record, and subject to the Commission's review of the concerns expressed in this Evaluation, the Department recommends that the FCC approve BellSouth's application.

1. State Commission Section 271 Proceedings

Both the Georgia and Louisiana PSCs have filed comments supporting BellSouth's Supplemental Application. During the past several months, the Georgia PSC has continued to work actively to create competitive conditions in the state, in part by overseeing workshops and conferences pertaining to performance measurement and enforcement, and by conducting an evaluation of the change management process.⁽⁵⁾ Likewise, the Louisiana PSChas continued to address BellSouth's service quality performance measures and associated self-executing penalties, and conduct technical and collaborative workshops on operational issues.⁽⁶⁾

BellSouth asserts that its OSS are regional and, therefore, as the Department has previously explained, it is appropriate to take into consideration the OSS testing conducted by KPMG at the direction of the Florida PSC.⁽⁷⁾ This testing has continued since the Department's Georgia/Louisiana I Evaluation was filed and, although incomplete, it has generated some results that are relevant to this application.

2. Entry into the Local Telecommunications Markets

In assessing whether the local markets in a state are fully and irreversibly open to competition, the Department looks first to the actual entry in a market.⁽⁸⁾ But the Department does not broadly presume that all three entry tracks -- facilities-based, unbundled network elements ("UNEs"), and resale -- are open or closed on the basis of an aggregate level of entry alone.⁽⁹⁾

Based on current data on CLEC entry in Georgia and Louisiana, the Department finds no basis to address issues beyond those discussed in its Georgia/Louisiana I Evaluation. The Department has not changed its positive assessment of the openness of markets in Georgia or Louisiana for the facilities-based and resale modes of entry.

1. Georgia

In its Georgia/Louisiana I Evaluation, the Department extended the presumption of openness to the fully-facilities-based mode of competition in Georgia based on the entry recorded there.⁽¹⁰⁾ Although the Department did not extend such a presumption to other modes of entry,⁽¹¹⁾ the lack of complaints from CLECs reselling BellSouth's voice services in Georgia led to a conclusion that the resale mode of entry was open to competitors.⁽¹²⁾

BellSouth's Supplemental Application reports that CLECs in Georgia serve approximately 913,000 lines, or approximately 18.8 percent of the total lines in BellSouth's service area, using all modes of entry, as of December 2001.⁽¹³⁾ As to the use of UNEs, BellSouth's Supplemental Application includes evidence that employment of the UNE-platform has increased recently. According to BellSouth data, the share of all lines in BellSouth's service area served by CLECs via the UNE-platform has increased from 3.9 percent to 4.7 percent since its Initial Application.⁽¹⁴⁾ CLECs' use of the UNE-platform to serve residential customers has grown from 110,927 lines, or 3.8 percent of residential lines in BellSouth's service area, to 136,306 lines, or 4.8 percent, and to serve business customers from 79,146 lines, or 4 percent of business lines in BellSouth's service area, to 91,405 lines, or 4.6 percent.⁽¹⁵⁾ This growth may reflect, in part, the improvements in BellSouth's OSS since the filing of its Initial Application, as described below.

2. Louisiana

In its Georgia/Louisiana I Evaluation, the Department found that the amount of entry in Louisiana did not entitle BellSouth to a presumption of openness as to any mode of entry.⁽¹⁶⁾ However,based on the absence of complaints from CLECs providing service entirely over their own facilities, the Department concluded that the Louisiana market was open to fully facilities-based competition.⁽¹⁷⁾ The Department reached the same conclusion with respect to resale entry.⁽¹⁸⁾ However, with respect to entry by means of UNEs, the Department indicated that BellSouth had yet to demonstrate adequate performance in serving CLECs in Louisiana.⁽¹⁹⁾

BellSouth estimates that competitors in Louisiana serve approximately 227,000 lines, or approximately 8.9 percent of the total lines in BellSouth's service area, using all modes of entry, as of December 2001.⁽²⁰⁾ Facilities-based competition for residential customers has been strengthened by the entry of Cox Louisiana Telecom, a cable telephony provider.⁽²¹⁾ Furthermore, use of the UNE-platform has increased recently in Louisiana. According to BellSouth data, the share of all lines in BellSouth's service area served by CLECs via the UNE-platform has increased from 0.7 percent to 1.2 percent since its Initial Application.⁽²²⁾ CLECs' use of the UNE-platform to serve residential customers has grown from 3,119 lines, or 0.2 percent of residential lines in BellSouth's service area, to 5,145 lines, or 0.3 percent, and to serve business customers from 15,745 lines, or 1.8 percent of business lines in BellSouth's service area, to24,995 lines, or 2.9 percent.⁽²³⁾ Although the growth in entry via the UNE-platform in Louisiana may reflect, in part, improvements in BellSouth's OSS since the filing of its Initial Application, as described below, entry via the UNE-platform in Louisiana is still minimal.

3. Access to BellSouth's Operations Support Systems

Since the filing of the Department's Initial Evaluation a number of developments have occurred that bear upon the concerns raised in that Evaluation regarding BellSouth's OSS.

1. Changes to Facilitate Automated Handling of CLEC Orders

The Department's Georgia/Louisiana I Evaluation stressed the importance, for competitors using UNEs, of efficient OSS that would facilitate the automated handling of orders by the CLECs and BellSouth. Since that Evaluation was filed, several important developments have occurred that should facilitate the use of UNEs. As described more extensively below, BellSouth has (1) implemented telephone number migration ("TN migration") for UNE-platform orders, (2) altered its pre-order systems to offer a more fully parsed customer service record, (3) changed its OSS so as to reduce rejects based on a double database edit to CLEC orders, and (4) improved the electronic ordering capabilities for digital subscriber line ("DSL") competitors. Although the Department is encouraged by these developments, it notes that in implementing these changes to its OSS BellSouth appears to have violated important change management principles and that this has resulted in exceptions being opened by KPMG in the Florida OSS test.⁽²⁴⁾

The Department's Georgia/Louisiana I Evaluation noted that BellSouth did not offer CLECs the option of ordering service based on a subscriber's telephone number (as opposed to a subscriber's complete address), a capability known as TN migration.⁽²⁵⁾ This feature is provided by other RBOCs.⁽²⁶⁾ TN migration reduces the possibility of conflicts between the addresses in an RBOC's database and the addresses provided in CLEC orders. In response to filings by CLECs, the Georgia PSC ordered BellSouth to provide TN migration by November 3, 2001, in its order recommending approval of BellSouth's Section 271 application.⁽²⁷⁾ The Georgia PSC recognized that not having to enter the customer's address "will facilitate mass-market competitive entry by reducing address-related errors."⁽²⁸⁾ The introduction of TN migration represents an important step in upgrading BellSouth's OSS.

The procedures by which BellSouth implemented this change and other software upgrades, as discussed below, however, raise questions about BellSouth's consistency in adhering to proper change management principles.⁽²⁹⁾ Because of inadequate provision for CLEC testing, problems that could cause a substantial increase in the number of rejects were discovered the day before the initial release was implemented.⁽³⁰⁾ A single day's notice that orders should be submitted differently than BellSouth had previously indicated is not nearly sufficient to permit CLECs to modify their ordering systems. BellSouth, however, did promptly act to correct the problems and since mid-November 2001 CLECs have been able to make full use of TN migration.⁽³¹⁾

InDecember and January there were more than 325,000 UNE-platform requests submitted region-wide by numerous CLECs.⁽³²⁾ As a result of the TN migration upgrade "the overall reject rate for UNE-P migration requests dropped over 35% from October 2001 to January 2002 [and] address related errors for these same requests have been reduced by over 60% during this same time period."⁽³³⁾ WorldCom, the leading proponent of TN migration, acknowledges that its reject rate decreased by approximately 10 percentage points.⁽³⁴⁾

In addition to implementing TN migration, BellSouth made a software change on February 2, 2002, whereby it ceased performing address checks on two separate, and sometimes inconsistent, address databases,⁽³⁵⁾ also known as a "double edit."This change should further reduce BellSouth's rejects of CLEC orders.

In response to the Georgia PSC's order requiring BellSouth to provide "fully fielded parsed CSRs by January 5, 2002," BellSouth implemented the pre-order functionality by which customer service records are parsed on its side of the interface.⁽³⁶⁾ The Department believes that this improvement may facilitate competitive entry by lowering the operating costs of the new entrant. It should also further reduce the costs associated with mistakes that are made when CLEC service representatives are required to retype pre-order information on orders to be sent to BellSouth.

However, this improvement, like that pertaining to TN migration, was introduced without adequate testing and with defects.⁽³⁷⁾ Although BellSouth and the Georgia PSC dispute the significance of the defects in this software,⁽³⁸⁾ KPMG regards them as significant and strongly criticized BellSouth for not completing important internal testing before implementing the release.⁽³⁹⁾ As discussed more fully below, competitive harm may occur where BellSouth does not provide for smooth transitions between software releases in accordance with appropriate change control principles.

The Department's Georgia/Louisiana I Evaluation expressed concern that the lack of electronic ordering capability for certain order types could impair the ability of DSL providers to compete with BellSouth.⁽⁴⁰⁾ In response to a Georgia PSC order, BellSouth deployed electronic ordering for line-splitting on January 5, 2002.⁽⁴¹⁾ BellSouth made electronic ordering of UDC/IDSL loops, another popular product for DSL providers, available on February 2, 2002.⁽⁴²⁾ After receipt by BellSouth, UDC/IDSL loop orders will continue to be processed manually until May 19, 2002, when BellSouth is scheduled to implement electronic processing, which will allow the orders to flow through to provisioning without manual intervention.⁽⁴³⁾ In the interim, DSL competitors will be able to use electronic systems for the generation of their orders instead of having to fax them to BellSouth.⁽⁴⁴⁾

Although the introduction of TN migration, the cessation of double database edits for customer addresses, the implementation of parsed CSRs, and the improvements in electronic ordering capabilities for DSL competitors are positive developments that should permit new entrants to operate more efficiently, the changes are so recent that their full effect is not yet reflected in the performance reports. However, BellSouth's CLEC order reject rate has significantly improved since the Department's Georgia/Louisiana I Evaluation was filed, likely due to the introduction of TN migration in November. In particular, the reject rate for UNE-platform orders has fallen from 19.38 percent in September to 14.33 percent in December 2001.⁽⁴⁵⁾

2. Efforts to Improve Service Order Accuracy

The Department's Georgia/Louisiana I Evaluation noted that BellSouth's performance reports indicated that its service order accuracy was substantially below the established benchmarks and that this increased the burdens on the CLECs, thereby constraining their ability to compete effectively.⁽⁴⁶⁾ Since that Evaluation was filed, BellSouth has implemented an intensive program to improve service order accuracy, and the data for December 2001 show it met 95 percent of the performance benchmarks for all 7 of the UNE sub-metrics and 8 of the 11 resale sub-metrics.⁽⁴⁷⁾ BellSouth has also agreed to include the service order accuracy measure in its Self-Executing Enforcement Mechanism ("performance enforcement plan").⁽⁴⁸⁾

The Georgia PSC Comments support BellSouth's claim of improved service order accuracy, reporting that, through November 2001, BellSouth had missed the benchmark for 2 of the 3 previous months in only 4 of the 24 service order accuracy sub-metrics, and asserting that BellSouth's December 2001 performance was equally strong.⁽⁴⁹⁾ Birch also supports the BellSouth claim of improved service order accuracy, but suggests that these improvements have resulted from extraordinary manual efforts that may not continue once BellSouth obtains Section 271 approval.⁽⁵⁰⁾

BellSouth's ability to sustain its improved performance deserves close examination. The Georgia PSC is considering requiring BellSouth to develop an electronic process to review all partially mechanized CLEC orders so as to obtain a more reliable means of measuring service order accuracy.⁽⁵¹⁾ It also proposes including the revised measure in BellSouth's performance enforcement plan.⁽⁵²⁾ The Department supports this effort. BellSouth has agreed to make penalty payments under the existing measure until the service order accuracy measure is revised and incorporated in the plan.⁽⁵³⁾

The significance of the improvements in BellSouth's service order accuracy is unfortunately obscured, however, by significant changes in how BellSouth has measured accuracy starting with November data.⁽⁵⁴⁾ These changes, which were made without prior approval of the Georgia PSC or notice to the CLECs,⁽⁵⁵⁾raise two questions. The first is how to evaluate BellSouth's claim that service order accuracy has improved given such differences in the metrics.⁽⁵⁶⁾ In response to the Department's request, BellSouth has provided data showing performance over time using a consistent methodology in order to address whether improvement in service order accuracy actually has occurred.⁽⁵⁷⁾ The second question is how the FCC and state commissions can be expected to determine that BellSouth has continued to meet its obligations pursuant to Section 271 if BellSouth can unilaterally change metrics without notice to or input from interested parties. Such a practice substantially diminishes the value of an independent, third-party audit of the performance system. Future changes should be made only with public notice and the concurrence of the state commissions.

3. Improved Availability of Testing Environments

The Department's Georgia/Louisiana I Evaluation stressed the importance of a robust test environment so that CLECs may be able to ensure that their software interfaces correctly with BellSouth's OSS, particularly as changes are implemented.⁽⁵⁸⁾ The Department raised concerns regarding the degree of separation between BellSouth's test and production environments because certain CLECs claimed that the commingling of test and production orders had impeded their ability to test new software.⁽⁵⁹⁾ In addition, the limited availability of the system for testing software changes ("CAVE") was also a factor in constraining CLECs' testing.⁽⁶⁰⁾

The Georgia PSC believes that concerns about the adequacy of the testing environment have been adequately addressed by BellSouth.⁽⁶¹⁾ The Georgia PSC concludes that the CAVE test environment is sufficiently separate from the production environment even though they share the same service order processor, and states that a number of CLECs other than WorldCom have successfully used CAVE without having their test orders and production orders mixed up.⁽⁶²⁾ The Georgia PSC also reports that the concerns expressed in the Department's Georgia/Louisiana I Evaluation regarding the inability of CLECs to test EDI orders for DSL loops or to test the LENS interface have been resolved.⁽⁶³⁾

In the period since BellSouth's Initial Application, CLECs have used the CAVE system on numerous occasions and, according to BellSouth, have made no further claims that test or production orders have been misdirected.⁽⁶⁴⁾ Unfortunately, CAVE's limited availability continued to create problems since it could not be used for testing the introduction of TN migration,⁽⁶⁵⁾ although it was available and was used by CLECs for testing the introduction of "parsed CSRs."⁽⁶⁶⁾ The scheduled availability of the CAVE system, however, has been substantially improved for the balance of this year and that should facilitate its effective use by the CLECs.⁽⁶⁷⁾

4. Developments Regarding the Change Management Process

There are two aspects to the change management process. The first relates to the procedures governing the introduction of new software, including notice of changes to be made by BellSouth in the OSS interface, exchange of specifications necessary for recoding, and adequate opportunity for both internal and CLEC testing of the proposed release. The second component of change management is the process by which proposed changes necessary to improve the incumbent's OSS are selected for development.

The Department notes that there are outstanding KPMG exceptions covering both of these areas in the Florida OSS test.⁽⁶⁸⁾ The Department's Georgia/Louisiana I Evaluation expressed concern that the change management process did not prompt efficient fixes for known defects or system enhancements desired by CLECs.⁽⁶⁹⁾ As discussed above, BellSouth's recent introductions of new OSS features providing for TN migration and parsed CSRs were marked by its failure to provide required information to CLECs regarding the effects of the upcoming changes and, apparently, by its failure to adequately perform internal testing.⁽⁷⁰⁾

BellSouth and the Georgia PSC report a number of positive developments that have occurred with regard to the change management process.⁽⁷¹⁾ The Georgia PSC is conducting a comprehensive, two-phase examination of the change control process ("CCP").⁽⁷²⁾ The first phase involves considering additional measures to monitor BellSouth's compliance with CCP procedures.⁽⁷³⁾ In this process, three measures were proposed by the industry and agreed to by the Georgia PSC staff: (1) a measure of the timeliness of correction of BellSouth software defects; (2) the percentage of change requests accepted or rejected within 10 business days; and (3) the percentage of change requests rejected for each reason allowed by the CCP.⁽⁷⁴⁾ The Georgia PSC asserts that these measures will help assure that BellSouth corrects defects and handles change requests promptly and efficiently.⁽⁷⁵⁾

The second phase of the examination covers changes to the CCP itself, and the Georgia PSC is now reviewing CLECs' proposals and BellSouth's response.⁽⁷⁶⁾ BellSouth is supporting a number of the modifications proposed by the CLECs and has made its own proposals to address CLEC concerns regarding the scope of the CCP, the time required to implement changes, and the adequacy of the prioritization process.⁽⁷⁷⁾ The Georgia PSC concludes that BellSouth is making a good faith effort to improve the processes and correct the deficiencies that the Department identified in its Georgia/Louisiana I Evaluation, and notes that BellSouth has begun distributing a complete schedule for release implementation this year, as well as a report outlining the status of the "'top 15' change requests prioritized by CLECs" it is committed to implementing in 2002.⁽⁷⁸⁾

BellSouth also proposed to the Georgia PSC a process by which 40 percent of its annual release capacity would be allocated for implementing CLEC change requests and/or CLEC-driven regulatory mandates.⁽⁷⁹⁾ The CLECs are advocating an approach that would require implementation of all their requested changes within a certain interval.⁽⁸⁰⁾ Resolving this issue will be difficult and may require the Georgia PSC to balance important, conflicting interests.

The Department is encouraged by the positive steps taken in the area of change management and expects BellSouth to comply with whatever approach is chosen. The record in this application supports approval, but in subsequent applications the Department will continue to review closely BellSouth's OSS and change management performance.

4. Reliability of BellSouth's Performance Data

While commending the efforts of the state commissions to develop robust sets of performance measures, the Department's Georgia/Louisiana I Evaluation expressed concern regarding the reliability of BellSouth's performance data.⁽⁸¹⁾ The Department discussed a number of problems affecting the accuracy of BellSouth's data, and noted that frequent restatements of data made it difficult to conclude that BellSouth's performance metrics were accurate.⁽⁸²⁾

BellSouth now maintains that the stability of its performance reporting systems has improved greatly as demonstrated by the fact that it restated no performance reports between September and December 2001.⁽⁸³⁾ The Department is encouraged that performance reports are being restated less frequently and agrees that this could indicate that the current systems are becoming more stable.⁽⁸⁴⁾ In and of itself, however, this reduced number of restatements is not proof that the underlying problems that led to the former pattern of restatements have been resolved or that the current data are accurate.⁽⁸⁵⁾

Also, BellSouth relies heavily on KPMG's Revised Interim Status Report ("KPMG Status Report") on the Georgia metrics audit as support for the accuracy of its data.⁽⁸⁶⁾ The KPMG Report describes those parts of the audit that are complete, the status of open issues, and the items that remain to be tested. The KPMG Status Report is useful in that it provides a more complete picture of the progress that has been made toward completing the Georgia metrics audit, but it offers no conclusions regarding the reliability of BellSouth's metrics because it was not issued for that purpose. Based on its own analysis of the open issues identified in the KPMG Status Report, BellSouth maintains that they are minor enough to suggest that the items yet to be audited will "close" successfully and should not be a cause for concern.⁽⁸⁷⁾ As in its Initial Application, BellSouth again argues that the three phases of the audit together provide ample grounds for concluding that its metrics are accurate.⁽⁸⁸⁾

Although the number of tests KPMG reports as having been completed or deemed "satisfied" is not insignificant,⁽⁸⁹⁾ the audit is not complete.⁽⁹⁰⁾ The limited number of outstanding issues from past phases of the Georgia audit does suggest that inaccuracies discovered in the audit are being addressed and resolved,⁽⁹¹⁾ but the number of open exceptions and unsatisfied test criteria in past phases do not provide a basis for predicting that other significant issues will not be discovered during the third phase. Moreover, BellSouth has made many systems changes since the first two phases of the audit which affect the majority of its metrics.⁽⁹²⁾

On balance, the progress on the audit, coupled with the Georgia PSC's recent judgment that the data are reliable in light of its close monitoring of the audit,⁽⁹³⁾ suggests that the stability and accuracy of BellSouth's performance data are improving. However, until the Georgia metrics audit is complete or until there is additional commercial experience with the reported metrics, the Commission should not rely solely on BellSouth's performance reports in reviewing otherwise credible complaints that BellSouth is not meeting the requirements of the Act.

5. Conclusion

BellSouth's Supplemental Application demonstrates that it has made substantial progress in addressing issues previously identified by the Department. The Department recognizes that additional improvements in BellSouth's OSS have been identified and will be implemented under the direction of the Georgia and Louisiana PSCs, and that final completion of the metrics audit under the auspices of the Georgia PSC should further improve the accuracy and reliability of BellSouth's performance reports. Based on this record, and subject to the Commission's review of the concerns expressed in this Evaluation, the Department recommends that the FCC approve BellSouth's application.

March 21, 2002