Evaluation Of The United States Department Of Justice

February 14, 2000

TABLE OF CONTENTS

Table of Contents

Index of Full Citations

Introduction and Summary

1. Laying the Foundation For Competition
2. Competition In Local Telecommunications Markets In Texas
3. SBC Has Not Met Its Burden of Demonstrating That It Is Providing Non-Discriminatory Access to DSL
Loops

1. SBC's DSL Performance Data Are Unreliable

1. Data Regarding SBC's Provision of Loop Make-Up Information
2. Data Regarding FOCs for DSL Loop Orders
3. Data Regarding Missed Due Dates for DSL Loops
4. Data Regarding DSL Installation

2. The Performance Reports Submitted by SBC Demonstrate That in Important Respects Its Provision of Loops to Its DSL Customers Is Markedly Inferior to That Provided to Its Retail Operation

1. SBC's Provisioning of DSL Loops
2. SBC's Provisioning of ISDN BRI Loops

3. SBC Has Not Demonstrated That Its Proposed Advanced Services Affiliate Will Eliminate Discrimination

4. SBC's Wholesale Performance In Providing Competitors with Unbundled Local Loops Is Inadequate

1. Hot Cut Provisioning

1. CHC Hot Cut Provisioning

1. Timeliness of Provisioning
2. Service Outages Upon Provisioning CHC Hot Cuts
3. Post-Provisioning Trouble Reports for CHC Hot Cuts

2. Timeliness and Service Outages for FDT Hot Cuts

2. SBC's UNE-Loop Order Processing Performance Has Fallen Below the Standards Set by the Texas PUC as Volumes Have Risen

1. Order Processing Problems Reflected in SBC's Performance Reports
2. Order Processing Problems Adversely Impact CLECs

5. Facilities-Based Entry: SBC Does Not Provide Non-Discriminatory Interconnection Trunking to Its Competitors
6. SBC's Wholesale Performance in Providing Competitors With the UNE-Platform
7. Conclusion

Introduction and Summary

SBC has filed an application for long distance entry in Texas that shows substantial progress in the development of local competition in that state. In sharp contrast to its earlier application for Oklahoma, which failed to meet even the threshold requirements of section 271,⁽¹⁾ this application demonstrates that the ground rules for competition have been largely established, that SBC has implemented most of the operational details of providing access and interconnection to its competitors and that numerous carriers are beginning to compete in offering a wide range of services. This progress reflects well on SBC, which has devoted substantial resources towards implementation of the requirements of the Telecommunications Act,⁽²⁾ and on the Texas Public Utility Commission ("Texas PUC") and its staff, which has worked tirelessly to create an environment in which competition may emerge.

This Evaluation by the Department of Justice principally focuses on SBC's actual commercial performance in providing access and interconnection. In a great many respects, despite some continuing start-up problems, that performance appears to be adequate. In the critical area of providing unbundled loops for advanced services, however, SBC's application is clearly deficient. In this proceeding, the Federal Communications Commission ("Commission") must decide concretely how it will interpret and apply the requirements of section 271 to provide access to such unbundled loops. It is very important for the Commission to ensure that SBC satisfies those requirements, for a failure to do so will seriously retard the deployment of such services and competition in their provision. SBC has not demonstrated that it is providing non-discriminatory treatment to competitors offering xDSL services, or that its planned (but not implemented) use of a separate affiliate to provide such services will address this shortcoming. The Commission should deny SBC's application because of its deficiencies in this area.

There are other shortcomings in this application. In its recent decision approving Bell Atlantic's section 271 application for New York, the Commission found Bell Atlantic's performance in providing "hot cuts" of unbundled loops to be "minimally acceptable." As best we can determine, SBC's performance in this area falls short of that "minimally acceptable" level. Because of SBC's deficient performance, carriers seeking to use unbundled loops are constrained, and the market is not fully open to competition.

Finally, the record leaves considerable doubt about whether SBC can provide interconnection trunks in a timely manner, and whether carriers will be able to compete effectively using the UNE-platform. A careful analysis of additional commercial experience, through the entirety of the current quarter, will provide valuable evidence of whether recently implemented measures have successfully resolved problems in these areas, and whether SBC's wholesale support systems will function adequately as the volume of CLEC activity increases. Because of the limitations of time and information, and because of the critical need to protect the fairness and efficacy of the Commission's process for reviewing section 271 applications, a review of this additional experience should not be attempted in connection with the current application. But since this application should be denied in any event because of the deficiencies relating to DSL and hot cut loops, we recommend that the Commission defer judgment on interconnection and UNE-platform issues until a subsequent re-application, when it will have the benefit of evidence reflecting additional commercial experience.

1. LAYING THE FOUNDATION FOR COMPETITION

The Texas PUC has shown a great commitment to open the Texas market to local competition. Through a consolidated set of arbitrations known as the "mega-arbitration" and the subsequent nearly two-year 271 review process, the Texas PUC and its staff worked carefully and extensively to define the terms, conditions and operational details necessary for the development of competition in the state of Texas. Recognizing that the appropriate offering terms are not a sufficient demonstration of market openness, the Texas PUC also contracted with Telcordia Technologies, Inc. to oversee a carrier-to-carrier readiness test of SBC's operational support systems, and the Texas PUC staff reviewed the reported commercial performance results.⁽³⁾

While providing some evidence of the functionality and capacity of SBC's operational support systems ("OSS"), Telcordia's test has significant limitations.⁽⁴⁾ First, the Telcordia Final Report as written does not clearly identify all major problems encountered during the test, assess the impact of these problems on CLECs, or investigate the root causes of these problems. Second, the carrier-to-carrier style test used in Texas was limited primarily to the system capabilities of the two main test participants, AT&T and MCI WorldCom. Thus, unlike the test in New York, the Telcordia test was not broad enough to test the wholesale support processes for other CLECs with different target markets and business plans.⁽⁵⁾ Third, Telcordia focused narrowly on SBC's computer systems and not on SBC's wholesale support systems generally. Consequently, Telcordia's test does not provide evidence that SBC provides adequate wholesale services overall to CLECs in Texas.

Finally, and perhaps most importantly with respect to future applications, uncertainty remains regarding the validity of some of SBC's performance reports because Telcordia reviewed only a subset of the performance measures on which SBC reports. In evaluating the actual commercial experience of SBC's competitors, the Department and the Commission place great weight on the reported performance data; the reliability of the reported data is critical. To properly validate metrics, one must verify that they are meaningful, accurate and reproducible. Meaningful metrics require clear definitions that will allow measurement of activities or processes in a way that has real-world, practical significance. Accurate metrics are faithful to established definitions in that they are correctly calculated from the proper subset of raw data using processes that ensure the data are accurately handled and transferred. Reproducible metrics can be reproduced at future dates for verification purposes because the raw data have been archived for an appropriate period in a secure, auditable form and because changes to the systems and processes used for gathering and reporting metrics are carefully controlled and fully documented.⁽⁶⁾

Telcordia's review does not provide an adequate basis for determining that presently reported SBC performance data are reliable. First, Telcordia did not examine whether the metrics as defined are meaningful. Second, its review is too dated and limited to ensure the accuracy of current data.⁽⁷⁾ Third, Telcordia's reviews, both in conjunction with the OSS test Final Report and its subsequent Performance Measures Data Control Integrity Analysis, identified concerns relating to the manual handling of performance data, the integrity of data files and the auditability of performance data.⁽⁸⁾ Thus, we find that Telcordia's reviews do not assure the accuracy and reliability of SBC's performance data.

The third-party test that was part of the record in Bell Atlantic's New York application was broad, independent and robust and played a valuable role in opening that market to competition. In comparison, the Telcordia test in Texas was far less comprehensive, blind and independent, and therefore provides much less persuasive evidence.⁽⁹⁾

2. COMPETITION IN LOCAL TELECOMMUNICATIONS MARKETS IN TEXAS

As the Department has previously stated, in-region, interLATA entry by a Bell Operating Company ("BOC") should be permitted only when the local market in a state has been fully and irreversibly opened to competition.⁽¹⁰⁾ In applying this standard, the Department examines whether all three entry paths contemplated by the 1996 Telecommunications Act -- facilities-based entry involving construction of new networks, the use of unbundled elements of the BOC's network and resale of the BOC's services -- are fully and irreversibly open to competitive entry to serve both business and residential customers. To do so, the Department looks first to actual competitive entry. The actual experience of competitors seeking to enter a market can provide highly probative evidence concerning the presence, or absence, of artificial barriers to entry. The degree to which such existing competition is broad-based determines the weight the Department places on it as evidence. For any entry path where competitively significant entry is reasonably foreseeable but broad-based commercial entry is still absent, the Department examines whether new technical and operational arrangements are available and shown to be working to support the entry mode and whether benchmarks to prevent backsliding by the incumbent have been established.⁽¹¹⁾

In its application, SBC contends that CLECs serve 1,408,558 lines, or 12.8 percent of the total number of access lines in SBC's Texas service area.⁽¹²⁾ We presume that SBC's estimates of the number of resale and UNE access lines (both UNE-loop and UNE-platform) are accurate, since SBC, in its role as a wholesaler, should possess reliable information about CLEC activity in these areas. However, SBC appears to have substantially over-estimated the number of lines served by facilities-based carriers, about which it has no direct evidence.⁽¹³⁾ While SBC estimates that CLECs had 920,140 full facilities-based lines in service in November 1999,⁽¹⁴⁾ we believe that a more reasonable estimate is 350,000-400,000.⁽¹⁵⁾ Using our estimate, CLECs have a total of approximately 840,000-890,000 access lines, about eight percent of SBC's Texas market. A CLEC market share of eight percent of local services is significantly above the national average of five percent,⁽¹⁶⁾ but the level of CLEC penetration in Texas appears to be somewhat less than that seen in New York at the time of Bell Atlantic's application, where CLECs collectively had achieved approximately an 8.9 percent market share.⁽¹⁷⁾

A substantial amount of CLEC entry in Texas is primarily by full facilities-based providers serving urban, business customers.⁽¹⁸⁾ CLECs use UNE-loops and the UNE-platform to serve a small, but competitively important number of business customers. Residential customers have historically been served through resale. Many residential customers are now receiving UNE-platform based service. The record in this application does not raise concerns about competitive entry for CLECs choosing to resell SBC's retail services, but the more limited amount of competition using other modes of entry requires us to closely examine the market conditions that affect those other modes of entry.

As we explain below, the Department has concluded that markets for local services in Texas are not fully and irreversibly open to competition by carriers seeking to offer advanced services using unbundled xDSL-capable loops, or by carriers seeking to offer services using unbundled voice-grade loops. The record also raises significant doubts about the ability of facilities-based competitors to obtain adequate access to interconnection trunks, and about the ability of other carriers to compete effectively using the UNE-platform.

3. SBC HAS NOT MET ITS BURDEN OF DEMONSTRATING THAT IT IS PROVIDING NON-DISCRIMINATORY ACCESS TO DSL LOOPS.

In the recent New York Order, the Commission stated that applicants for 271 approval should "make a separate and comprehensive evidentiary showing with respect to the provision of xDSL-capable loops . . . ."⁽¹⁹⁾ The Department emphatically agrees. Such a separate showing is vitally important because DSL technology, using copper wire loops that are ubiquitous throughout the telephone network, offers one of the best prospects for broadband services. In the New York Order, the Commission set forth two methods that an applicant may use to show that it provides DSL loops to CLECs in a nondiscriminatory manner. First, the Commission indicated that the establishment of a "fully operational"⁽²⁰⁾ separate affiliate for advanced services "may provide significant evidence" of nondiscrimination.⁽²¹⁾ Second, the Commission stated that 271 applicants could demonstrate that they are providing nondiscriminatory access to DSL-capable loops through comprehensive and accurate reports of performance measures even without a separate affiliate.⁽²²⁾

SBC states that the record in this proceeding contains sufficient evidence to satisfy the Commission's separate affiliate alternative as well as performance data to prove that it is providing nondiscriminatory treatment of CLEC orders for DSL-capable loops.⁽²⁴⁾ We disagree and find that SBC cannot satisfy either requirement. SBC's performance data are fundamentally flawed in some cases, and in other cases reveal significant discrimination. SBC fails to show that its proposal to offer DSL services through a separate affiliate will be adequate to prevent a continuation of this discrimination.

1. SBC's DSL Performance Data Are Unreliable.

As a critical threshold matter, it appears that key portions of the DSL performance data that SBC has submitted to the Commission are seriously flawed. In some cases, SBC has acknowledged important facts that preclude reliance on these performance data. In other cases, serious questions about the reliability of the data have been raised by commenters. In the context of the known inadequacies in the data, we believe these questions should be resolved before placing any reliance on the challenged data.

1. Data Regarding SBC's Provision of Loop Make-Up Information

Performance Measure ("PM") 57 is a measure of the time within which SBC provides loop make-up (or "qualification") information in response to CLECs' requests for such information. SBC has been on notice at least since the publication of the Commission's order approving the merger of Ameritech and SBC of the appropriate way to measure these response times: "The time starts when a request is received by the CLEC [sic, ILEC] and ends when the information on the loop qualification has been made available to the CLEC."⁽²⁵⁾ Notwithstanding this clear, common sense definition, the data submitted by SBC to the Texas PUC and to the Commission measure only the time SBC's representative worked on the request, and exclude both the period of time the request was with SBC before the representative started working on it and the period of time the response remained with SBC after the representative's work was completed. Excluding these time periods will obviously understate the length of time it actually takes SBC to respond to CLEC requests for loop qualification information.⁽²⁶⁾ Although the Texas PUC stated that SBC had been ordered to fix this problem and that SBC had implemented the change, the Texas PUC apparently relied upon the faulty data when it reported in its comments that PM 57 shows SBC's statewide performance for September through November exceeded the parity requirement.⁽²⁷⁾ SBC has acknowledged to the Department that its PM 57 performance data through December, which were submitted to the Commission on February 1, 2000, also fail to capture these two important time periods.⁽²⁸⁾ Given this flaw in the data, they cannot be relied upon to show parity in this dimension of performance.

2. Data Regarding Firm Order Confirmations ("FOCs") for DSL Loop Orders

In response to the Department's questions concerning Covad's complaint that SBC had excluded most of its orders from Covad-specific reports for PM 5 ("Percent Firm Order Confirmations (FOCs) Received Within "X" Hours") and PM 6 ("Average Time to Return FOC (Hours)"), which measure SBC performance regarding FOC timeliness,⁽²⁹⁾ SBC has acknowledged that it excluded DSL FOCs from its performance measures. SBC asserted that "[t]he business rule for FOC was developed prior to August 1999, and did not contemplate DSL"⁽³⁰⁾ though in fact, the business rule does not provide for any exclusion of DSL loops.⁽³¹⁾

The actual experience reported by the CLECs in their comments suggests that delays in receiving FOCs for DSL orders are lengthy and that such delays impair CLECs' ability to compete.⁽³²⁾ Clearly, the data on FOC timeliness submitted with this application cannot be relied upon to support a conclusion that SBC is supplying DSL competitors with timely order confirmations.

3. Data Regarding Missed Due Dates for DSL Loops

Another incident that casts doubt on the reliability of SBC's performance data relates to data for PM 62 ("Average Delay Days for SWBT Caused Missed Due Dates"), which the Texas PUC has designated as a Tier 1 and 2 performance measure.⁽³³⁾ In reports submitted to this Commission and the Texas PUC as recently as February 1, 2000, SBC reported that the average number of "delay days" was "0.00" for September through December 1999.⁽³⁴⁾ This statistic might have led one to infer that there was little competitive significance in SBC's failure to install a DSL-capable loop at the time promised on the FOC because the average reported delay in installing such loops was insignificant. After SBC was asked to demonstrate how the reported result was mathematically possible, it filed a correction, discussed below, which shows a significant disparity between average delays for CLECs and average delays for SBC's retail operations in November and December.⁽³⁵⁾

4. Data Regarding DSL Installation

The performance measure for the average installation interval for DSL loops (PM 55.1) is one of the most competitively significant indicators of a DSL provider's ability to compete with SBC. SBC's reported data should not be relied upon to show nondiscrimination, absent a complete examination of the accuracy of the reported data.

The data for PM 55.1 that SBC submitted with its application failed to reflect data for a majority of the loops provisioned by SBC. After NorthPoint pointed out that none of its DSL loops had been tracked in PM 55.1, even though it had submitted more than 900 orders for DSL loops,⁽³⁶⁾ SBC acknowledged that a processing error had resulted in a substantial portion of DSL loops being reported under a DSL resale measure.⁽³⁷⁾ SBC submitted corrected data to the Commission on February 1, 2000.⁽³⁸⁾ Unfortunately, SBC has not yet issued corrected reports for individual CLECs, which are necessary for them to reconcile disputed data with SBC. Although the corrected performance data show slightly better performance by SBC, there is still reason to question the reliability of the data reported in PM 55.1, for two reasons.

First, Covad claims that SBC is improperly excluding almost half of its orders from PM 55.1, strongly disputing SBC's contention that it has properly excluded the orders because Covad requested due dates outside the available interval.⁽³⁹⁾ A resolution of this dispute should be relatively straightforward if SBC identifies to Covad (and other CLECs) the particular orders that it claims to have excluded because of out-of-interval due dates. SBC has not done so to date.

Second, the measurement is clouded by disputes over whether SBC has actually installed a working loop at the point in time when it claims to have done so. For example, if the CLEC claims the loop needed additional work, its data would show a longer installation interval than SBC's data. Agreement on the time that loops are installed can best be determined by joint-acceptance testing of loop installations. Rhythms' comments, however, state that until very recently, SBC had refused to engage in acceptance testing for DSL loops, although ordered by the Texas PUC to do so.⁽⁴⁰⁾ It may be necessary to wait until performance data that reflect acceptance testing are available before SBC's report on PM 55.1 can be considered reliable.

Even if the deficiencies in SBC's performance data are limited to the specific examples discussed above, those deficiencies substantially undermine SBC's claim to have provided nondiscriminatory access to DSL loops, since the deficient performance data relate to several competitively important dimensions of SBC's DSL performance. We are concerned, moreover, that those deficiencies may be symptomatic of more serious problems in the reliability of SBC's performance measurement systems and processes. The Department relies heavily on these

performance measures as an objective indicator of the quality of SBC's performance, and we regard it as critical that applicants insure the utmost reliability of such data.

2. The Performance Reports Submitted by SBC Demonstrate That in Important Respects Its Provision of Loops to Its DSL Competitors Is Markedly Inferior to That Provided to Its Retail Operation.

SBC's DSL competitors rely principally on two types of unbundled loops: the DSL loop and the BRI ISDN loop.⁽⁴¹⁾ SBC's application fails to demonstrate acceptable performance in processing orders for DSL and BRI (ISDN) loops because, as described above, its performance data regarding the provision of loop make-up information to CLECs are fundamentally flawed, and because they fail entirely to report on the return of firm order commitments for DSL. With respect to provisioning and maintenance and repair, SBC's performance reports clearly indicate discrimination with respect to both types of loops for carriers offering DSL services.⁽⁴²⁾

1. SBC's Provisioning of DSL Loops

Even with the limited data available, the performance reports demonstrate that SBC

is failing to provide nondiscriminatory performance, as indicated by several performance measures that the Texas PUC has deemed competition affecting.

PM 58. One of the most significant measures for DSL provisioning is PM 58 ("Percent SWBT Caused Missed Due Dates"), a high Tier 1 and 2 measure.⁽⁴³⁾ The measure shows a rate of 12.1 percent missed due dates for CLECs in December (on a total of 495 loops), compared to 6.3 percent missed due dates for SBC's retail service. This result is especially troubling because the number of missed due dates has steadily increased over the last three months as the number of CLEC orders has increased -- although the total number of loops is still far below expected commercial volumes.⁽⁴⁴⁾

PM 60. PM 60 ("Percent Missed Due Dates Due To Lack of Facilities [>30 days]") measures the percentage of loops where SBC missed a committed due date by more than 30 days and attributed the miss to an absence of available facilities.⁽⁴⁵⁾ SBC missed 6.7 percent of due dates for CLECs (on a total of 495 loops) compared to only 0.6 percent for SBC's circuits.⁽⁴⁶⁾

PM 62. PM 62 ("Average Delay Days for SWBT Missed Due Dates") measures the calendar days between the scheduled due date and the actual completion date.⁽⁴⁷⁾ SBC's corrected data for this measure show that SBC missed 60 due dates for December, with an average number of delay days of 6.25 for the CLECs and 4.06 for SBC.⁽⁴⁸⁾

PM 55.1. PM 55.1 ("Average Installation Interval DSL") is a high Tier 1 and 2 measure of the time from completion of loop qualification to the completion of the service order.⁽⁴⁹⁾ For December, the average interval for conditioned loops was 14.23 days for CLECs compared to 11.50 days for SBC. This difference was not statistically significant, but, as noted previously, there are serious questions about the accuracy of the data for PM 55.1 arising from the large number of excluded orders. The results reported for December represented a significant improvement over the results for November which were 19.30 days for CLECs and 11.50 days for SBC.⁽⁵⁰⁾

PM 59. PM 59 ("Percent [Trouble] Reports" ) is a high Tier 1 and 2 measure that tracks the percentage of trouble reports for newly installed DSL loops.⁽⁵¹⁾ SBC tracked 398 DSL loops used by CLECs for December, which had a 15.8 percent rate for trouble reports, compared to a trouble report rate of only 5.2 percent for SBC. This performance deteriorated badly as volumes increased in December.⁽⁵²⁾

PM 65. PM 65 ('Trouble Report Rate [For DSL Loops]") is a high Tier 1 and 2 measure of the monthly repair rate for all installed DSL loops.⁽⁵³⁾ In December 1999, SBC reported 75 trouble reports on 974 CLEC circuits, a 7.7 percent rate compared to a 4.6 percent rate for SBC's retail lines. This measure also shows decreasing performance over time as volumes rise.⁽⁵⁴⁾

2. SBC's Provisioning of ISDN BRI Loops.

SBC's performance reports for BRI loops also demonstrate substantial discrimination and, as in the case of DSL loops, in several cases display a clear trend of declining performance as volumes have increased.

PM 56. PM 56 ("Percent [BRI Loop] Installations within "X" Days") is a high Tier 1 and Tier 2 measure of installation intervals for BRI loops that excludes missed due dates caused by the CLECs or their end-user customers requesting installation intervals longer than the defined standard interval. CLEC orders of one to ten BRI loops are supposed to be installed in three days.⁽⁵⁵⁾ For these BRI loops, SBC's on time performance has dropped from 84.6 percent in October to 71.6 percent in December.⁽⁵⁶⁾

PM 58. PM 58 ("Percent SWBT Caused Missed Due Dates [for BRI Loops]"), a high Tier 1 and 2 measure, reports the percent of installations not completed on the due date.⁽⁵⁷⁾ For BRI loops in December, SBC reports that it was responsible for missed due dates on 23.3 percent of CLEC circuits compared to 15.5 percent for its own customers. SBC's performance on this measure also deteriorated significantly over the past several months.⁽⁵⁸⁾

PM 60. PM 60 ("Percent Missed Due Dates Due To Lack of Facilities") records the percentage of loops for which SBC does not have facilities after giving the CLEC a committed due date.⁽⁵⁹⁾ Out of 374 circuits tracked in December, SBC missed 12 percent of CLEC due dates (45) compared to only 0.8 percent for its own customers. This problem worsened significantly as the number of circuits increased in December.⁽⁶⁰⁾

PM 59. PM 59 ("Percent [Installation Trouble] Reports") is a high Tier 1 and 2 measure that tracks the percentage of trouble reports for newly installed loops.⁽⁶¹⁾ This measure tracked 444 CLEC BRI loops for December which had a 20.9 percent trouble rate (93 trouble reports) compared to a trouble rate of 5.1 percent for SBC's retail lines. This performance metric appears to show deteriorating performance as volumes have increased over the last three months with the reported rate doubling between October and December 1999.⁽⁶²⁾

PM 65. PM 65 ("Trouble Report Rate [For BRI Loops]") is a high Tier 1 and 2 measure of trouble reports for all installed loops received within the month.⁽⁶³⁾ For December, SBC reported a 19.3 percent trouble rate (116 troubles) for CLEC BRI loops, compared to a 2.5 percent rate for SBC BRI loops. Like other measures noted above, this problem worsened as volumes have increased.⁽⁶⁴⁾

PM 67. PM 67 ("Mean Time to Restore (hours) -Dispatch") is a high Tier 1 and 2 measure.⁽⁶⁵⁾ For December the mean time to restore service for CLEC lines was 7.81 hours, compared to 5.84 hours for SBC's retail BRI lines. Performance on this measure also appears to be steadily deteriorating as volumes increase.⁽⁶⁶⁾

Taken as a whole, these performance reports show a service environment in which CLECs attempting to compete against SBC's retail DSL services are seriously disadvantaged at present by SBC's inadequate wholesale performance, and may well face greater disadvantages in the future if SBC's performance continues to decline in the face of higher volumes of CLEC orders.⁽⁶⁷⁾

3. SBC Has Not Demonstrated That Its Proposed Advanced Services Affiliate Will Eliminate Discrimination

The Commission's New York Order stated that a future 271 applicant would have the option of demonstrating nondiscriminatory treatment of DSL competitors by implementing a fully operational, separate affiliate for advanced services.⁽⁶⁸⁾ This application provides the first opportunity for the Commission to determine the specific requirements that will be needed to prevent discrimination against DSL competitors when a BOC chooses this alternative. It is critically important for the Commission to establish two sets of requirements in this context,⁽⁶⁹⁾ without which a separate affiliate structure cannot rationally be said to prevent discrimination.⁽⁷⁰⁾

First, the Commission should rigorously examine the relationship that will exist between the BOC and its affiliate to ensure that the affiliate's relationship to the BOC is the same in all relevant respects as the relationship between CLECs and the BOC. SBC provides virtually no information about these matters, and thus on its face cannot be deemed to provide satisfactory evidence of nondiscrimination.

Second, whatever the ostensible relationship between SBC and its affiliate, there must be adequate mechanisms to detect, punish and deter any discrimination that may occur. An essential component of this mechanism must be meaningful, accurate and reproducible performance measures, as we have previously discussed. Absent the ability to detect discrimination through such measures, SBC could blatantly discriminate in favor of its own retail operations, whether those operations are conducted by a separate affiliate or not. As we have previously explained, SBC's performance data with respect to DSL services have serious flaws, and therefore cannot be relied upon as an effective mechanism to detect discrimination.

Apart from the two elements described above, which the Commission should require in connection with any assessment of a separate affiliate proposal, an applicant that has failed to provide nondiscriminatory treatment prior to the establishment of a separate affiliate should be required to demonstrate that the implementation of the separate affiliate structure has in fact resulted in nondiscriminatory performance. We believe it would be quite unrealistic to expect a BOC and its affiliate to be unmindful of their mutual economic interests, and quite difficult to foreclose all of the avenues by which they might seek illegitimate marketplace advantages over their competitors. A concrete demonstration of nondiscrimination, rather than a abstract promise, will provide much greater assurance that the separate affiliate structure is in fact sufficient to prevent discrimination. Such a demonstration should be a simple matter if the separate affiliate is "fully operational" and if meaningful, reliable and reproducible performance measures are in place.

SBC's affiliate, Advanced Solutions Inc. (ASI), is not yet fully operational. SBC has not demonstrated that it has actually provided nondiscriminatory performance, or even explained why the separate affiliate structure should be expected to remedy the documented current performance problems. We presume that SBC would not implement a separate affiliate that would degrade the quality of performance for its own retail operations. But unless it does so, the current discrimination could be ended only by improving the quality of performance provided to the CLECs. SBC's application offers no explanation of how this would be accomplished merely by transferring its own retail operations to the new affiliate.

In sum, we conclude that SBC has failed to establish that it is providing, or will provide, nondiscriminatory access to unbundled loops for DSL services.

4. SBC'S WHOLESALE PERFORMANCE IN PROVIDING COMPETITORS WITH UNBUNDLED LOCAL LOOPS IS INADEQUATE.

The use of unbundled loops is an important component of CLECs' efforts to provide service to small and medium-sized business customers. SBC's application fails to demonstrate adequate performance in providing unbundled loops. As best we can determine, SBC's performance with regard to "hot cuts" is worse than Bell Atlantic's performance in New York, which the Commission concluded was "minimally acceptable."

SBC uses two hot cut processes in Texas. SBC states that the fully coordinated hot cut ("CHC") process is to be used for conversions of orders of 20 or more lines, or those which must be accomplished outside normal business hours.⁽⁷¹⁾ CHC orders are manually processed in SBC's order processing center and require intensive coordination and communication between SBC and the CLEC during the performance of the hot cut. SBC states that the frame due time ("FDT") hot cut process is to be used for cuts of fewer than 20 lines that are taking place between 8 a.m. and 5 p.m., Monday through Friday.⁽⁷²⁾ FDT orders are capable of flowing through SBC's order processing center without manual work by SBC's representatives. FDT cuts require both the CLEC and SBC to do the necessary work at pre-arranged times, but no communication between them is required at the time of the hot cut.⁽⁷³⁾

SBC has encouraged, if not required, CLECs to switch from CHC to FDT for smaller volume loop cuts.⁽⁷⁴⁾ SBC has expressed the view that CHC is too resource-intensive to support commercial levels of demand for these lower-loop-volume orders and that transition to FDT would alleviate CHC capacity constraints.⁽⁷⁵⁾ SBC may charge a premium if CLECs select the intensively manual CHC process.⁽⁷⁶⁾ In light of this, the use of CHC appears to be rapidly declining, while the use of FDT appears to be rapidly increasing. In November 1999, SBC provisioned 2375 CHC loops and 653 FDT loops.⁽⁷⁷⁾ In December 1999, SBC provisioned 1284 CHC loops and 1666 FDT loops.⁽⁷⁸⁾

SBC's application provides performance data for CHC hot cuts, but not for FDT hot cuts, and the Texas PUC's analysis of SBC's hot cut performance appears to be based solely on CHC performance.⁽⁷⁹⁾ As we explain below, the currently available CHC performance data indicate inadequate provisioning and perhaps other problems. After its initial application, SBC submitted limited data on FDT performance, but those data fail to address the most serious concerns relating to FDT performance, and therefore fail to show that SBC is providing satisfactory performance.

The most apparent problems, both for CHC and FDT, arise in connection with SBC's provisioning of hot cuts. Other potential problems relate to SBC's processing of UNE-loop orders -- specifically its ability to provide timely order confirmations and rejections -- and post-provisioning failures that may lead to double billing and dropped directory listings. These problems appear to be increasing as the volume of UNE-loop orders rises.

1. Hot Cut Provisioning

In its recent New York Order, the Commission found that Bell Atlantic had provided adequate hot cut performance, but emphasized that:

The currently available evidence indicates that SBC's hot cut performance in Texas is not as good as Bell Atlantic's "minimally acceptable" performance in New York. As a result, CLECs are constrained in their ability to enter the Texas market using UNE-loops.⁽⁸¹⁾

1. CHC Hot Cut Provisioning

1. Timeliness of Provisioning

In New York, the Commission concluded that Bell Atlantic completed 90 percent of hot cut orders within the benchmark time, specifically citing Bell Atlantic's completion of orders for fewer than ten loops within one hour.⁽⁸²⁾ The only directly comparable data provided by SBC are (1) "sample" data⁽⁸³⁾ which purport to show that from August through October 1999, 85.6 percent of its CHC hot cut orders were completed within one hour,⁽⁸⁴⁾ and (2) December 1999 data showing that 86.3 percent of all CHC hot cut loops⁽⁸⁵⁾ were completed within one hour, after excluding "misses" for which SBC claimed CLECs were responsible.⁽⁸⁶⁾

2. Service Outages Upon Provisioning CHC Hot Cuts

In New York, the Commission found that fewer than five percent of Bell Atlantic's hot cuts resulted in service outages. In Texas, a joint SBC/AT&T analysis of CHC data for August, September, and October determined that service outages attributable to SBC performance occurred on 8.2 percent of AT&T's CHC orders,⁽⁸⁷⁾ an outage rate nearly twice as high as Bell Atlantic's rate of 4.5 percent in New York.⁽⁸⁸⁾

3. Post-Provisioning Trouble Reports for CHC Hot Cuts

In New York, the Commission found that fewer than two percent of hot cuts generated trouble reports (i.e., service problems reported after a hot cut had been accepted by a CLEC) within seven days. There are no precisely comparable data from Texas, but the data that are available suggest that SBC's performance is no better, and may be slightly worse, than Bell Atlantic's performance in New York: SBC's reported data indicate that 2.18 percent of CHC hot cuts generated trouble reports within ten days.⁽⁸⁹⁾

2. Timeliness and Service Outages for FDT Hot Cuts

As noted above, SBC has provided very little evidence of its performance in providing hot cuts through the FDT process. This failure is cause for concern in light of SBC's policy to strongly encourage the use of the FDT process, and in light of problems with that process that have been documented by commenters.⁽⁹⁰⁾

AT&T's initial trial of the FDT process, in August 1999, resulted in failures on nine of 17 orders; each failure was due to SBC failing to begin the cut while AT&T had ported the number as scheduled.⁽⁹¹⁾ A second trial by AT&T, in November and December, resulted in failures on 20 percent of the orders (7.7 percent of November orders and 33.3 percent of December orders).⁽⁹²⁾ AT&T's FDT data, which have been reconciled with SBC, show (1) that a significant number of AT&T's FDT orders experienced service outages and (2) that 50 percent of these outages were caused by the SBC technician not starting the cut at the scheduled time. Other commenters report similar experiences with the FDT process.⁽⁹³⁾ Given that FDT cuts are only done during business hours, problems such as these are immediately and significantly impacting to the typical, small business end-user customer, who loses phone service during the business day.

SBC has provided only limited information on the duration of FDT hot cuts, and no information on the extent of service outages. After excluding delays in completing hot cuts that it attributed to CLECs, SBC reported that 90.8 percent of FDT hot cut loops were completed within the 30-minute period contemplated under this process, and that 93.8 percent of hot cut loops were completed within one hour.⁽⁹⁴⁾ Service outages may result, however, from a failure to complete the hot cut by the scheduled time (whether due to a late start or an overlong duration), and from SBC's premature initiation of the hot cut. SBC's FDT data do not address these timeliness issues.⁽⁹⁵⁾ SBC also fails to provide any data directly measuring the extent of service outages associated with the FDT process,⁽⁹⁶⁾ despite the apparently well-documented complaints about these problems.⁽⁹⁷⁾

In an effort to improve its hot cut provisioning performance, SBC has instituted new procedures and work groups.⁽⁹⁸⁾ The effects of these changes, however, have not yet been established, and therefore provide no basis for disregarding the inadequacies of historical performance reflected in the record.

2. SBC's UNE-Loop Order Processing Performance Has Fallen Below the Standards Set by the Texas PUC as Volumes Have Risen.

SBC has experienced a disturbing number of problems in processing orders as the volume of orders has increased; there is a significant risk that these problems may become even more acute as UNE-loop orders continue to rise. Similarly, SBC has failed to resolve problems observed in its post-provisioning processing of order information.

Over one-half of the UNE-loop orders which are electronically submitted via the EDI or LEX interfaces are manually processed by SBC's Local Service Center ("LSC").⁽⁹⁹⁾ Manual processing may occur at several points in the order process flow.⁽¹⁰⁰⁾ SBC's systems are designed so that some types of electronically submitted orders, such as CHC hot cuts or Related Purchase Orders ("RPONs"),⁽¹⁰¹⁾ must be manually input into SBC's back-end legacy systems by SBC representatives. In addition, some electronically submitted orders which have errors are either manually rejected by the SBC representatives, or, if the error is not the responsibility of the CLEC, are manually entered into SBC's order processing systems by SBC representatives. The use of manual processing is not in itself an indication of inadequacy in an order processing system, but systems that rely on a high degree of manual processing may experience serious problems with the timeliness and accuracy of order processing and provisioning, as well as scalability of the order processing systems.⁽¹⁰²⁾

1. Order Processing Problems Reflected in SBC's Performance Reports

Order processing problems are reflected in a number of SBC's performance reports:

Timely Return of Order Confirmations. Performance problems seem to be increasing for each of the three categories of UNE-loop orders.⁽¹⁰³⁾ First, for FOCs returned via the EDI interface, SBC argues that in the six-month period from May to October 1999, it satisfied the Texas benchmark for timeliness.⁽¹⁰⁴⁾ But it was in the most recent month of that period, October, that performance fell below the benchmark, and SBC's performance was even worse in November and December, with only 85.8 percent returned within five hours.⁽¹⁰⁵⁾ Second, for FOCs returned via LEX, SBC admits that FOC returns have been chronically late, and states only that it can find no systemic explanation for missing the benchmark.⁽¹⁰⁶⁾ Third, for manually submitted orders, SBC's performance in returning timely confirmations declined from September through November. In December, SBC reported improvement in the percentage of orders returned on time, though the average time for these returns rose precipitously.⁽¹⁰⁷⁾ SBC has attributed these problems to a summer reorganization in its LSC, coupled with a period of increasing CLEC volume, a failure of its LASR GUI and then more retraining of its order processing representatives.⁽¹⁰⁸⁾

Timely Return of Order Rejections. SBC's time to manually process rejects on electronically submitted orders⁽¹⁰⁹⁾ has been significantly increasing. Fewer and fewer of these rejects are being returned within the benchmark interval of five hours,⁽¹¹⁰⁾ while the average time for return has increased from 6.86 hours in July to 35.65 hours in December 1999.⁽¹¹¹⁾ SBC explicitly attributes this continuing decline in performance to the increasing volume of electronically submitted orders, which has increased the number of orders that entail manual reject processing.⁽¹¹²⁾

Post-provisioning Order Processing. After a UNE-loop order has been processed and provisioned, additional processing of the order information through SBC's back-end legacy systems is required to generate an official completion notice (Service Order Completion, or "SOC") to the CLEC and then to change the information in SBC's billing systems so that the end-user is recognized as having moved from SBC to the CLEC. This is known as "posting," and delays may occur both before and after the SOC is generated. Timely and accurate order completion notices are important, as they are the means by which the CLEC knows to begin billing the end-user and addressing any maintenance problems experienced by the end-user.⁽¹¹³⁾ SBC, however, has been returning late SOCs for UNE-loop via LEX from May through December 1999.⁽¹¹⁴⁾ In addition to the late SOCs, UNE-loop CLECs have also experienced late posting of their orders to SBC's billing system, which SBC acknowledges, stating that it has discovered an error in the rate table that caused posting delays in UNE orders.⁽¹¹⁵⁾ SBC asserts that it has a specialized Error Resolution Team ("ERT") "focused solely on clearing errors on orders that have been completed, but for some reason cannot post for proper billing," and is making "concentrated efforts to reduce the clearing time for errors to post to the bill."⁽¹¹⁶⁾ Meanwhile, however, UNE-loop CLECs must deal with the apparent manifestations of these posting delays, including double-billing of end-users and the loss of directory listings.⁽¹¹⁷⁾

2. Order Processing Problems Adversely Impact CLECs.

Order processing problems such as these can adversely affect the ability of CLECs to compete effectively. First, lateness or inaccuracy in order status notices requires CLECs to spend extra time and money attempting to confirm (or revise) schedules with SBC and with their customers.⁽¹¹⁸⁾ Second, problems in processing orders may lead to problems in provisioning, though we lack sufficient information to know whether the order processing problems described here are currently a major factor in the provisioning problems that have been experienced in Texas. If a CHC order is held in error in the LSC, the central office technicians may not get the order in time to perform the pre-cut test procedures that are supposed to minimize the instances of outages upon provisioning. Similarly, if a supplemental order (for instance, a CLEC postponing a due date at the request of the end user) is not promptly processed, SBC may perform an FDT cut prematurely, before the CLEC -- having timely submitted its supplemental order -- has done its necessary work to complete the cutover in service.⁽¹¹⁹⁾

Our concern about those problems arises principally because of their appearance at the relatively low volumes of orders that CLECs have generated to date, and because the processing problems appear to be growing with even a modest trend of increasing order volume. These facts, combined with SBC's attribution of the problem to factors associated with manual processing, indicate a risk that the problems may significantly increase in magnitude as manually handled UNE-loop volumes increase.⁽¹²⁰⁾ SBC's UNE-loop order processing performance thus appears quite different from Bell Atlantic's performance in New York. There, the FCC found (1) timely return of order confirmations and rejects, (2) accurate processing of manually handled orders, and (3) successfully scaling systems so that performance (although below the benchmark levels set by the New York PSC) had improved even as volumes had risen.⁽¹²¹⁾ Here, SBC's performance in the face of increasing volume is deteriorating.⁽¹²²⁾

5. FACILITIES-BASED ENTRY: SBC DOES NOT PROVIDE NON-DISCRIMINATORY INTERCONNECTION TRUNKING TO ITS COMPETITORS.

A CLEC in Texas that wishes to provide local telephone service over its own facilities must interconnect its network with SBC's so that telephone calls can be exchanged between the CLEC's subscribers and SBC's subscribers. The ability to obtain interconnection trunks on a reasonable and timely basis is critically important to facilities-based CLECs as they seek to maintain service quality while their subscriber base, and hence their need for interconnection trunks, increases. The record in this proceeding raises substantial doubt that SBC is providing interconnection trunks in a manner that permits CLECs to compete to their full potential.

Facilities-based CLECs have identified, in Texas PUC proceedings and again in their comments on this application, a variety of serious problems that occurred throughout 1999. These CLECs have indicated that they have been unable to obtain from SBC the number of interconnection trunks they need, in the time frames that they require, despite having followed the required forecasting and network planning processes. They indicate that SBC limited the number of trunks that a CLEC could order; refused to allow CLECs to increase their existing trunk facilities;⁽¹²³⁾ extended by months the time it takes to process CLEC trunk orders;⁽¹²⁴⁾ and missed the installation due dates that were eventually assigned.⁽¹²⁵⁾ These limitations and delays appear to have occurred in all areas of the state -- not only in the technologically sophisticated markets of Houston⁽¹²⁶⁾ and Dallas/Fort Worth,⁽¹²⁷⁾ but also smaller cities such as Amarillo.⁽¹²⁸⁾ The CLECs contend that these problems limited their ability to adequately serve their existing customers and to expand their service to new customers. In some cases, these problems have resulted in call blocking on the CLECs' networks,⁽¹²⁹⁾ customer complaints and lost or forgone sales.⁽¹³⁰⁾

In this proceeding, SBC has submitted performance data from 1999 which appear largely, although not entirely, compliant with the relevant parity and benchmark standards.⁽¹³¹⁾ These performance reports, however, do not refute the complaints of the CLECs. If a CLEC failed to submit an order because of limitations imposed by SBC, that limitation is not reflected in the performance reports, which capture performance only if an order is placed. Similarly, as the Texas PUC learned in November, 1999, orders that were placed but were not processed due to SBC's lack of facilities are excluded from the reported data.⁽¹³²⁾ Because of these omissions, the reported data do not provide a reliable indication of SBC's actual performance.⁽¹³³⁾

In late 1999, the Texas PUC intervened to address the reported problems associated with interconnection trunking. After learning that SBC was excluding "held orders" from its performance reports, the Texas PUC required SBC to implement a new measure specifically to capture the percentage of held orders greater than 90 days. The new measure, PM 73.1, will be finalized at the April 2000 six-month review, and January data under this measure will first be made available in late February 2000. Texas PUC Commissioners stressed that CLECs should maintain their forecasts and place trunk orders -- regardless of any countering SBC suggestion -- so that the complained-of problems would be clearly captured in the new performance data.⁽¹³⁵⁾ SBC agreed to improve its trunk forecasting, data collection, data calculation and ordering processes. SBC has also committed to publish new training materials on February 15, 2000, to provide additional training of its personnel by March 1, 2000, and to mechanize its Trunk Group Service Request process by May 15, 2000 (and retrain its trunk group personnel in the interim).⁽¹³⁶⁾

The Texas Commission has concluded that these changes "should result in parity performance to competitors" that should permit SBC to timely install CLEC trunks.⁽¹³⁷⁾ Unfortunately, though, the trunking problems have been difficult to resolve in the past, and there is no assurance that the most recent efforts will succeed.⁽¹³⁸⁾ While we are hopeful that the admirable efforts of the Texas PUC will resolve the difficulties of the interconnection forecasting, ordering and provisioning process, SBC should be held to a standard of demonstrated success, rather than optimistic prediction.

6. SBC'S WHOLESALE PERFORMANCE IN PROVIDING COMPETITORS WITH THE UNE-PLATFORM

Commenters in this proceeding have complained of a variety of performance problems that may affect the ability of CLECs to compete effectively through the use of the UNE-platform.⁽¹³⁹⁾ These complaints include weaknesses in the documentation that SBC provides for its EDI interfaces; SBC's failure to adhere to its documented change management process; the absence of a stable testing environment for new interfaces; and concerns about the scalability of SBC's systems that might impede competitors' ability to operate at full volumes for mass-market services. It seems clear that SBC's performance in these areas has been flawed in a number of respects, but it is unclear to the Department at this time how significant those shortcomings have been, or more importantly, the extent to which they may seriously impede competition in the future.

There have also been disturbing allegations of service outages, such as the loss of dial tone, an inability to make outbound calls and the loss of features such as hunt groups or preferred long distance or local toll provider, at the time a customer's service is converted to UNE-platform service. Most of the CLECs using the UNE-platform in Texas affirm that some of their customers suffer from the service problems identified above at the time of conversion to the UNE-platform. Two smaller CLECs, Birch and CapRock describe but do not quantify, the problem.⁽¹⁴⁰⁾ AT&T avers that at current order volumes, three percent of its lines are affected by these service problems, while Network Intelligence, a small CLEC avers that 14 percent of its orders are affected.⁽¹⁴¹⁾ Birch, which has been experiencing conversion problems since July 1999, did not begin to see any improvement until it filed a complaint with the Texas PUC in September.

With the involvement of the Texas PUC, SBC committed to manually monitor the problem and began weekly meetings with Birch, which are still ongoing, to deal with this and other issues. The problem, however, continues to persist for Birch.⁽¹⁴²⁾ The magnitude of the conversion problems may have increased recently; SBC's User Forum minutes from December 1999 state that all CLECs have "experienced a notable increase in these outages during the past several weeks."⁽¹⁴³⁾ This time period coincides with rising sales of UNE-platform lines and SBC's decreasing performance on manual rejects of electronically submitted orders.

SBC's process for controlling this problem, as we understand it, is to manually monitor the service orders in its back-end systems to ensure that they complete properly. Hence, there is a risk that if order volumes significantly increase, the scope of these problems may become disproportionately greater. If UNE-platform service providers attract new customers at a rate of one million lines per year, a three percent problem rate (i.e., the rate now reported by AT&T) would generate 30,000 customer problems per year or 2500 problems per month, a rate that may impose heavy demands on SBC's maintenance systems and harm CLECs' reputation in the marketplace.

We regard these service outages as a very serious potential problem, but it is unclear to us at this time the magnitude of the current problem or how likely it is to increase. There is evidence in the record that suggests that these problems are significant and may become more so in the future as order volumes increase, but both that evidence, as well as SBC's response to it, have been inconclusive in our view. Moreover, the record remains unclear as to the precise causes of the outages.

We are also concerned about the extensive amount of manual work required to process current volumes of UNE-platform orders through SBC's order processing center.⁽¹⁴⁴⁾ Current experience in New York shows that at high order volumes, systems problems that increase the manual work in the order processing center make it more difficult to timely confirm or reject manually processed orders.⁽¹⁴⁵⁾

We believe that additional commercial experience which would be outside the scope of this application may provide valuable evidence that will clarify whether the service outages, and the other concerns noted above, will in fact operate as a serious constraint on competition using the UNE-platform, or whether, as SBC contends, these problems are minor or non-existent. Since we believe that this application should be denied for wholly independent reasons, we recommend that the Commission reserve judgment on these issues for a subsequent re-application.

7. CONCLUSION

This application should be denied.