On behalf of the
U.S. Department of Justice
May 20, 1997
I have also attended meetings both with SWBT and several CLECs interconnecting with or negotiating to interconnect with SWBT, or reviewed notes of such meetings.
Additionally, I have reviewed performance measures proposed by other BOCs, such as the attached Ameritech proposal, in various proceedings in other states.
Finally, in reviewing SWBT's proposals, I have drawn upon my significant experience with quality performance measures. As a telephone company line manager and officer, my performance was judged in part by how well I met customer service objectives. Further, as a staff manager, I had responsibility for the development and implementation of quality performance measures.
- My name is Michael J. Friduss. My business address is 1555 Museum Drive, Highland Park, IL 60035. I am an independent consultant working with CA Hempfling & Associates under contract with the Antitrust Division of the Department of Justice.
- I received a Bachelor of Science degree in industrial engineering from the Illinois Institute of Technology in 1964, and a Masters degree in Management from Northwestern University in 1971.
- I began my telecommunications career in 1964 as a Management Assistant for Illinois Bell Telephone Company ("Illinois Bell"). In this capacity, I filled a variety of non management and management positions designed to familiarize me with all departments of the company.
- From 1966 to 1969, I was a Manager in Illinois Bell's Plant Department. In this capacity, I supervised installation or repair operations in three different territories on the South side of Chicago.
- In 1969, I was promoted to District Engineering Manager, responsible for the engineering and design of outside plant, also on Chicago's South side. In 1970, I was appointed District Manager-Outside Plant Engineering Staff for Chicago Operations, responsible for methods and procedures and approval of major
- outside plant capital expenditures. In 1971, I was appointed District Plant Manager, responsible for installation and repair activities in Chicago's Hyde Park area. During my tenure in Hyde Park, I also headed up an Operations Review team that assessed the quality and cost performance of each district in Chicago Operations.
- I was promoted to Division Manager-Corporate Planning at AT&T in New York in 1973 and served through 1975. In this capacity, I headed a small group responsible for the study of the telecommunications interexchange industry at that time and what AT&T's future strategy should be in that segment of the industry.
- In 1975, I returned to Illinois Bell as Division Plant Manager, responsible for installation and repair in the South suburban area. In 1978, I was named Division Manager-Corporate Planning for the company, responsible for Illinois Bell's planning and operations budgeting, including operations planning for the implementation of the FCC's Computer Inquiry II and divestiture.
- In 1983, I was promoted to General Manager-Distribution Services, responsible for Illinois Bell's outside operations, construction, and engineering. In this capacity, I supervised 7,000 employees and a budget of $500 million.
- In 1986, I was promoted to Vice President-Personnel and Support Services for Michigan Bell and in 1989 was named Vice President-Customer Sales and Service for the same company. In
- the latter role, I was chief operating officer of the company and a member of the Board of Directors, with responsibility for operations and sales, including 11,000 employees and expenditures in excess of $1 billion.
- In 1992, I returned to Ameritech Services as Vice President-Customer Service and Information Technology, responsible for the strategic and tactical direction of Ameritech's customer service and operations, as well as planning, building, and maintaining high quality and efficient computer
- In late 1993, I formed MJ Friduss & Associates, consultants to the telecommunications industry. Our clients are carriers, primarily current and new local service providers, and small to medium sized companies that provide hardware, software, and operating systems to those service providers. We are currently working with a number of firms in the areas of strategic planning, marketing, operations, customer services, and supplier management.
- Additionally, I am Editor of the Friduss Report, a newsletter focused on carrier procurement processes.
SCOPE OF ASSIGNMENT
- I have been asked by the Antitrust Division of the U.S. Department of Justice for my opinion regarding the appropriateness and comprehensiveness of the performance measures Southwestern Bell (SWBT) proposes to provide to competitors and regulators. In particular, I have been asked whether these performance measures will reasonably depict the performance of wholesale functions SWBT is obligated to perform pursuant to the competitive checklist of section 271 of the 1996 Act, and whether such measures will enable competitors and regulators to determine both the adequacy of SWBT's performance and the parity of such performance when compared to SWBT's retail operations.
- The primary source upon which I relied for my analysis is SWBT's Section 271 application for Oklahoma. I generally reviewed the application for any discussion of performance measures. Additionally, I have reviewed:
- Oklahoma Corporation Commission's Operating and Maintenance Requirements pertaining to Southwestern Bell.
- The FCC's Quality of Service report, which summarizes quality of service based on data submitted by the BOCs, GTE and Sprint.
- SWBT's Statement of Generally Available Terms and Conditions ("SGAT") before the Corporation Commission of the State of Oklahoma to provide interLATA telephone service in Oklahoma.
- Testimony before the Corporation Commission of the State of Oklahoma related to Southwestern Bell's application for full interLATA competition in Oklahoma.
- The Telecommunications Act of 1996.
- Interconnection Agreements between SWBT and Competitive Local Exchange Carriers ("CLECs") in Oklahoma.
- SWBT's Interconnection Agreement with AT&T in Texas.
- Performance Measures proposed by other BOCs as well as proposals by several CLECs.
- Letters from SWBT to the Department of Justice regarding the performance measures on which SWBT proposes to report.
- Comments SWBT filed with the FCC related to section 272 of the 1996 Act.
PERFORMANCE MEASURES AND THEIR ROLE
The 1996 Act obligates incumbent local exchange carriers (ILEC's), and thus Bell Operating Companies (BOCs), to provide requesting carriers with, among other things, interconnection,
access to unbundled network elements, and resale services. In fulfilling these obligations BOCs will perform a variety of functions for competitors, many of which BOCs also perform in providing retail services. Some of these functions, however, will be new.
The ability to detect discrimination in the performance of these functions is dependent on the establishment of performance measures, allowing competitors and regulators to measure the BOC's performance. The development of appropriate measures is critical to establishing that the local market is open. On an ongoing basis, the measures must be able to assure that the local market remains open and that any BOC backsliding will be detected.
Performance measures serve as criteria for indicating the performance of such wholesale functions. Performance measures enable competitors and regulators to compare a BOC's performance of a function with that provided a BOC's retail customer, or make an assessment of such function in the abstract. For example, to measure how well a BOC performs the function of provisioning resold local service, we can define a performance measure -- "the percentage of orders not completed within three days" -- and use it to describe the BOC's performance and to compare it to the BOC's retail performance of the same function. In general, performance measures are used to determine quality, measuring how long an activity takes to complete -- cycle time -- and how well the activity is performed -- reliability.
A performance measure may take the form of an objective or target, such as the example cycle-time measure "three days to complete an order" above, where the result is a percentage of orders meeting or not meeting the target. A performance measure can also be a raw time interval, such as the average number of days to complete resale orders. In neither case, however, does the outcome of the measure -- the percentage or cycle time -- itself indicate "good" performance or "bad" performance. Thus, performance measures themselves are not the barometers of performance, but rather the yardsticks with which to measure such performance. Accordingly, my review is limited to the adequacy of SWBT's performance measures, rather than the adequacy of its performance.
The highest priority performance measures should be those that describe the end-to-end quality of service -- cycle time and reliability -- from the customer's viewpoint. Studies over the years have identified performance measures that correlate highly with the customer's perceptions of service quality, such as the percentage of repeat reports of trouble, while others have a lower correlation.
While performance measures are generally easy to identify, there is no universally accepted definition of what the measure proposes to reveal nor specifically how to gather the necessary data that comprises the measure. For example, cycle-time performance measures are dependent on the specific definition of start and stop times, while reliability measures are dependent on
the specific definition of what constitutes a failure. This affidavit does not attempt to specify these definitions. However, it is critical that SWBT and interconnecting CLECs do so. To further ensure the usefulness of the results, I have assumed that all parties will commit to reporting results that reflect the spirit of the performance measure as well as its paper definition. For example, in measuring the level of missed appointments, the result should be measured against the original due date; due date changes could only be considered when explicitly requested by the end user.
My review of SWBT's proposed performance measures includes an assessment of:
A. BOC PERFORMANCE MEASURES TO DATE
Over the past 120 years, telephone companies have developed extensive measures of customer service. These performance measures have generally served two purposes: first, to allow for the comparison of performance between managers, territories, organizations, and companies; and second, to provide regulators
with indicators of potential problems. These measures cover all areas of customer-affecting performance, including customer care, provisioning, repair, billing, and network maintenance. Regulatory requirements notwithstanding, these performance measures comprise a key indicator of management success. Objectives are set, data is gathered, reports are published, and results become part of the corporate, organizational, and individual success determination.
Using performance measures, most state public utility commissions require achievement of certain levels of performance for customer service. For example, the Oklahoma Corporation Commission requires the following:
- The scope of the functions measured.
- Whether the proposed measures will allow CLECs and regulators to compare BOC wholesale and retail performance of the functions measured.
- The value or applicability of the measures in terms of disaggregating appropriate functions, markets, and products.
- The stability of the measures.
- The scaleability of the measures.
The FCC requires the BOCs, GTE, and Sprint to submit quality of service data that is summarized annually in a report entitled "Quality of Service for the Local Operating Companies Aggregated to the Holding Company Level." Without specifying particular levels, the report includes the following performance measures:
- 95% of all installations provided within 4 working days.
- No held service orders over 30 days.
- 90% of subrate (56kb) circuits within 90 days of the service order.
- 80% of toll calls answered within 10 seconds.
- 75% of directory assistance calls answered within 10 seconds.
- 75% of repair calls answered within 20 seconds.
- 95% dialtone within 3 seconds.
- 90% call completion without trunk busy condition within a central office.
- 95% call completion without trunk busy condition on local interoffice trunks.
- 97% call completion without trunk busy on intralata toll trunks.
- No more than 30 decibels above reference level on interoffice calls.
- No more than 33 decibels above reference level on access facilities.
- No more than 10 decibels on access lines measured to the network interface.
- 90% out of service repairs completed next working day.
- Repair report rate (not including CPE or inside wiring):
- 12 per 100 access lines in exchanges of less than 300 access lines.
- 9 per 100 access lines in exchanges between 301 and 2000 access lines.
- 7 per 100 access lines in exchanges over 2000 access lines.
- Percent of installation appointments met
- Average missed installations in days
- Average repair interval
- Initial trouble reports per 1000 access lines
- Troubles found per 1000 access lines
- Repeat trouble as a percent of initial trouble reports
- Complaints per million access lines
- Switches with downtime
- Average switch downtime in seconds per switch
- Unscheduled downtime over 2 minutes per occurrence
- Scheduled downtime over 2 minutes per occurrence
- Trunk groups with blocking as a percent of total trunk groups
B. PARITY VERSUS ADEQUACY PERFORMANCE MEASURES TO DATE
Given the dual retail and wholesale roles BOCs must now play under the 1996 Act, there are two approaches to measuring the performance of a particular function: parity performance measurements and adequacy performance measurements. When a BOC's performance of certain functions for its retail units or "end user" customers is identical or analogous to the performance of those functions for competitors or their customers, parity performance measures apply. Parity performance measures merely juxtapose performance results, such as trouble reports per month per customer placed by the BOC's customers compared with those of a competitor's customers. Thus parity performance measures are used for "apples to apples" comparisons, and are most often applied in the resale environment, where the functions a BOC performs for a competitor's customers are almost identical to those performed for its own retail customers.
In contrast, adequacy performance measures establish an objective or target pertaining to functions a BOC either (1) performs only for competitors, or (2) performs for competitors in
a manner sufficiently different from that performed for the BOC itself such that a comparison is meaningless or unhelpful. Thus adequacy performance measures apply in "apples-to-oranges" comparisons. An example might be the average time to provision an unbundled loop.
C. BOC WHOLESALE FUNCTIONS
It is helpful to divide the functions BOCs will perform for CLECs under the 1996 Act into five primary categories: preordering, ordering, provisioning, maintenance and repair, and billing functions. These categories describe the functions through which CLECs acquire new customers and subsequently maintain facilities for them and bill them. Within each category, performance measures identify the cycle-time and reliability of each function. Performance parity is achieved if CLEC customers enjoy cycle time and reliability of functions equivalent to that experienced by the BOC's customers or its affiliates' customers.
Pre-ordering describes the up-front process of a CLEC or BOC customer service representative obtaining information to place an order for new, additional, or changed service. Pre-order cycle time performance measures generally refer to operations support system (OSS) response times that allow the representative to complete the service order with the customer on the line (e.g. customer address verification or appointment scheduling). Pre-order reliability performance measures refer to the accuracy and completeness of the data received. These pre-ordering functions are generally visible to the end user.
Ordering describes the process of the service representative transmitting the service order into the BOC's OSSs for facility assignment, data base updates (including 911, directory listing, and repair), switch updates, and dispatch of a technician if required. For a CLEC, this includes successfully moving the service order across an agreed-upon interface into the BOC's OSSs. Ordering cycle time performance measures refer to BOC response times for notices of order confirmation, jeopardy, or rejection. Ordering reliability performance measures refer to the accuracy and completeness of these notices. Ordering is generally transparent to end users.
Ordering performance measures also relate to the measurement of service order "flow-through." Flow-through measures the percentage of service orders that flow from the service representative to completion if no technician dispatch is required or to the point of dispatch if dispatch is required.
OSS availability and BOC service center answer time performance measures can also be considered to be part of the ordering process.
Provisioning involves the execution of a request for a set of products and services or unbundled network elements with attendant acknowledgments and status reports. Provisioning performance measures measure how quickly and how well customer service orders are completed. Provisioning results are highly visible to end users and are critical to the determination of performance parity. Provisioning cycle time performance measures refer to measuring the interval, from the end user's perspective, from order placement to order completion. Example measures include average POTS completion interval and percent missed appointments. Provisioning reliability performance measures refer to the accuracy of the work (i.e., did the end user receive what they ordered) and to the quality of the work done (i.e., did everything work). An example measure is the percentage of new service failures within an agreed upon time.
For purposes of this review, I have evaluated categories of repair and maintenance separately. Repair is the process by which end users report a case of trouble and the trouble is subsequently cared for. This process is highly visible to the end user and has a high correlation with the end user's perception of the service provider. Repair cycle time performance measures measure the interval from end user report to trouble clearance and notification. Examples include mean time to repair and percent missed appointments. Repair reliability performance measures measure the quality of the repair operation. An example is the percentage of trouble recurring within an agreed upon time.
Maintenance refers to how well the network itself is maintained and associated performance measures generally refer to reliability rather than cycle time. The most visible performance measure is the mean time between troubles, often referred to as the trouble report rate. Other performance measures measure how well the BOC's switching and transmission elements are maintained. Examples include percent dial tone delay, percent switches with unscheduled downtime, and transmission signal to noise ratio.
Billing performance measures measure the speed, accuracy, and completeness of end user usage data from the BOC to the CLEC. While the process may be transparent to the end user, the end product is highly visible. Examples of performance measures include the percentage of billing records delivered on time and the percentage of accurate and complete bills. 50. There are several miscellaneous functions that must also be measured. For example, toll and directory assistance operator services and directory listing must be included as performance parity categories. Typical performance measures include operator speed of answer and directory listing accuracy.
D. MARKET AND PRODUCT DISAGGREGATION OF PARITY PERFORMANCE MEASURES
As discussed above, meaningful determinations of parity performance require "apples-to-apples" comparisons of the functions performed by a BOC. Where, however, the same function is performed, for example, by different personnel, with different facilities, for different customer classes, or for different products, more refined comparisons are required. Thus, for example, the function of installing POTS service for consumer and business customers may be identical, but because business customers may be more sensitive to installation delays, a meaningful comparison may require juxtiposition of only business customer installation intervals.
There are two general categories of such further disaggregation. First, market parity refers to equality between appropriate customer groups. Customer groups may be broken out geographically or by class of service. Geographic market parity means comparing CLEC results to BOC results within the geography the CLEC has chosen to offer service. For example, if a CLEC offers resale service only in city A, a meaningful comparison may require the BOC to provide their retail results only for city A.
Class of service market parity means comparing CLEC results to BOC results within the classes of service the CLEC has chosen to offer. For example, if a CLEC offers service to small business end users only, for purposes of comparison a BOC would have to provide its retail results for such small business users.
A second category of disaggregation is product parity. Product parity can be divided into wholesale and retail product types. The first breakout is by the type of wholesale product: resale services, unbundled network elements, or facilities-based interconnection. Resale performance measures are generally parity measurements, while unbundled element and facilities-based interconnection performance measures are generally adequacy measurements. The second breakout is by the retail product or service offered to the end user: POTS, Hicap, Subrate, ISDN, Centrex, etc. For example, if a CLEC chooses to offer ISDN, the BOC would provide performance measurements for their own ISDN retail product.
E. REPORTING REQUIREMENTS
Once appropriate performance measures have been agreed to and the data gathered, the results must be formatted into reports and provided to CLECs and regulators. My review will include proposed report formats, report frequency, the appropriateness of result comparisons, report accuracy and completeness, and the availability of raw data.
Report format relates to how performance measure results are presented. Are they presented in table or graph form? Are they readable and understandable? Can a CLEC or regulator determine whether parity has been achieved? Report frequency relates to how often reports will be provided. Report accuracy and completeness relate to the statistical validity of the proposed data. Appropriateness of results comparisons relates to the entities for which the data will be provided. BOC retail? BOC subsidiaries? The CLEC? All CLECs? Other?
IV. REVIEW OF SWBT'S PROPOSED PERFORMANCE MEASURES
This section of the affidavit turns to the performance measures explicitly cited in SWBT's application, performance measures implied by existing interconnection agreements or comments on Section 272 service requirements, and performance measures not explicitly or implicitly included that are important to measuring functions required under the 1996 Act.
A. PERFORMANCE MEASURES REFERENCED IN THE APPLICATION
SWBT's application for provision of in-region, interLATA service in Oklahoma commits to equal quality of interconnection to new entrants. Section II.B.1. of SWBT's Brief in support of the application states, "To ensure equal quality, interconnection with CLECs will be accomplished using the same facilities, interfaces, technical criteria, and service standards as SWBT uses for its own internal operations." SWBT Brief at 19, citing Deere Aff. ï½¶25. Further, with regard to resale, SWBT commits to making services available for resale that are "equal in quality, subject to the same conditions, and provided with the same provisioning time intervals as the services SWBT provides to other customers, including end users." Id. at 40, citing Kaeshoffer ï½¶64.
SWBT's application states that their experience providing exchange access services to the long distance industry, together with "established, objective performance measures and monitoring mechanisms, make a reversal to lower quality service utterly implausible." Deere Aff. ï½¶160; Kahn Aff. ï½¶45. The application goes on to identify currently filed and available regulatory reports that relate to service quality, customer satisfaction, and infrastructure investment. The application cites trunk blockage, total switch downtime, consumer satisfaction, and installation and repair intervals as examples of performance measures currently available on reports filed with the FCC. As
discussed above, these measures, if properly measured and reported, can be important parity determinants. FCC ARMIS reporting by itself, however, is not sufficient to judge performance parity.
Obviously, SWBT would need to provide separate data for retail versus wholesale performance to make a comparison. The ARMIS data filed with the FCC does not provide such a breakout. Nor does the ARMIS data cover many of the new functions BOCs will have to perform for CLECs under the 1996 Act.
SWBT's application also refers to sixteen negotiated interconnection agreements in Oklahoma, with six approved by the Oklahoma Corporation Commission. Zamora Aff. ï½¶24. SWBT primarily relies on these agreements as providing all the performance measures necessary to gauge its performance of wholesale functions.
Most interconnection agreements entered into under the 1996 Act, including SWBT's agreements with CLECs, include no or few references to specific performance measures. Based on discussions with numerous CLECs, a primary reason for this appears to be the weakness of CLEC negotiating positions and a higher priority placed on entering the market versus delaying negotiations or enduring arbitrations to establish long-range safeguards such as performance measures. The CLECs reason that once in the market, they'll attempt to renegotiate the subject of performance measures, or merely rely on those established by larger carriers such as AT&T. As a result, interconnection agreements in general, and as discussed below SWBT's in particular, provide insufficient performance measures necessary to allow for a Section 271 determination of nondiscriminatory performance of wholesale functions.
Finally, I reviewed SWBT's Statement of Generally Available Terms and Conditions ("SGAT") filed with the Oklahoma Corporation Commission on January 15, 1997. The SGAT commits to providing new entrants with network elements, resale services, and access to OSS functions. For network elements, it also commits to performance "at least equal in quality and performance as that which SWBT provides to itself." SGAT Appendix UNE 2.14.1. Further, it provides for liquidated damages if cycle time objectives in the installation and repair of unbundled loops and the installation of interim number portability are missed. These liquidated damage provisions also appear in SWBT's executed interconnection agreements. However, these few performance measures are inadequate in both number and scope to monitor and guard against whether SWBT will have the ability to discriminate against new entrants. B. PERFORMANCE MEASURES INCLUDED IN INTERCONNECTION AGREEMENTS
SWBT has sixteen interconnection agreements in the State of Oklahoma. These agreements commit to several explicit performance measures that are also identified in SWBT SGAT. Most commit to specific objectives for:
- Truck blockage - an excellent measure of network engineering and maintenance
- Total switch downtime - a fine reliability measure, but not a telling determinate of parity.
- Customer satisfaction - the ultimate measure of end user customer service, but would need to differentiate between SWBT and the CLEC to be meaningful as a parity determinate.
- Installation interval - an excellent measure of provisioning cycle time and therefore an integral performance parity determinant. Agreement needs to be reached on stop and start time and appropriate reporting requirements for BOC - CLEC comparisons.
- Repair interval - an excellent measure of maintenance cycle time and integral to performance parity determination. As above, agreement needs to be reached on start and stop times and appropriate reporting requirements.
In its interconnection agreement with AT&T in Texas, SWBT commits to providing the following resale "Performance Metrics":
- UNE loop provisioning intervals- This can be an excellent measure of product specific provisioning cycle time and determinant of performance adequacy. However, start and stop times need to be defined for this measure to be an effective gauge of performance.
- Interim Number Portability provisioning intervals- This can be an excellent measure of provisioning cycle time and determinant of performance adequacy. However, start and stop times need to be defined for this measure to be an effective gauge of performance.
- UNE loop out-of-service repair intervals- Excellent measure of product specific maintenance cycle time and determinant of performance adequacy.
In the AT&T agreement, referring to the Performance of (Unbundled) Network Elements, SWBT commits to meeting "applicable performance measures and be at least equal in quality and performance as that which SWBT provides to itself."Texas interconnection appendix UNE 2.16.1 Oklahoma SGAT appendix 2.14.1. At AT&T's request, SWBT "will:
- Expedited order status notification- A good measure of ordering performance adequacy.
- Percent missed due dates- An excellent measure, when tied to provisioning interval, of provisioning cycle time performance. It is critical that this measure is determined with respect to the original due date, rather than a new' due date set in response to work force or other delays. Data needs to be gathered and reported by product and market. Reporting required by many state regulatory bodies and the FCC. This performance measure has also been proposed by Ameritech and PacTel.
- Percent right the first time- An excellent measure of provisioning reliability performance, if properly gathered and reported in an appropriate format. This has also been proposed by Ameritech in their Michigan SGAT and by PacTel.
- Percent no access- Not a reliable measure of performance parity because achievement levels are generally determined by customer behavior rather than company performance.
- Service Center response time- An excellent potential measure of ordering cycle time performance. However, response time needs to be specifically defined as "time to speak to the SWBT representative."
- "Repair service that is at least equal in quality to that provided to SWBT customers; trouble calls from AT&T will receive response time and priorities that are at least equal to that of SWBT customers." Attachment Maintenance - Resale 4.1. This commitment, while commendable, needs to be matched by specific cycle time and reliability measurements.
SWBT's Interconnections Agreement with Sprint in Oklahoma is the most comprehensive in terms of explicit discussion of and commitment to performance measures. In Attachment UNE 2.17.7, SWBT and Sprint "will jointly define data consistent with that provided by SWBT to other LSP's, that is to be provided monthly to Sprint to measure whether Unbundled Network Elements are
provided at least equal in quality and performance to that which SWBT provided to itself and other LSP's." In addition to this joint commitment, SWBT clearly commits to equal quality of service to Sprint in both Resale and Unbundled Network Element modes. Explicit performance measures committed to include: Resale: "For all resale service ordered under this agreement, SWBT will provide preorder, order, and provisioning service equal in quality and speed (speed to be measured from time SWBT receives the order from Sprint) to the services SWBT provides its end users." (Attachment Ordering and Provisioning-Resale 1.2) SWBT further commits to equal response times and priorities on trouble reports as well as equal service from SWBT technicians.
- maintain data that compares the installation intervals and maintenance/service response times experienced by AT&T's customers to those experienced by SWBT customers and the customers of other LSP's; and
- provide the comparative data to AT&T on a regular basis."Attachment UNE 2.16.7. These are fine commitments, but need to be defined explicitly with specific performance measures.
Unbundled Network Elements: "Each Network Element provided by SWBT to Sprint will meet applicable regulatory performance standards and be at least equal in quality and performance as that which SWBT provides to itself." (Attachment UNE 2.17.1) SWBT further commits to equal response times and priorities on trouble reports, as well as equal answer times in the repair center.
- Dialing parity, including post dial tone delay
- SWBT Service Center for ordering open to Sprint same hours as that offered to SWBT end users. SWBT Operating Systems available Monday-Saturday 7 AM - 11 PM; Sunday 11 AM - 11 PM.
- Firm Order Commitments within 24 hours
- Percent missed due dates
- Percent right the first time
- Percent no access
- Service Center response time
- Repair Center response times
- Percent repair missed appointments
- Ave duration time for trouble (Mean time to repair for designed services)
- Percent right the first time (Repeat reports)
- Percent report rate
- Percent no access
- Firm Order Commitments within 1-4 days (1 day for 2-wire analog loop)
- Specific intervals for order completion
- Speed of answer in the Repair Center
- Percent missed commitments
- Average outage duration time
- Percent right the first time
- Percent report rate
- Percent no access
Liquidated damages for non-performance in:
In its agreement with Sprint for interconnection in the state of Kansas, SWBT commits to the above performance measures related to Unbundled Loop Provisioning, Interim Number Portability Provisioning, and Out-of-Service Repairs.
Additionally, SWBT commits to measure order intervals for the following unbundled network elements, although many apparently are negotiated on an individual case basis (ICB): Network Interface Device (NID), Local Loop, Local Switching without Customized Routing, Operator Service/Directory Assistance, Interoffice Transport, Signaling Link Transport, SCP Databases, and Local Switching with Customized Routing. If meaningful ICB intervals are agreed upon, these can be excellent measures of product-specific performance adequacy. C. SWBT COMMENTS IN RESPONSE TO SECTION 272 FURTHER NOTICE OF PROPOSED RULEMAKING
This affidavit deals primarily with SWBT's fulfillment of requirements under Section 251 of the Act, requiring BOCs to provide wholesale inputs to carriers competing in the local exchange market.
However, in December 1996, the FCC, in its First Report and Order and Further Notice of Proposed Rulemaking, adopted non-accounting safeguards pursuant to Section 272 of the Act. Section 272 governs the entry by BOC's into the interLATA telecommunications services, interLATA information services, and manufacturing markets. These safeguards included a requirement that the BOCs make publicly available the intervals within which they provide services to themselves and their affiliates. The Commission proposed a standardized report format that included seven service categories as an appropriate means of making the information available. While Section 272 requirements do not
directly impact the requirements under Section 251, SWBT comments on those requirements may have a bearing on their ability and willingness to provide similar performance measure results for interconnection to CLECs.
The seven service categories proposed by the Commission in the Section 272 order are as follows:
- Loop provisioning intervals
- Interim number portability intervals
- Out of service repairs over 24 hours
SWBT proposes to update results for these seven performance measures on a monthly basis and would provide SWBT information on a corporate-wide basis. Key to determining market parity would be SWBT willingness to provide these measures more frequently on a geographic and class of service basis.
- Successful Completion According to Desired Due Date (measured as a percentage). SWBT states that they can provide this information. Additionally, they agree, as I do, that interval measurements should be based on company offered appointments only and should not include customer requested or desired due dates. This category also applies to the requirements under Section 251 of the Act.
- Time from BOC Promised Due Date to Circuit being placed in service (measured in terms of percentage installed within each successive 24 hour period until 95% installation completed). SWBT generally agrees with this requirement.
- Time to Firm Order Confirmation (measured in terms of percentage implemented within each successive 24 hour period, until 95% completed). With some needed clarification of start and stop times and new intervals for supplemental requests, SWBT agrees to provide this information in the format requested. As noted in this affidavit, this category also applies to the requirements under Section 251 of the Act.
- Time from PIC change request to implementation (measured in terms of percentage implemented within each successive 6 hour period, until 95% completed). This category is not pertinent to the requirements under Section 251.
- Time to restore and trouble duration (percentage restored within each successive 1 hour interval, until resolution of 95% of incidents). With some clarification, SWBT agrees to provide this interval information in the format requested. Although the 1 hour interval requirement does not pertain, this category clearly fits under the requirements of Section 251. (In their comments, SWBT argues against providing measurements related to New Circuit Failure Frequency and Trouble Report Rate. These measures, are important in assessing parity under the requirements of Section 251. However, SWBT has committed to both these measures in their Oklahoma interconnection agreement with Sprint.)
- Time to restore PIC after trouble incident (measured by percentage restored with each successive 1 hour interval, until resolution of 95% restored). This measure does not pertain to requirements under Section 251.
- Mean time to clear network / average duration of trouble (measured in hours). SWBT agrees to report interval information in this format, although they argue that this category duplicates the Time to Restore requirement. This category is critical to determining parity under the requirements of Section 251.
D. PERFORMANCE MEASURES NOT INCLUDED IN SWBT'S APPLICATION
SWBT's assertion that they will perform wholesale functions for CLECs at least equal in quality to those performed for itself or its subsidiaries is a sound basis for meeting the requirements of the Act. However, the ability to test whether parity exists or whether discrimination is taking place is dependent on the existence of explicit and specific performance measures and the reporting of results therein for SWBT and new entrants.
This affidavit is not an attempt to prescribe a model set of performance measures. Nor does it attempt to lay out a minimum set of performance measures that would meet the requirements of the Act. However, it is a discussion of typical performance measures for each of the wholesale functions BOCs will perform under the 1996 Act, required to provide resale services, unbundled network elements, and facilities-based interconnection. It also discusses examples of market and product parity measurements as well as administrative reporting mechanisms. The performance measure examples discussed below are not new. Most have been tracked and reported by BOCs internally, are reported to state or federal regulatory bodies, or have been proposed as parity measures by at least one BOC.
Pre-ordering: Pre-ordering performance measures revolve around the ability of a CLEC service representative to complete an order with an end user on line with at least the speed and accuracy of a BOC service representative taking a similar order from a retail end user. Since CLEC service representatives will likely interface with BOC OSSs and with BOC service representatives, performance measures are needed to measure the cycle time and reliability of both interactions. These measurements will ensure that BOC service representatives do not have an unfair advantage in creating a superior end user perception of speed and efficiency. Typical pre-order performance measures not specifically proposed by SWBT in their Section 271 application.
BOC OSS Response Time--Measure, in seconds, the speed with which CLEC service representatives receive the following information:
- Pre-order OSS Availability--Measures the percentage "up-time" of BOC interconnect systems. SWBT agrees to provide availability of systems in its Oklahoma interconnection agreement with Sprint.
- Pre-order BOC Service Center Availability--Measures the hours the BOC service center is open to CLEC queries. SWBT agrees to equal availability in its Oklahoma interconnection agreement with Sprint.
- Pre-order BOC Service Center Response Time--Measures how quickly BOC service representatives respond to CLEC queries.
- Agreed to in SWBT's Sprint Oklahoma interconnection agreement. Also proposed by Ameritech in their Michigan SGAT.
- Address Verification
- Request for Telephone Number
- Request for Customer Service Record (CSR)
- Service Availability
- Service Appointment Scheduling
Several such measures are proposed by Ameritech. These are important in creating a customer perception of equal calling time when placing an order with a CLEC.
Ordering: Ordering performance measures revolve around measuring the CLEC's ability to process end user service orders into the BOC and through the BOC OSSs with speed and accuracy at least equal to the BOC itself. Ordering cycle time is primarily measured by the promptness of communications between the BOC and the CLEC and by the success of order "flow-thru." Ordering reliability is measured by the accuracy of the service order. Typical ordering performance measures not specifically proposed by SWBT in its Section 271 application include:
As noted above, SWBT has agreed with this measure under the requirements of Section 272, calling for a renewed measure each time an order is subsequently submitted. I don't disagree with this requirement, however an overall measure per service order would be worthwhile in meeting the spirit of the Section 251 requirements.
- Firm Order Response Time provided by product, e.g., Resale POTS, UNE Loop, Trunk Order- An important adequacy performance measure because it measures whether CLEC service orders are processed in a manner that allows overall provisioning intervals to be at parity. If the service order does not flow speedily into the BOC OSSs, a lengthy provisioning interval and a due date miss is likely.
- Firm Order Commitment- SWBT agrees to this measure in its Oklahoma interconnection agreement with Sprint. Proposed by Ameritech in their Michigan SGAT. This notifies a CLEC that its service order has been accepted.
- Order Jeopardy- This notifies a CLEC that a due date must be changed.
- Order Reject- This notifies a CLEC that a service order contains errors.
- Order Completion- This notifies a CLEC that a service order has been completed.
Provisioning: Provisioning Performance Measures measure how quickly and how accurately end user service orders are completed. Parity in performing provisioning functions results in CLEC customers receiving service with speed and quality at least equal to that received by BOC retail or subsidiary customers. Provisioning measures have a long and detailed history within the BOCs. They are used to review and compare manager performance, as well as required by state and federal regulatory bodies. Provisioning is a process highly visible to end users and, therefore, is a key determinant to CLEC success in the marketplace. Typical provisioning performance measures not provided by SWBT in its Section 271 Application or any existing interconnection agreements, include:
- Flow-Through- Measures the percentage of service orders that flow to and through BOC OSSs without human intervention. This is an important measure in determining not only parity related to the service order processing cycle time, but also in the cost of the process to both the BOC and the CLEC.
- Service Order Accuracy--Measures the percentage of service orders prepared by the BOC exactly as ordered by a CLEC.
Maintenance: Maintenance performance measures depict two subprocesses:
- Installation Interval- Measured as a percentage of service orders completed in more than X days. Should be reported on a disaggregated product and market basis. Mentioned by SWBT in their application as a part of submitted FCC ARMIS data, but not defined as a performance measure. Proposed by Ameritech as a performance measure in their Michigan SGAT.
- Mean Installation Interval- Measured in days from end user request to order completion when the appointment is specified by the BOC. End user requested or desired due dates should not be included. Should be reported on a disaggregated product and market basis. This "raw" interval is as important, and perhaps more important, than the percentage of completions beyond a set objective. For example, if SWBT completes 95% of its own retail service orders within 5 days and 95% of CLEC resale orders within 5 days, it is still possible that the mean interval for SWBT retail orders could be significantly different (higher or lower) than the CLEC orders. Proposed by Ameritech as an audit process.
- Held Orders- Measures non-completed service orders held more than X days, usually held for lack of network facilities. This is an important measure in determining whether SWBT prioritizes new facility work in a nondiscriminatory manner.
- Completed Order Accuracy--Measures whether the end user received what the CLEC ordered.
- 911 Database Update Speed and Accuracy- Measures the percentage of missed due dates updating 911 database and the percentage accurate updates. Proposed by Ameritech in their Michigan SGAT.
Performance parity exists if a CLEC customer trouble is cleared with at least the same speed and quality as the BOC retail or subsidiary customer. This is a highly visible process to the end user and has significant impact on the end user's perception of the service provider. Typical maintenance performance measures not provided by SWBT in its Section 271 Application or any existing interconnection agreements, include:
- Trouble reporting and clearance, and
- Network quality. Trouble reporting performance measures describe how quickly and how well end user trouble is cared for.
Network Quality performance measures measure how well SWBT's network is maintained and whether SWBT's network performance discriminates against new entrants. Comparisons are between the performance distribution for SWBT retail or subsidiary customers and the performance distribution for CLEC customers. While it's not clear that this type of discrimination would be likely, network performance measures are critical to customer service and are also historically readily available. Typical network quality performance measures not provided by SWBT in its Section 271 Application or any existing interconnection agreements, include:
- Trouble Report Rate- Measured as the number of trouble reports per customer or access line. Data is gathered by product and market categories and can be analyzed by cause and other factors. This is the key measure of service reliability and, as a historical matter, positively correlates with an end user's perception of their provider. SWBT agrees to provide report rate results in its Oklahoma interconnection agreement with Sprint. Also proposed by Ameritech in their Michigan SGAT and by PacTel.
- Repeat Reports- Measured as the percentage of end user troubles on the same access line within an agreed number of days of the original trouble. Repeat reports are a key indicator of maintenance process reliability and, historically, have a positive correlation with an end user's perception of provider quality. SWBT agrees to provide repeat report results in its Oklahoma interconnection agreement with Sprint. Proposed by Ameritech as part of their Michigan SGAT and by PacTel.
- Mean Time to Repair- Measured as the average interval from trouble report to clearance. This is the key measure of trouble report cycle time. Should be gathered and reported on a product and market basis. SWBT specifies UNE Mean Time to Repair in many of their interconnection agreements and promises equal repair treatment in their interconnection agreement with AT&T in Texas, but does not propose specific Mean Time to Repair measures for all products and markets. SWBT has agreed to this measure under Section 272 requirements and in its Oklahoma interconnection agreement with Sprint. Ameritech includes this measure in their Michigan SGAT and PacTel has proposed it as well.
- Out of Service Over 24 Hours- Measured as a percentage of out-of-service troubles cleared within 24 hours. This measure relates to Mean Time to Restore, but specifically measures parity in out-of-service restoral. Required by many state regulatory bodies. Agree to by SWBT in its Oklahoma interconnection agreement with Sprint. Proposed by Ameritech in their Michigan SGAT.
- BOC Service Center Speed of Answer- Measures how quickly BOC repair service representatives respond to CLEC queries. This is an important measure of performance adequacy, relating to an activity not required by the BOC. Proposed by SWBT in their interconnection agreement with Sprint, but not specified as a performance measure. Also proposed by Ameritech in their Michigan SGAT.
Billing: Billing performance measures measure the timeliness, accuracy, and completeness of end user billing records and wholesale bills. These are measures of performance adequacy, and are important because, once provisioned, billing is the most frequent and visible contact an end user has with the provider. Typical billing performance measures not provided by
SWBT in its Section 271 Application or any existing interconnection agreements, include:
- Number of Major Network Events--Measures whether CLEC customers are disproportionately affected by significant switch or transmission down time. Because of their significance, Major Events are reported by all BOCs to the FCC as a part of Network Reliability Council requirements.
- System Signaling 7 (SS7) Link and Database Failures--Link Failure measurements proposed by Ameritech in their Michigan SGAT.
- Post Dial Tone Delay--Measured in seconds on various call combinations made by CLEC customers through BOC network to CLEC platform.
- Blocked Call Attempts--Measures blocked call attempts by CLEC customers through BOC network to CLEC platform.
- Various transmission measures, including loop transmission loss, signal-to-noise ratio, balance, and idle circuit noise.
Toll and Directory Assistance: Toll and Directory Assistance performance measures measure the speed of response to CLEC customer by BOC operators. They are measures of performance parity. Typical Toll and Directory Assistance performance measures not provided by SWBT in its Section 271 Application or any existing interconnection agreements, include:
- Bill Timeliness--Measures the percentage of end user and wholesale billing records delivered on time.
- Bill Accuracy--Measures the percentage of accurate end user and wholesale billing records.
- Bill Completeness--Measures the percentage of complete end user and wholesale billing records.
Market Parity: Market parity ensures that agreed to performance measures present appropriate customer group comparisons between SWBT and CLEC's. Customer groups generally fall into two categories: Geographic and Class of Service. For example, if a CLEC offers service in only one city, appropriate performance measures would provide comparable SWBT retail results
for that city only. Similarly if a CLEC targets only small business customers, appropriate performance measures would provide comparative SWBT results for its small business customers only. SWBT does not explicitly discuss geographic or class of service market parity in its application.
Product Parity: SWBT, in its Application and negotiated interconnection agreements, does include both Resale and UNE performance measures, but has not formally agreed to this breakout. Ameritech has proposed performance measures for both Resale and UNE in its Michigan SGAT. Product parity also requires that performance measures be identified, measured, and reported for product or product families a CLEC offers to end users. Examples include POTS, Subrate data, HICAP data, Centrex, and ISDN. If a CLEC offers DS1 service to its end users as part of a UNE loop resale arrangement, SWBT would need to provide results for service provided to those customers and for its own DS1 customers. Ameritech has proposed product-based performance measures in its Michigan SGAT.
Reporting Requirements: SWBT makes no mention of performance measure data availability. This would allow CLEC access to SWBT partitioned results databases, in turn allowing a CLEC to pull reports themselves. Further, SWBT does not explicitly specify entities to be measured. Examples include results for a particular CLEC, all CLECs, SWBT retail, and any appropriate SWBT affiliates. In its comments on service requirements under Section 272, SWBT argues against providing 101. results for individual affiliates. Ameritech has proposed to provide results for each CLEC, all CLECs, and their own retail end users in its Michigan SGAT, but not for its own affiliates. SWBT has not specified or provided examples related to performance report frequency, accuracy, or format.
- Average Speed of Answer-Toll--Measures raw interval in seconds or as a percentage under a set objective. Proposed by Ameritech in their Michigan SGAT.
- Average Speed of Answer-Directory Assistance--Measures raw interval in seconds or as a percentage under a set objective. Proposed by Ameritech in their Michigan SGAT.
SWBT's Section 271 application to provide in-region interLATA service in the state of Oklahoma includes a commitment to provide wholesale functions to new entrants at least equal in quality to that provided to its own retail end users. Further, the application proposes several specific performance measures that would allow, if properly disaggregated, a test of that commitment to parity. These proposed measures are nominally those reported to the FCC as part of ARMIS reporting requirements.
The application also refers to negotiated interconnection agreements as including other specific performance measures SWBT would be committed to for particular CLECs in particular markets. In Oklahoma, specified measures are UNE loop provisioning and maintenance cycle time and Interim Number Portability provisioning cycle time. Its agreement with Sprint is particularly robust with respect to performance measures. Finally in its agreement with AT&T in Texas, several Resale performance measures are also specified.
SWBT also agrees with a number of performance measures proposed by the Commission under Section 272 of the Act. Five out of the seven proposed measures also pertain to requirements under Section 251, implying SWBT's support for these measures.
While few performance measures are explicitly proposed in SWBT's Section 271 Application, many are implicitly discussed and others are identified or discussed in interconnection agreements or regulatory proceedings. It follows that SWBT could make these additional performance measures an explicit part of their 271 application. However, some performance measures needed to determine parity in SWBT's provision of wholesale products are not identified in any document or proceeding. Examples include:
Additionally, SWBT has not discussed providing appropriate market parity reports. They have discussed performance measure report frequency and comparison entities in their Oklahoma interconnection agreement with Sprint, but have not provided explicit examples. Product parity is implied by SWBT's separate treatment of resale and unbundled network elements, but no commitment is made to a broader recognition of different CLEC offerings.
Although SWBT has clearly committed to adequate and parity performance, their application should include more explicit identification of performance measures, including sample reports, that would allow competitors and regulators to judge whether adequacy and parity have been achieved for all wholesale functions. As at least a rough guide to providing such explicit identification, SWBT's Oklahoma interconnection agreement with Sprint and Ameritech's Michigan SGAT and subsequent performance measure proposals, attached, represent a good beginning.
- BOC OSS response time for preorder functions
- Order jeopardy, reject, and completion notice cycle time
- Service order accuracy
- Service order flow through
- Installation interval measured as a percent of agreed to intervals
- Mean installation interval
- Held orders
- 911 database update speed and accuracy
- Major network events
- SS7 link failures
- Blocked call attempts
- Various transmission measures
- Bill timeliness
- Bill accuracy
- Bill completeness
- DA and toll speed of answer