ADMINISTRATIVE REPORT

Date: Mar 6, 2000
Author/Local: M. Crocker/7647
RTS No. 00064 CC File No.: 1151 CS&B: March 30, 2000

TO: Standing Committee on City Services and Budgets

FROM: Acting Director of Information Technology

SUBJECT: Information Technology Infrastructure - 1999/2000 Expansion and

Replacement Program

RECOMMENDATION

THAT Council approve the 1999/2000 information technology infrastructure expansion and replacement program consisting of expansion and upgrade of networking and telecommunications infrastructure, replacement of obsolete desktop computers, and upgrade of telephone system hardware and software, as detailed in this report, at an estimated capital cost of $2.78 million. Additional annual operating costs of $100,000 will be incurred to cover the upgrades of the network and telecommunications infrastructure and the telephone system. Both capital costs and operating cost increases will be funded from the Information Technology Long Term Financing Plan.

GENERAL MANAGER’S COMMENTS

The General Manager of Corporate Services RECOMMENDS approval of the foregoing, noting that the approach to microcomputer acquisition and management will be reviewed in 2000. Options to be considered will include leasing, possibly bundled with other services.

COUNCIL POLICY

On December 8, 1994, Council, while considering the “Next Steps to Better City Government” reports, which included the Information Systems Plan Report, directed:

- THAT staff prepare a detailed infrastructure and software implementation plan and budget for consideration in the 1997 - 1999 Capital Budget.

On July 23, 1996, Council approved a “Long Term Financing Strategy - Information Technology Replacement Program”, that recommended, amongst other things:

- THAT Council approve the financing strategy as outlined in this report to provide a sustaining source of funds for information technology implementation and renewal.

PURPOSE

The purpose of this report is to seek approval to continue the build-out and replacement of the City’s information technology, communications and systems infrastructure.

Council approved the second year (1998) Technology Infrastructure Expansion and Replacement Program on Oct 8, 1998. The achievements of that program are described briefly in the next section of this report. This third year, 1999/2000, includes:

… Continued expansion and upgrade of network and telecommunications facilities to provide staff throughout the organization with access to the information and line-of-business systems they need to serve their customers effectively;
… Continued replacement of obsolete microcomputers;
… Upgrade of various components of the City Hall telephone system;
… Continued upgrading of the City’s e-mail infrastructure.

BACKGROUND

The City organization has a capital investment of at least $25 million in computers, software, networks and related equipment. This does not include the investment in services or in staff resources that has been made to implement these systems. It is a measure of the size of the City’s computer-related asset base.

All assets age. Information technology assets age faster than many others because the rate of technological change frequently leads to functional obsolescence even before physical obsolescence.

At the same time, driven in part by internal efficiencies and in part by public expectations, the City is continuously adopting new information technology based approaches to service maintenance and delivery.

The need for technology and systems solutions were identified through:

… The 1992 Information Systems Plan, recommending replacement of mainframe-based systems with networked systems;
… CityPlan and subsequent initiatives during 1995/1996, which encouraged, among other things, a more neighbourhood-based delivery system, and recognized that information systems and communications technologies were key to achieving this goal;
… Development and approval in 1996 of a long-term financing strategy for information technology replacement.

The 1996 long-term financing strategy report identified not only the need to replace core systems (like the financial and human resource systems), but also the need for continuous replacement and upgrade of infrastructure items that are shared by staff and by information systems. The Information Technology Infrastructure Expansion and Replacement Program addresses the timely replacement and upgrade of those items to meet evolving business needs.

THE 1998 PROGRAM

Most of the goals of the 1998 program were achieved. Notable infrastructure upgrades included:

… Telecommunications services to a number of Park Board facilities, firehalls, the twocivic theatres, VPL, Parking Enforcement, the building at 456 W. Broadway, and the Dog Pound. The switches and routers that support the City Hall complex were replaced to accommodate increased demand;
… Around 23 network servers were replaced or upgraded;
… The roll-out of Windows 95 throughout the City was completed, along with the replacement of almost 600 obsolete microcomputers;
… The City’s internal and public web servers were replaced;
… The IBM mainframe was replaced, with savings in software licensing, maintenance and power consumption offsetting the capital costs over its projected 3-year life;
… An uninterruptible power supply (UPS) was installed. In concert with the emergency generator, this protects the critical computer systems at City Hall from interruption and possible loss of data resulting from a sudden loss of power;
… An enterprise tape backup system was installed to provide robust data protection for over 40 departmental and corporate servers and over 3 terrabytes (the equivalent of over 1 billion text pages) of data.

Two of the sub-programs did not achieve their goals:

… The upgrade of the City e-mail system was only partially completed, and will become part of the 1999/2000 program, using funding approved for the purpose in 1998. The progress that has been made is described in the appendix to this report;
… The program to provide web-based access to document databases was deferred pending completion of a review of the City’s document and records management strategy by the City Clerk’s department, and selection of a product that supports that strategy.

The table on the next page shows actual sub-program costs against budget:

Sub-Program	1998 Budget ($000's)	Spent as of Nov 30,1999 ($000's)
Networking and Telecommunications	692	689
Desktop Computer Upgrade & Replacement	1,810	1,834
E-mail	500	75
Desktop & Service Management	295	294
Web Access to Document Databases	75	0
Internet Servers	40	25
Electrical Power Backup	40	41
Enterprise Tape Backup	80	61
TOTAL	3,532	3,019

The 1998/1999 period was one of unusual pressures on the Information Technology organization:

… The implementation of the SAP financial and human resource modules made demands on both staff and the technology infrastructure.
… The threat of the Y2K bug diverted staff resources and prompted some acceleration of infrastructure replacement.

The 1998 report to Council anticipated these pressures and stated that “There is a risk that some components of the program will not be completed in the 1998 budget cycle”. Most were completed, with the exceptions noted above.

THE 1999/2000 PROGRAM

The 1999/2000 program is described briefly in the following table. Each sub-program is classified as one of:
… Ongoing: Consists of numerous small items which can be expected to continue at similar levels from year to year;

… One-time: A project expected to be completed within the current year, or a one-time acquisition (although it will still need replacing at the end of its life cycle);
… Multi-year: A project that will take more than a single year to complete, for which additional funding approval will be sought in subsequent years.

Capital costs and ongoing operational costs are also identified.

Projected Projected

Sub-Program Description Capital Cost* Annual Cost

Increase

* Capital costs include acquisition, implementation and staff training.

More detailed descriptions of each of these infrastructure sub-programs, with an explanation of what they will help the City achieve, are contained in Appendix A.

The 1998 infrastructure replacement program extended right through 1999, leaving scheduled replacements backlogged in some areas. Some catch-up is included in this program.

The costs of developing and maintaining a technology infrastructure that can support the City’s many and varied service delivery needs are significant. Technology infrastructure replacement and upgrade will demand continued annual funding at levels comparable to those in this report.

CONCLUSION

The City has replaced a number of its core information systems, and is implementing systems which support different ways of working and new ways of delivering services to the public. These systems demand a technology infrastructure that extends to all parts of the organization, regardless of location. It needs to be robust, dependable and fast, and it needs to be adequately supported.

This report describes the third year of an ongoing program in which the technology infrastructure is recognized as a corporate utility, its maintenance as a corporate responsibility. While the details and emphasis may change from year to year, continuing maintenance and upgrade of this infrastructure is essential as the City looks increasingly to technology to facilitate the delivery of its services.

- - - - -

Networking and Telecommunications

The Telecommunications Network ($317,000):

The City has around 120 work sites - libraries, community centres, recreation facilities, firehalls, works yards, residences and others. It currently uses a mix of City-owned cable plant and relatively inexpensive low-speed leased telecommunications services to provide data and systems connectivity to these sites. Annual cost of the leased services is around $140,000. This approach, a bridging strategy that anticipated the imminent arrival of ubiquitous inexpensive high-speed broadband telecommunications services, is no longer adequate. The demands on the network, a result of the growing use of shared information systems, are starting to outstrip its capacity. A new approach must be developed that can accommodate higher network throughput.

The City has considered several options, ranging from the extremes of building its own network using optical fibre and wireless, to leasing commercial services exclusively. Both extremes were found to be expensive, with the recommended approach being a “Build/lease” hybrid of the two:

… Use existing City-owned underground cable plant where available;
… Deploy a leased high-speed telecommunications backbone to around 20 geographically distributed hub sites;
… Deploy City-installed and managed wireless links where line-of-sight is available between the hub sites and neighbouring work sites (about 40 sites);
… Use lower-speed leased services for sites not serviced by the above. There are a number of new service offerings and growing competition in this market, and prices are dropping.

An comparison between the hybrid and lease approach is summarized in the following table:

Operating Cost Comparisons by Network Type
	Current	Build/Lease	Lease Exclusively
Network type	-City cable plant -Very low-speed leased	-City cable plant -Wireless -High-speed leased -Low-speed leased	-High-speed leased -Low-speed leased
Annual Operating Costs (estimated)	$140,000	$290,000	$1,000,000

As this table illustrates, the “Build/lease” approach is very cost-effective. It uses the City’s investment in cable plant, it realizes economies of scale in the sharing of high-speed leased services, the wireless links have a very low operating cost, and the City can take advantage of new service offerings as they become available and cost-competitive. The cost savings of this approach result primarily from the low operating cost of the proposed wireless links and the City cable plant, which can between them provide services to around 60% of City sites.

24 work sites have an immediate need for increased data communications capacity, resulting from their expanding use of e-mail, the Internet, financial, human resource and other administrative systems, departmental line-of-business systems, and direct delivery of services to the public. Other applications are anticipated - the convergence of voice and data networks into a single unified network, disaster response and recovery initiatives, remote security monitoring, integrated point-of-sale systems, public self-help kiosks, centralized document and records management systems, and multimedia use for training or videoconferencing, as examples. All would demand significant increases in capacity.

The City proposes to follow the “build/lease” hybrid approach described above to satisfy these needs. Telecommunications providers will be invited to provide information, to bid on the services they offer, which are changing rapidly, and to confirm the City’s cost projections.

Local Area Networks ($320,000):

At around 60 work sites, Local Area Network (LAN) file and print servers provide information storage and application services, minimizing data transfer over the wide-area network. These servers are critical to day-to-day business operations. Most are second-generation. A few are first-generation and need to be replaced. Many need increased storage capacity. An upgrade of the network operating systems on many of the larger City Hall networks is under way. The new versions permit cost-effective incremental expansion in the number of users (which a licensing constraint on old versions inhibits), and provide additional functionality. The older versions are six years old and no longer supported by the manufacturer.

Local Cabling ($20,000)

Other infrastructure items included in this budget are upgrades of local telecommunications hubs and workstation cable drops so that every microcomputer in the City can have access to shared information services.

One-time funding requested: $657,000
Annual operating costs:

Telecommunications services and hardware maintenance: $90,000

Microcomputer Replacement

The City has around 4,000 microcomputers. Close to 1,000 need to be replaced annually, given a 4-year useful life before technological obsolescence (too slow, incapable of running current software, no longer supported by extended warranty, failure-prone and expensive to maintain and support) sets in.

Four hundred terminals at the Vancouver Public Library (VPL) are used by the public to access the Dynix automated library system. These are up to 9 years old. Not only are they showing their physical age, they are also incompatible with the current generation of Dynix software, and must be replaced with microcomputers.

This request will replace a further 400 VPL terminals and 380 microcomputers that are more than 4 years old and no longer serve the needs of their users. The target group is those machines that contain processors from a 486/66MHz up to a Pentium 120MHz.

This is normal replacement, consistent with the City’s replacement policy. No training costs are included. An allowance is included for installation and replacement of monitors and directly associated peripheral cards as needed. Unit cost of replacement microcomputers is approximately $2,000, which includes a 3-year on-site extended warranty.

One-time funding requested: $1,630,000

Annual operating costs:

Software support, maintenance and upgrades: (Included in warranty)

Telephone System Upgrades

A number of components of the telephone system are overdue for upgrade or replacement:

Replacement of Backup Power Supply ($45,000)

The purpose of the power supply is to keep the telephone system operational during a power failure until the generator can take over. The existing power supply (batteries and rectifiers) was installed approximately 20 years ago. Recent testing by staff indicates that the batteries may not be capable of holding the full load for the eight hours of standby they are supposed to.

Replacement of old line and trunk cards ($140,000)

Each telephone trunk and telephone set at City Hall terminates on either a trunk or line card. A large number of these cards (and the cabinets that house them) are old and will soon not be supported by the manufacturer. Funding is needed to start replacing these basic components in the telephone system.

Addition of two shelves in the telephone switch ($30,000)

The need for telecommunications services (telephones, modems, fax lines, ICR, voice mail, etc.) is continually on the rise. Not only is there very little space left on the existing common equipment for expansion, but as the old line and trunk cards are replaced, the new ones have to be installed in a new shelf.

Upgrade of telephone system administrative software ($15,000)

This software provides call detail recording, call accounting, traffic analysis and system administration for the telephone system. It also has a module that will integrate multiple vendor bills into one report. This will streamline and greatly assist in the payment and allocation of telecommunications costs.

Upgrade of telephone trunks between Manitoba Works Yard and City Hall to Digital ($30,000)

There are currently four analog trunks connecting the City Hall and Manitoba Works Yard. These support only a portion of the calls that are made between the two sites. Once they are busy, the calls overflow to the public telephone network, tying up leased trunk lines at either site.

After upgrading, operating costs will remain about the same and capacity will increase by almost six times (to 23 trunks). This will also allow for a cleaner integration of the systems at either end, including the voice mail.

Upgrade Automated Call Distribution system ($70,000)

Automated Call Distribution (ACD) is the component of the telephone system that takes large numbers of incoming calls and distributes them (in order) to the first appropriate, available agent (staff member). If all agents are busy, the system will answer the call, play an appropriate announcement to the caller and hold the call until an agent is available.

With the telephone inquiries from the public increasing, the existing ACD system does not provide thefunctionality and management tools needed to ensure the high volume of incoming calls are answered within acceptable time limits.

There have been several requests from many different department managers to provide them with better tools to allow their staff to better serve the public. To this end, a needs analysis is currently underway to determine the short and long term ACD requirements.

Upgrade Voice Mail ($165,000)

The voice mail system is an integral business tool used by the City to manage telephone calls and provide information to the public. Like the telephone system, it has to be up and working at all times. The voice mail system is also closely integrated with the Automated Call Distribution service.

The existing voice mail system was a low-end system when acquired 6 years ago. It has difficulty serving the current needs of over 1,000 users. A new, stable system is required to ensure that all calls answered by the voice mail system (either on an individual’s phone or on an ‘information tree’) are answered reliably.

A new voice mail system can also provide a basis for unified messaging, which will be of great assistance in managing e-mail, voice mail and faxes.

One-time funding requested: $495,000

Annual operating costs:

Software support, maintenance and upgrades: $10,000

E-Mail

The roll-out of Outlook 98, the current generation of e-mail that is to replace the City’s 8-year old MS-Mail system, has taken longer than expected. This has been largely a consequence of other, more urgent, priorities. Considerable work has been done in preparation:

… A server running Microsoft’s Exchange software was installed early in 1999;
… Staff have developed the skills needed to install, configure and maintain the new system;
… A pilot group of over 100 staff were switched to the new system in 1999, providing valuable insights into solving the technical challenges of the roll-out, like migration of saved mail and calendars, and retention of staff’s ability to communicate with one another during the change.
… The pilot group has learned to use a new, more functional, but also more demanding tool;
… Work is under way to develop training materials and approaches;
… A migration plan has been developed.

Some 2,400 staff use MS-Mail. Of these, around 400 have been moved to Outlook 98. The remaining 2,000 are expected to take through to the end of 2000. Deployment and staff training are significant components of this effort.

To reiterate the major benefits of the new e-mail system:

… It will be more reliable and more available. The current system works well on a local area network, but has many potential points of failure when extended to the enterprise. This results in frequent interruptions in mail delivery, particularly with external mail, and considerable staff resources allocated to fixing problems;
… It offers features, now considered standard, that are not available on the old system, for which no enhancements have been developed for a number of years. This impacts on the professionalism of the City’s communications with its public and with the vendor community.

City staff send and receive around 20,000 e-mails a day, of which 5,000 are exchanged with the outside community of public and vendors. E-mail has become, along with the telephone, essential to the City’s conduct of its business, both internal and external. Failures can and do cause disruption, and it is important that the City replace its dysfunctional legacy system.

Funding was approved by Council in the 1998 program.

Comments or questions? You can send us email.

(c) 1998 City of Vancouver