ADMINISTRATIVE REPORT
Date: Sept 23, 1998
Author/Local: M. Crocker/7647RTS No.: 92
CC File No.: 1186
TO: Standing Committee on City Services and Budgets
FROM: Chief Information Officer
SUBJECT: Information Technology Infrastructure - 1998 Expansion and
Replacement Program
RECOMMENDATION
THAT Council approve the balance of the 1998 information technology infrastructure expansion and replacement program consisting of:
A. Expansion and Upgrade of Networking and Telecommunications Infrastructure;
B. Replacement, Upgrade, and Standardization of Desktop Computers;
C. Acquisition of Desktop and Service Management Software;
D. Upgrade of City E-mail Software;
E. Provision of Universal Access to Document Databases;
F. Upgrade of Internet and Intranet Servers,as detailed in this report, at an estimated cost of $3.41M, funding to be provided from the Technology Infrastructure Replacement financing plan. Additional annual operating costs in 1999 and thereafter, estimated at $126,000, will be accommodated through savings within the existing Information Technology operating budget.
GENERAL MANAGER'S COMMENTS
The General Manager of Corporate Services RECOMMENDS approval of the foregoing.
COUNCIL POLICY
On December 8, 1994, Council, while considering the "Next Steps to Better City Government" reports, which included the Information Systems Plan Report, directed:
- THAT staff prepare a detailed infrastructure and software implementation plan and budget for consideration in the 1997 - 1999 Capital Budget.
On July 23, 1996, Council approved a "Long Term Financing Strategy - Information Technology Replacement Program", that recommended, amongst other things:
- THAT Council approve the financing strategy as outlined in this report to provide a sustaining source of funds for information technology and renewal.
PURPOSE
The purpose of this report is to seek approval to continue the build-out and replacement of the City's information technology, communications and systems infrastructure.
An interim report, involving three high-priority items at a cost of $120,000, was approved by Council on July 30, 1998.
BACKGROUND
The 1997 Information Technology Infrastructure report (July 31, 1997) presented a chronology of:
· The 1992 Information Systems Plan, recommending replacement of mainframe-based systems with networked systems;
· A recognition, formalized in CityPlan and subsequent initiatives during 1995/1996, of the need to provide services at the neighbourhood level, and a growing recognition that information systems and communications technologies are key to achieving this goal;
· Development and approval in 1996 of a long-term financing strategy for information technology replacement.Council approved the first year (1997) Technology Infrastructure Expansion and Replacement Program on July 31, 1997. The achievements of that program are described briefly in the Background section of this report
Since that report, Council has also approved replacement of the City's core financial and human resource systems. This project, now known as Infoleap, along with other Better City Government initiatives, has highlighted the City's technology infrastructure needs.
The 1997 Infrastructure Program
In July, 1997, Council approved a budget of $2.83M for the Technology Infrastructure Expansion and Replacement Program. 1997 was the first year for which a long-term source of funding was available for corporate technology infrastructure replacement. The 1997 budget was developed with the expectation that 1998 funding would be approved in May, 1998, for a program year of May through April. Spending against budget for each of the sub-programs is summarized in the following table:
Sub-Program
Spent as of April 30, 1998 ($000's)
Budget ($000's)
Networks and Servers
1,034
1,317
IBM Terminal Replacement
635
893
9
199
Network Management
47
120
Desktop Upgrade / Microcomputer Replacement
829
270
Associated 1997 Operating Costs
10
30
TOTAL
2,564
2,829
The funding balance remaining on April 30 was subsequently spent, primarily on microcomputer replacements originally planned for the second year of the program, and the funding request for 1998 has been reduced accordingly. Variances from anticipated budget levels are due to timing differences and to revisions in estimates for multi-year programs. Most significantly:
· The Desktop Upgrade (microcomputer replacement) budget was significantly exceeded. This was the first year of a two-year program. As described in the appendix, total projected program costs have increased, but the primary reason for the over-expenditure is timing - we were further into the two-year program than anticipated in the 1997 report;
· The E-mail budget was underspent by $190,000 because the planned upgrade of e-mail server software was deferred to the 1998 program year, where it is part of a more comprehensive program that also includes upgrading the desktop e-mail software;
· The Network Management budget was underspent because the evaluation of desktop management products took longer than expected. A recommendation is now ready, and this item is included in the 1998 program.Many of the goals of the 1997 program were achieved during the year. Highlights include:
· Replacement of network servers or upgrade of server hardware or software in Engineering, Park Board, Community Services, Corporate Services, Legal Services and Police;
· Installation of high-speed telecommunications services to most fire halls;
· Installation of lower-speed leased telecommunications services to all branch libraries;
· Installation of optical fibre connectivity in Manitoba Yards;
· Upgrade of the City Hall telecommunications backbone to higher speed and throughput;
· Installation or upgrade of telecommunications services to Police, several Community Centres, Evans Yard, Cambie Yards, the Dog Pound, and Bylaw Fines' new office;
· Replacement of some 200 IBM mainframe terminals with microcomputers;
· Upgrade of the Internet firewall to keep pace with changing security threats;
· Acquisition of network problem diagnostic and management tools;
· Upgrade of around 1,000 microcomputers to a common Windows 95 standard, replacing obsolete computers where necessary, and upgrading where possible.A few projects, targeted for completion in 1997, are incomplete and now become part of the 1998 program:
· Installation of high speed telecommunications services to 4 fire halls not connected in 1997;
· Installation of telecommunications services to the Gathering Place, Carnegie Centre, Parking Enforcement, and upgrade of telecommunications services to 6 branch libraries;
· Server and telecommunications upgrades for a number of Community Centres, bringing them all up to an acceptable standard of capacity and reliability.DISCUSSION
The City's technology infrastructure includes
· shared services - the IBM mainframe, Internet/intranet, the phone and e-mail systems, dial-in and security facilities, central address database, corporate directory;
· telecommunications facilities - the data lines, switches, routers and gateways needed to provide all workgroups in the City with access to the shared services above;
· local area networks (LANs) - network servers and operating systems needed to provide basic file storage and printing services;
· local data and phone cabling "to the wall outlet";
· core desktop software - operating system and utilities, e-mail, scheduling, browser.As infrastructure components become technologically obsolete, they must be replaced. As usage grows, and existing capacity is no longer adequate, equipment must be upgraded. As new needs are identified, the infrastructure must be expanded to accommodate them. Not only is this infrastructure critical to the City's operation, it is also complex, and the staff costs of supporting it are increasing. Containing these costs is a major objective.
The 1998 Systems Infrastructure Program concentrates on the following areas:
· Continued expansion and upgrade of network and telecommunications facilities to provide staff throughout the organization with access to the information and line-of-business systems they need to serve their customers effectively;
· Continued replacement of obsolete microcomputers and upgrade of others so that all will support a common desktop operating system. In conjunction with the acquisition and implementation of desktop and service management software, this will help to contain the burgeoning costs of supporting the City's installed base of microcomputers;
· Programs to upgrade the City's e-mail infrastructure, the replacement of web servers, and the installation of a web interface that will provide the public with access to City document databases, enabling the City to further pursue the goals of accessibility and provision of on-line services to the public, and(these items were approved by Council on July 30, 1998)
· Replacement of an aging IBM mainframe with current technology, which offers improved reliability, space and power savings, and better network connectivity, at lower cost;
· Power protection and enterprise tape backup that will help to protect against service disruptions and data loss.The 1998 program components and their capital and ongoing operating costs are described briefly in the following table. More detailed descriptions of each of these infrastructure sub-programs, with an explanation of what they will help the City achieve, are contained in Appendix A. Each sub-program is classified as one of:
· Ongoing: Consists of numerous small items which can be expected to continue at similar levels from year to year;
· One-time: A project expected to be completed within the current year, or a one-time acquisition (although it will still need replacing at the end of its life cycle);
· Multi-year: A project that will take more than a single year to complete, for which additional funding approval will be sought in subsequent years.
Projected Projected
Sub-Program Description Capital Cost* Annual Cost
Networking and Telecommunications
(ongoing)Upgrade existing or extend new network communications services to 35 sites and replace, upgrade and consolidate local networking facilities;
$692,000
$70,000
Upgrade and Standardization of Desktop Computers
(Year 2 of 2-year)Replace obsolete microcomputers and accelerate the upgrade of desktop microcomputer operating systems to a common (Windows `95) standard;
$1,810,000
None
Desktop and Service Management
(Year 2 of 2-year)Build, through the acquisition and implementation of desktop and service management tools, the City's ability to manage a growing number of workstations while containing support costs;
$293,000
$23,000
Upgrade City E-mail software (One-time)
Upgrade e-mail and scheduling client software and server infrastructure;
$500,000
$25,000
Web-Based Access to Document Databases
(One-time)Provide universal public & staff access to City document databases;
$75,000
$8,000
Internet / Intranet Server Upgrades
(One-time)Upgrade Internet and intranet servers to support increased use by public and staff;
$40,000
None
TOTAL
$3,410,000
$126,000
* Capital costs include acquisition, implementation and staff training.
This is an ambitious program, a response to the growing demands made on technology in the City. Planning and resourcing a program of this magnitude is a significant challenge for the City information technology organization, at a time when retaining and recruiting skilled staff is becoming increasingly difficult, implementation of the financial and human resource systems and other systems initiatives are absorbing a lot of staff resources, andYear-2000 issues have to be addressed within a fixed timeframe and may require un-anticipated staff resources.
The costs of developing and maintaining a technology infrastructure that can support the City's many and varied service delivery needs are significant. Technology infrastructure replacement and upgrade will demand continued annual funding at levels comparable to those in this report. An information technology business planning committee has been established to review the infrastructure planning on an ongoing basis, and a process is being put in place to regulate the replacement of technology as a regular activity in a planned and controlled manner.
FINANCIAL IMPLICATIONS
Funding for the implementation costs of the 1998 Systems Infrastructure program ($3,410,000) is to be provided from the Systems Replacement Long Term Financing Plan approved by Council in July 1996. This cashflow, funded from the annual Operating Budget and augmented by operating savings arising from new technology and software solutions, is being managed to provide funding for the upgrading and replacement of the City's information systems to support the Information Systems Plan.
Ongoing operating costs related to the 1998 program, estimated at $126,000 annually for telecommunications costs and for ongoing software licencing and support can be provided from within the existing Information Technology budget through operating savings associated with systems upgrades.
CONCLUSION
The City is looking to replace or significantly upgrade a number of its core information systems, and to implement systems which will support different ways of working and new ways of delivering services to the public. These systems will demand a technology infrastructure that extends to all parts of the organization, regardless of location. It will need to be robust, dependable and fast, and it will need to be adequately supported.
This report describes the second year of an ongoing program in which the technology infrastructure is recognized as a corporate utility, its maintenance as a corporate responsibility. While the details and emphasis may change from year to year, continuing maintenance and upgrade of this infrastructure is essential as the City shifts to technology to conduct business.
Networking and TelecommunicationsThe City has in excess of 120 remote work sites. Many are small, with few staff and relatively minor requirements for telecommunications connectivity with the rest of the organization. Others are large, and have demanding requirements which may involve:
· Sharing information and information systems with the rest of the organization;
· Exchanging e-mail, either internally or with the public or vendor community;
· Direct data entry into, for example, payroll and purchase requisition systems;
· On-line dispatch of emergency and other vehicles;
· Direct electronic services to the public.Most work sites fall between these two extremes.
In all cases, the telecommunications requirements are growing. In 1997, the City installed or upgraded more than 50 telecommunications lines in its wide-area network (WAN). In 1998, plans call for another 35. The City is following a strategy of using whatever telecommunications facilities are available and are most cost-effective for a particular work site, while minimizing capital investment. When the telephone and cable companies have built out their data communications infrastructure, and the City's requirements can no longer be met by a private wide-area network, the City will wish to take advantage of their services. At the moment, these services are expensive and inaccessible from many City work sites. We expect a mix of City-owned and leased services for several years.
At many work sites, Local Area Network (LAN) file and print servers provide information storage and application services, minimizing data transfer over the wide-area network. These servers have become critical to day-to-day business operations. They were typically installed 3 to 5 years ago, and many need to be replaced or upgraded because they lack either reliability or capacity. A few new ones are being installed. Approximately 15 were upgraded in 1997; another 22 will be upgraded in 1998.
Other networking and telecommunications infrastructure cost items included in this budget are upgrade of the central telecommunications hub(s), and workstation cable drops so that every microcomputer in the City can have access to shared information services.
One-time funding requested: $692,000
Annual operating costs:
Telecommunication leases & h/w maintenance: $70,000
Staffing implications: NoneUpgrade and Standardization of Desktop Computers
There are many benefits to standardizing on a current-technology desktop operating system throughout the City organization; the need to implement new applications which demand a current operating system (e.g. the SAP user interface), the desire to implement desktop management tools which function better in a homogeneous network, and the need to control the costs of support, which are higher in diverse environments.
Upgrading to a standard desktop operating system would naturally be coordinated with the microcomputer replacement program, through which around 500 obsolete microcomputers are replaced annually. As described in the 1997 report, it was recommended that this process, which would take 4 or 5 years to complete, be accelerated and completed in 1998. This target is still expected to be achieved, although completion is now scheduled for the end of the year.
The $2.75M total cost of the program is higher than the original $1.5M estimate, made in 1997. Primary reasons for this are:
· the scope was widened to include replacement of laptop computers, previously excluded;
· Vancouver Public Library microcomputer replacements, excluded from the original estimate, are included for 1998;
· more microcomputers than anticipated had to be replaced instead of upgraded.However, it should be noted that the bulk of these costs are just those that can be expected as the result of a normal replacement cycle of obsolete equipment. While the cost will be higher than usual for this year, the portion attributed to earlier than expected replacement will reduce costs in the next few years.
The cost estimates include installation, using temporary staff to perform some of the work and to backfill for more experienced staff assigned to co-ordinate and lead the installation activities. Funding is also included for:
· A temporary Help Centre support position ($25,000) through the end of the program, as experience has shown that the support load is increased while the upgrades are being rolled out and staff are dealing with a new and unfamiliar interface;
· Staff training of $50,000;
· Contingency of $75,000.Total projected program costs $2,745,000
Less expenditures from the 1997 program: ($935,000)
_________
One-time funding requested: $1,810,000
Annual operating costs:
Hardware maintenance: (included in 3-year warranty)
Staffing implications: NoneThe use of microcomputers will likely continue to expand and exceed the capacity of current replacement funding. Under the current funding we have two choices: either extend the life-cycle or constrain supply. Extending life-cycle would put a limit on what we can achieve with software, as we must standardize on software which can be run on the oldest equipment installed. Constraining supply would limit the number of computers in the ongoing replacement program and put pressure on the user departments to make cost tradeoffs between technology and other priorities. We intend to pursue a model of constraint and select a life-cycle of approximately four years on which to base replacement.
Desktop and Service Management
Supporting over 3,000 desktop microcomputers, their users, another 100 or so file and application servers, and a network which connects all these together, is a task which consumes a growing number of staff resources, and still the level of support provided is inadequate. The City is not alone in facing this challenge, and a number of vendors have developed suites of software tools to aid this task through a mix of automation and remote management. These tools can broadly be divided into three categories:
· Systems management tools include automated scheduling of system tasks, monitoring of availability, provision of "early-warning" diagnostics, data backup, data archiving and security management, across a range of different file and application servers;
· Network management tools perform many of the same tasks for the communications network;
· Desktop management tools help to manage the microcomputers, the software they use, the screen interface they present to the user, and the management and quality of user support.Systems management will be addressed in the future. Network management tools were acquired in 1997 and are being implemented. This request relates to desktop management. Early in 1998, the City issued a Request for Proposal (RFP) describing the City's requirement for tools that provide the following:
· Electronic software distribution, so that new software, upgrades and software patches can be distributed, potentially overnight from a central location, reducing the need for technical staff to travel to the site;
· Remote control, enabling a central support group to respond to a support call by sharing a view of the remote user's computer screen, and taking control if necessary, again without needing to travel to the site;
· License monitoring, which provides a way to monitor what software is being used on a network. This information is needed to ensure license compliance;
· Asset discovery, which can identify hardware and software configurations on remote machines, and load them into a central database;
· Desktop management, which allows system administrators to customize what icons their users see on their screens, serving both ease of use and license management purposes by ensuring that their desktops are not cluttered with icons for software they don't use or that provide access to software for which they aren't licensed.The RFP asked vendors to quote on implementation services as well as one-time acquisition and annual licensing costs. Five responses were received. In the absence of funding approval, no award has been made, but the evaluation has been completed, and funding is requested to cover the following estimated costs:
One-time funding requested: $293,000
Annual operating costs:
Software license renewal and maintenance: $23,000
Staffing implications: NoneUpgrade City E-mail Software:
In 1997, the City upgraded its e-mail "backbone", both to improve connectivity with Internet mail (bringing this function in-house) and to provide a basis for the implementation of current-technology end-user e-mail software.
At present, City staff are using MS Mail, which is 6-year-old technology, and lacks many of the functions that are now considered standard. Most significantly, it is architecturally weak, demanding a complex topology with many post offices and many points of failure. As happens when technology becomes obsolete, no enhancements are being offered and integration with other systems is being discontinued. As a consequence, communication between staff and the public suffers from some constraints and interruptions, a considerable amount of technical staff time is spent monitoring the system and recovering from failures, and a number of more advanced functions are not available to staff.
The e-mail client replacement, currently expected to be Outlook `98 although this is yet to be finalized, will use Microsoft Exchange servers on the upgraded backbone for message storage, rather than simply for translation and forwarding to the MS Mail post offices. This will allow the number of post offices to be decreased from around 50 to 5, simplify the exchange of mail/calendar information and the synchronization of directories, and provide remote sites with e-mail connectivity at acceptable speeds without the need to configure and maintain their own post offices on their own file servers.
Upgrading the system is a significant task. It involves testing and configuring the new software, distributing it to around 2,500 workstations, migrating stored messages from the old system to the new, developing new maintenance procedures, and training the end-user community to use a different interface.
The cost estimate includes software licensing for 2,500 workstations, additional mail servers, and one person-year of work:
One-time funding requested: $500,000
Annual operating costs:
Software support, maintenance and upgrades: $25,000
Staffing implications: NoneWeb-based Access to Document Databases
Document management systems serve business needs like correspondence filing, distribution and tracking, document archiving and retrieval, and document-centred workflow. The Law Department, City Clerk, Engineering, Park Board, and Community Services groups have either implemented such systems or have an identified need and are waiting for City standards to be established.
Typically the business process involves two groups; a relatively small group which creates, collects or inputs the documents, and a larger group which needs to be able simply to refer to the documents. The needs of the smaller "updater" group are being met by purchasing full-functionality document management software and hardware. However, this approach is too costly to employ for the larger "enquirer" group, which in the long term will comprise all City staff. To some extent the needs of the enquirer group are currently being served by documents stored on the City's Internet and intranet web servers, but this is time consuming and not well integrated with the work of the updater group, and generally requires that documents be converted and maintained as separate copy.
The approach that is becoming universally accepted for the "enquirer" group is a web-based interface to a document database. The documents are maintained through processes that are well integrated with the normal flow of work, but are either stored or indexed by the document management system. A software interface then provides searching, retrieval and display through the standard browser. Software licensing is targeted more towards the large "enquirer" community, making this approach both more cost-effective and more easily-implemented. The audience can be both City staff and the public.
Funding is requested for acquisition of software and hardware to implement the document management system and universal web interface. A product/vendor decision has not been made, so cost estimates are based on a representative proposal, which would deliver universal browser-based access to document databases on both the Internet (public) and intranet (staff), and a limited number (100) of web-based "updater" licenses. The cost estimate includes software licensing, a dedicated web server, consultant support for implementation, and staff training.
This will provide universal access to Council reports and minutes, both current and archival (as far back as 1989, when electronic records were first retained, in some cases). It will also provide the ability to make correspondence addressed to Mayor and Council electronically accessible anywhere in the City, subject to access restrictions. It will facilitate the implementation of future document management systems by providing them with a universal access infrastructure.
One-time funding requested: $75,000
Annual operating costs:
Software support $8,000
Staffing implications: NoneInternet and Intranet Server Upgrades
Internally, through the City's intranet, and externally, through the City's public web site, the web browser is becoming the preferred interface for delivery of both static information and interactive applications that connect the user to the City's line of business systems. Plans include:
· Using the intranet to deliver staff training for the core financial and human resource systems implementation, extending its use for delivery of electronic forms-based administrative systems, providing access to shared corporate document databases, and providing an easy-to-use interface for the infrequent user of the core systems;
· Using the public web site to provide the public with the ability to renew business licenses, apply for permits, pay parking fines and other fees, order goods for sale, request services, and communicate with staff through template-like forms.The web servers installed in 1996 were intended to serve immediate needs, which they have done. However, they will no longer be adequate for the processing demands that will be made on them, nor will they provide the reliability and availability that staff and the public expect. They must be replaced. Additionally, the volume of e-mail traffic between the City and outside has increased to the extent that it is necessary to install a mail "hub" to receive and route Internet mail, and to separate this function from the protective role of the Internet firewall. Some funding for upgrading of the intranet server is available from the core systems project. Estimated net costs (which include two replacement web servers, a "hot-standby" for high availability, software upgrades, and an allowance for overtime so that the replacement can be effected with minimum inconvenience to staff or public) are:
Total cost: $50,000
Funding available (from core systems project): ($10,000)
________
One-time funding requested: $40,000
Annual operating costs:
Hardware maintenance: (included in 3-year warranty)
Staffing implications: None* * * * *
(c) 1998 City of Vancouver