Report on the Emergency Communication Transformation Program
Department of Investigation · report
At Mayor de Blasio's request, DOI investigated the Emergency Communications Transformation Program (ECTP), the City's decade-long effort to modernize its 911 system. DOI found that mismanagement between 2004 and 2013 caused the program to run years behind schedule and hundreds of millions over budget, with the capital budget rising from $1.345 billion to about $2.031 billion and completion pushed to roughly 2017. DOI found no overt criminal conduct but identified inadequate planning, ineffective governance, excessive reliance on consultants, undue pressure on staff to report positive information, no integrity monitor, understated total costs, and inconsistent recordkeeping. DOI issued seven recommendations for ECTP and future large-scale technology projects.
- Released
- 2015-02-06
- Kind
- Investigation
Key findings
- ECTP lacked a well-defined scope or vision. The City delegated scope definition to its systems integrator HP, which set forth program activities through 29 task orders and roughly 550 task order amendments, an ad hoc approach that caused frequent scope changes and missed deadlines. p. 11
- The program repeatedly missed deadlines. Originally scheduled to finish in September 2007, ECTP is projected to conclude around 2017, roughly a decade late, due to planning, staffing, and contractor performance problems including chronic software issues with the VESTA telephony software. p. 12
- The capital budget rose from $1.345 billion to about $2.031 billion, driven mainly by the decision to construct a new PSAC 2 facility in the Bronx instead of renovating an existing building and by hiring a second systems integrator (about $327 million paid to HP plus a $285 million contract with Northrop Grumman Systems Corporation). p. 12
- Governance was ineffective. No accepted central decision-making authority existed; a 2009 report indicated NYPD and FDNY had not accepted DoITT as program manager, the agencies resisted shared solutions such as a unified CAD system and a single physical network, and City Hall did not intervene to insist on a single vision. p. 13
- The City placed excessive reliance on outside consultants. It continued paying Northrop Grumman on the radio project despite an internal warning that it made no sense, incurring an 8 percent markup (about $1.4 million billed in 2013 for radio equipment), and in at least one instance multiple consultant layers produced markups of as much as 600 percent on product and service estimates. p. 15
- Program officials applied undue pressure on staff to present positive information to City Hall. Five City employees told DOI that senior management, as late as 2013, sought to sanitize, soften, or spin negative information by changing color ratings, and DOI found at least $211.4 million in ECTP-related costs were not recorded as program costs, understating total costs. p. 16
What DOI recommended
- ECTP's scope and direction must be well-defined in a written plan drafted and agreed upon by all stakeholders. p. 18
- The City must appoint a program manager empowered by the Mayor to lead large-scale technology projects such as ECTP. p. 18
- Where possible, the City should establish direct contractual relationships with vendors, avoid layers of subcontracting, and avoid ceding complete responsibility over projects to outside contractors. p. 18
- The City should set written criteria for reporting of ratings or metrics measuring program progress, and reporting should not consume so much staff time that it detracts from substantive program work. p. 18
- The City should retain an independent integrity monitor for large-scale technology projects. p. 18
- The City should account for all costs relating to large-scale technology programs like ECTP as costs of the program. p. 18
- The City should implement standardized recordkeeping practices on large-scale, multi-agency technology projects, and agencies including NYPD should create a document retention policy for ECTP and future projects. p. 18
Entities in this report
- Department of Information Technology and Telecommunications Subject
- Police Department Subject
- Fire Department of New York Subject
- Hewlett-Packard Company
- Northrop Grumman Systems Corporation
- Mayor's Office of Citywide Emergency Communications
- Gartner, Inc.
- NASA
- Motorola
- Verizon
- iXP Corp.
- Intergraph Corporation
- PlantCML
- Science Applications International Corporation