Client background: The client was a specialist insurance company in the Group and Individual Life and Disability market. The company had decided to implement a packaged software product about three years earlier for its Group Risk (Life and Disability) portfolio, and had chosen a system which had previously only been implemented for Group Pensions business.

The package selected was an integrated quotation, administration and accounting application implemented on a three tier architecture, using PowerBuilder, C, and SQL Server technology.

By March 2003, since implementation of the package in February 2002, around 7000 software defects or change requests had been logged on the Production and Test systems in the first 12 months and nearly 500 priority 1 and 2 defects remained outstanding. In addition to the quality control problems, users generally found the system complex and difficult to use, because the object oriented design did not fully reflect their business processes and error messages were frequently obscure. In many parts of the system, response time was very slow, measured in minutes rather than seconds. The overnight processing was in danger of not completing in the available elapsed time.

During the first 12 months of live operation, the company had acquired the Group Risk business of two competing insurers and moved its administrative support to a more distant location, requiring a local workforce to be recruited.

2003 development: The major package development during 2003 was the setting up of a new Quota Share Reinsurance Treaty with the clients reinsurance partner. This project involved a complex integration with existing Surplus Treaty arrangements, which had never been successfully implemented within the package when they became effective during 2002. Consequently the new project involved the catching up and calculation of up to 2 years of reinsurance premiums.

It became clear during the testing of the Reinsurance Treaty development that too many assumptions had been made about the adequacy of both the requirements and the developed conversion software from the previous abortive 2002 treaty implementation. In particular, the chosen method of retrospectively setting up the historic reinsurance premiums also caused the customer policy details to be recalculated, even though customer premiums had already been collected. For this reason, and the large number of database anomalies within the database (approximately 20%) the project never went live, and the reinsurance treaty reporting continues to run off-line using a copy of Production Data.

Besides Reinsurance, development work for two other major new product launches were undertaken in 2003, but were not delivered until 2004 partly because of delays principally caused by the Reinsurance Project. At the same time, a major initiative to introduce Automatic Renewals processing had to be postponed because of inadequate client requirements definition, and also a significant enhancement to adjustment processing had to be re-specified, in both cases after the code had ben delivered.

My role: I joined as an interim Systems Manager working for the Project Director. The incumbent manager had gone on long term sick leave. My job was to manage a team of four business analysts and two developers, who provided second level user support for the packaged product and also looked after the client's package interface, migration and document printing software. This involved the normal team leadership, resource management and individual staff appraisal functions.

One of the first things I had to do was organise the setting up of several task forces which had been created to review the different areas of the business which were badly affected by the very high number of software defects. These were: Finance, Credit Control, Renewals and Rate Review, Medical Underwriting, Overnight Processing and System Performance. Each task force had to establish underlying business problems and prioritise software defects. At the start, there were organisational disputes about the leadership of the task forces, which was eventually decided to be located with the business units affected.

During this period, my team gradually took over the full responsibility of specifying the requirements for major enhancements from the software house, using new document standards which I developed. In many cases, we had to go back and either re-specify or write for the first time requirements specifications for work which had arisen originally from other functions within the client organisation, and already partially delivered by the software house.

I also helped to stabilise the Reinsurance project by analysing what had gone wrong, and also finding a temporary solution to the need to produce reinsurance accounts by the end of the calendar year.

2004 actions taken: Early in 2004 a reorganisation of the Group Risk project took place which aimed to address the issues mentioned above and other problems experienced by the project. During the past 12 months the number of outstanding priority 1 and 2 software defects had grown from 500 to 1000. Despite the activities of the taskforces and the priority given to Finance defects, faults in the Aged Debt report in the package nearly caused adverse comments by external auditors. The client's service levels to its customers had also suffered badly, mainly because of the introduction of untested software in the early stages of the project, but also due to poor process integration and training of new staff in remote locations.

The new organisation attempts to introduce a focus on completion of the critical outstanding software defects affecting business stability, and also creates a new Process, Procedures and Training department which will apply a formal triage process to all incoming defects. Critical software and database defects with low impact will be applied in service releases, in between the major software releases which require full regression testing. Some of the outstanding development process issues not yet addressed are the introduction of more formal stages for process analysis, user interface design and database design including reviews and sign-offs.

• Maintain user morale: A large organisation very quickly passes the point when it can reverse out of a large scale project like this, however disastrous it might seem to its users. Everybody needs to learn to look forwards, not backwards
• Accept integration is tough: A highly integrated software product is far more demanding to specify, develop and test than the less functional legacy systems it replaces and it is inevitable that this will not be realised by many within the organisation
• Quantify objectives: The most important responsibility a client has is to define with precision its business objectives and quantify the improvement it expects in its business and system processes. Then it must use the criteria developed to prioritise, not dictate, the defects to be included in the next software release.
• Specify software requirements: Responsibilty for specifying requirements must be assumed fully by the client, not left to the software house. Typically, a software nouse will omit performance and usability criteria, and ignore detailed functional requirements where they think their package meets their client's needs.
• Review all design decisions: Besides poor requirements, the root cause of many logged defects is the poor design, definition and control of the database. Make sure the software house both holds design reviews and invites the client's most competent database administrators and business analysts
• Invest in testers: Poor testing in the pre-implementation and early stages of the project left a high number of defects which were both expensive to correct in Production and highly damaging to user confidence
• Invest in regression testing software: Regression testing quickly becomes a major scheduling constraint in planning releases of complex systems. Win Runner was used highly successfully to automate the testing process
• Triage help desk calls: It is critical that all incoming help desk calls are fully analysed and the correct action taken - be it user training, process change, or software change requested / defect logged
• Delivery software regularly: In a complex project recovery situation, frequent and routine software releases on scheduled dates is far more important than ensuring that particular software defects or change requests are delivered to time. This allows the organisation to evolve and not get stuck
• Final thought: Although project management methodologies like PRINCE 2 can be left standing by the evolutionary management style needed in situations like this, formal project initiation and close down procedures are worth adopting, along with formal management of risks and issues