Summary Session 13-14

Operation and Post-Implementaion (Session 13-14)

What?

A Post-Implementation Review (PIR) is an assessment and review of the completed working solution. It will be performed after a period of live running, some time after the project is completed.

Why?

There are three purposes for a Post-Implementation Review:

• To ascertain the degree of success from the project, in particular, the extent to which it met its objectives, delivered planned levels of benefit, and addressed the specific requirements as originally defined.
• To examine the efficacy of all elements of the working business solution to see if further improvements can be made to optimise the benefit delivered.
• To learn lessons from this project, lessons which can be used by the team members and by the organisation to improve future project work and solutions.

In some cases, the first of these objectives can be a contractual issue. Where that is the case, it may be safer to run separate reviews - one focused on contractual compliance and the other seeking to derive further benefit from a no-blame review.

When?

A Post-Implementation Review should be scheduled some time after the solution has been deployed. Typical periods range from 6 weeks to 6 months, depending on the type of solution and its environment. The PIR is intended to be an assessment and review of the final working solution. There should have been at least one full processing and reporting cycle completed. It should not be performed while the initial snags are still being dealt with or while users are still being trained, coached and generally getting used to its operation.

The PIR should be timed to allow the final improvements to be made in order to generate optimum benefit from the solution. There is no point in waiting too long as the results are intended to generate that final benefit for the organisation and team.

Who?

There is often a difference of opinion as to who should perform the Post-Implementation Review. Usually, members of the project team will want to complete the review as a natural extension of their responsibility to deliver optimum benefit from the solution. They understand what was required, what was changed, how it was achieved, how things are supposed to work, how to fix problems, etc.

There is a converse argument that the review should be performed by an independent team. This reduces the risk that any errors or omissions of the project team might equally be overlooked in their review.

A solution is to do both. An independent audit team, working in consultation with the business users and project team, could examine whether the results are satisfactory. The project team might then reconvene to consider that input and also to examine how to generate further value from the solution.

How?

A list of points should be drawn up to cover all elements of the operational solution. They should include such things as:

Current situation

• Is the required functionality available?
• Are the procedures properly documented, published and known about?
• Have users received adequate training and coaching to take advantage of the new facilities?
• Are staffing levels and skillsets appropriate for the actual workloads?
• Are staff displaying appropriate attitudes to get the best out of the system (confidence in its capabilities, belief in its purpose, willingness to make it work, etc)?
• How busy, usable, useful and adequate are support services such as the systems support function and help desk?
• Are third parties such as customers and suppliers satisfied with the service?
• Is the level and nature of identified faults acceptable?
• Are faults handled at an acceptable speed and with satisfactory results?
• Is data integrity being maintained within the system and in relation to other integrated or interfaced systems?
• Are systems controls being applied correctly?
• Are business, procedural and financial controls being applied correctly?
• Does the system and its usage meet current legal and regulatory requirements?
• Is the system able to process transactions at an adequate speed?
• Does the system have the capacity to deal with the actual peak loadings as encountered and foreseen?
• Are staff following operational procedures including backup, recovery, security and disaster recovery?
• Has the project been properly demobilised, eg documentation filed, team members appraised and reassigned, equipment and facilities returned, final accounting and reporting completed, success and completion communicated?

Benefits

• What were the final costs of the project?
• What is the actual operating cost of the new solution?
• What is the actual benefit being delivered by the new solution?
• How does that compare to the original project definition?

Future improvements

• Could further training or coaching improve the degree of benefit being generated?
• Are there further functional improvements or changes that would deliver greater benefit?
• Are specific improvements required in procedures, documentation, support, etc?

What learning points are there for future projects?

These questions will be investigated through a combination of investigative techniques including interviews, examination of documentation, performance statistics, hands-on tests and checks, etc. Implications and potential remedial options would then be assessed and evaluated. The findings and recommended actions would be prepared, normally in the form of a report or presentation.

Next Steps

The findings and recommendations will be presented to:

• the solution's business owners,
• the leading participants in the project, and
• other parties who may be concerned with the results.

Specific actions should be proposed to address any further work that is recommended. This might be handled in several different ways, for example:

• as routine support and maintenance,
• as remedial work to be performed by the original project team,
• for line management to address through user education and procedures etc,
• as further phases of development involving new projects.