Southern trusts hit by CRS failure

24 Apr 2008

The Millennium NHS Care Records System application, meant to become the critical clinical system used by NHS staff across Southern England, went down and stayed down for an hour Thursday morning, 24 April in three out of the eight trusts where it has been installed.

Though currently limited to patient administration tasks later versions of the Millennium CRS application -provided by Fujitsu from a central data centre - will include far more clinical tools. This would potentially making future region-wide failures hugely disruptive to patient care.

The Cerner Millennium application has so far been installed at eight NHS trusts by Fujitsu under the £12.4bn NHS IT project. By 2010 it is meant to be used by all NHS organisations in the South. The same American software is also being provided to the NHS in London by BT.

Fujitsu is currently locked in what appear to be fruitless ‘contract reset’ renegotiations with the NHS in the South of England. Thursday’s failure will not strengthen either Fujitsu or Cerner’s position.

E-Health Insider learned of the problem through frustrated NHS staff in the region. Fujitsu later acknowledged the outage, stating:“Failure of the primary node caused a system failure this morning however Fujitsu successfully failed over to the secondary node within our service level agreements with NHS Connecting for Health and the NHS.

"The trusts resorted to their business continuity plans/process which are set up and validated by the trusts. It affected Domain 1 which consists of Milton Keynes Hospitals NHS Trust; Nuffield Orthopaedic Centre NHS Trust; Buckinghamshire Hospitals NHS Trust and Buckinghamshire PCT.

"Fujitsu is investigating the situation. Once the route cause has been established Fujitsu will review further and develop actions to eradicate a repeat performance. Fujitsu continually reviews and monitors performance.”

A sign that the the Millennium CRS application has yet to become critical to patient care in some NHS sites installed so far came from the fact that almost none of the trusts appeared aware of the problem. Three clinical contacts said they were unaware of the problem.

A further indication that NHS trusts appear to have high tolerance levels for Cerner CRS problems came from the fact that of the eight trusts live with the system not one of the press officers said they were aware of a problem.

A CfH spokesperson told EHI: "We can confirm that there was an incident from 9:15 until 9:59 today affecting users of Cerner Millennium in three trusts and a diagnostic centre within the South Central SHA.

"It is usual for local trusts to have contingency processes for these types of incidents, to mitigate against any risk to patients."

Reader's Comments

Better than CSC?

25 Apr 08 10:12

At any rate the system (? after an hour?- I thought it was supposed to happen automatically and instantaneously.) "failed over" to its secondary node.. Different from CSC's experiences last year! Good thing both are introducing what appear to be somewhat basic administrative systems at present - but what *does* happen when they are running ITU applications? Should business contingency plans include local servers?

CRS failure ... or CRS failover?

25 Apr 08 11:11

Can we have some clarification of the facts with this story please?

"E-Health Insider learned of the problem through frustrated NHS staff in the region"

"Fujitsu later acknowledged the outage stating 'Failure of the primary node caused a system failure this morning however Fujitsu successfully failed over to the secondary node within our service level agreements ...' "

"Almost none of the Trusts appeared aware of the problem."

"Three clinical contacts said they were unaware of the problem."

"Not one of the [eight Trusts'] press officers were aware of a problem".

So did the failover to the secondary node mean that there was no disruption to service to users? If so, this surely demonstrates positive aspects of CRS

Failure indeed

28 Apr 08 16:01

That the clinicians were oblivious shows how NCRS has failed to become an essential part of their working life.

That the Press Officers were unaware could just mean they didn't have a response ready.

That the system to be down for an hour before failover is 'within the SLA tolerances' suggests that the contract does not protect the users, and that reliance on these remote datacentres would not be sensible for patient critical care. Or are we to just accept that FJA and CfH appear to only be delivering poor quality Patient Administration Systems, and that is OK then ?