APD Data Audit Program

Release of the APD Audit Report - Cycle 1 (2007-2009)

Download the APD Audit Report-Cycle 1:  APD Audit Report-Cycle 1 (2007-2009) (821.52 kB 2010-06-10 23:57:11)

Download the Audit Report Executive Summary:  APD Audit Report_Executive Summary (118.09 kB 2010-06-10 23:58:03)

The first cycle of the APD Data Audit Program, initiated in 2007, was completed in 2009. During this 3 year cycle, 44 units from across all contributing jurisdictions were audited.

The combined results of these audits are available in the APD Data Audit Program Report - Cycle 1: 2007-2009, which can be downloaded above.The Executive Summary is also available for download above.

This report provides a picture as to the overall reliability of data held by the APD, the impact unreliable data collection may be having on reported APACHE II scores and predicted mortality, and has been produced to highlight findings that will be important for improving APD data quality in the future.

We would like to thank everyone involved in making this first cycle of the audit program a success.

Audit Program Background

The APD currently receives data from over 140 hospitals throughout Australia and New Zealand. This data represents more than 1,000,000 records. The Data Audit Program was initiated to ensure that the data being entered into the APD is reliable, given that this data is used to both measure/compare ICU performance and to answer broader research questions.

An audit pilot study was run in 2006, enabling the auditing process to be optimised and fine-tuned. The results of this study are available in 'Martin J., et al., Designing and implementing an Australian and New Zealand Intensive care data audit study. International Journal of Health Care Quality and Assurance (2009) 22:572'.

The first cycle of the audit program ran from 2007-2009, focusing on the reliability of APACHE II variables. The second cycle will run from 2010-2012, with the aim of auditing 50 sites across Australia and New Zealand. This audit cycle will focus on both APACHE II and III-J fields, along with demographic and outcome variables. 

Aims

To determine the accuracy and reliability of data collected and entered for submission to the ANZICS APD. This will be measured by the proportion of recorded observations in the database that are correct (i.e., correctness) and by the proportion of observations that are actually recorded in the database (i.e., completeness).

To examine inter-observer reliability of the Acute Physiology and Chronic Health Evaluation (APACHE) II and III-J scores and identify major causes of variability in the collection of data.

To identify any causes of insufficient data quality and to suggest/implement procedures that can improve data quality for the future.

To determine whether the degree of training in data collection influences the data quality in the database. This will be assessed by determining which types of data collectors are used in the ICUs being audited.

Design

A descriptive, comparative analysis will be undertaken.

Randomly audit 25 records per site of all adult patients admitted into each ICU during a 6-12 month period.

The patient's ICU length of stay must be greater than 24 hours.

Audit the data fields that are submitted to the central database - the minimum dataset. These include patient demographics, acute physiology and chronic health evaluation (APACHE) and diagnosis.

Carry out a structured interview with the staff members responsible for the data collection process in order to understand what types of data collectors are used within the ICU:

Dedicated: Dedicated collectors may be nurses, ward clerks or doctors. These collectors have specific training in the collection process and are solely responsible for the data being collected.

General: General collectors are personnel in the intensive care unit who have minimal training in the data collection process.

Auditors

At this stage CORE is utilising in-house auditors, however the opportunity may arise for external auditors to participate in the program. External auditors would be experienced data collectors, preferably with a clinical background, and must attend a "Training the Auditors" session.

Auditors will be assigned to sites where they are not currently involved in data collection.

All data managers interested in performing data audits will need to attend a training session to ensure that there is a streamlined approach to the auditing process.
Training sessions will include a presentation covering:

- Aims of the data audit
- Reasons for the data audit
- What is required of an auditor
- Details of auditing methods and analysis of results
- Timelines
- Reports

Training sessions will be held at ANZICS House and interstate venues as required.