In March 1999, the Department of Health and Senior Services published its second coronary artery bypass graft (CABG) surgery report. The report contained information about the risk-adjusted mortality rates for both hospitals and surgeons in New Jersey using 1996-97 patient data. This document is the third release of the CABG report and is based on 1998 data. Most of the basic information related to bypass surgery is, therefore, the same as in the two previous reports.
Coronary heart disease (also known as coronary artery disease or ischemic heart disease) occurs when the coronary arteries, which carry blood to the heart, are clogged with fatty deposits on the artery walls. This can lead to angina (chest pain) that is caused by oxygen deficiency, particularly when the arteries are closed by 50 percent or more. Furthermore, angina is a warning sign for heart attack, when a coronary artery that carries blood to the heart is occluded or blocked. This leads to starvation of a portion of the heart muscle and an area of heart tissue that is no longer alive.
According to the American Heart Association, almost 14 million Americans have coronary artery disease. Coronary artery disease and other cardiovascular diseases claim one million lives in the United States each year. This represents 41.5 percent of all deaths, or one in every 2.4 deaths.
One of the primary treatments for coronary heart disease is coronary artery bypass graft (CABG) surgery. These operations are performed by using segments of veins or arteries from other parts of the patient's body to create a detour around the blocked portion of the patient's coronary artery. In 1996, an estimated 578,000 people in the nation underwent CABG surgery. In New Jersey, a total of 8,377 people underwent isolated CABG surgery (CABG surgery with no other major heart surgery during the same admission) in 1998; these people, and the hospitals in which they underwent surgery are the subject of this report.
New Jersey's Quality Improvement Program for CABG Surgery
Under the expert guidance of the Cardiovascular Health Advisory Panel (CHAP), the New Jersey Department of Health and Senior Services has initiated a program under which the 14 hospitals in the State that have the Department's approval to perform coronary artery bypass graft (CABG) surgery, submit detailed information to the Department for analysis and feedback to hospitals, surgeons and the public. Information that is collected for each patient undergoing CABG surgery in New Jersey includes demographics, pre-operative risk factors, complications that occur during or after surgery, and discharge status (died in the hospital or discharged alive). These data have been collected since 1993, and advice has been sought throughout the process from the Cardiovascular Health Advisory Panel, the clinical community and outcome analysis experts.
Analyses consisted of identifying the pre-operative risk factors that were significantly associated with patients' chances of being discharged alive, and using statistical methods to develop a formula for predicting each patient's probability of in-hospital mortality based on those risk factors. This information was then used to assess the average pre-operative severity of illness for each hospital's patients. The next step consisted of using the mortality rate for each hospital's patients in conjunction with their average severity of illness to arrive at a "risk-adjusted mortality rate" that serves as a measure of quality of care. This report provides risk-adjusted mortality rates for 1998 for all hospitals in which CABG surgery is performed.
The release of this information to hospitals and the public in New York and Pennsylvania has led to numerous hospital quality improvement initiatives and significant decreases in mortality rates in those states, and the Department is confident that this and subsequent releases in New Jersey will and have had the same effect. For instance, several New Jersey hospitals are collaborating in a "best practice" program. Another reason for the report is to enable consumers and potential patients to become more informed about their options and to be aware of patient outcomes associated with CABG surgery providers (hospitals and surgeons).
The following is a description of the patient population and the process for calculating risk-adjusted mortality rates for hospitals. The statistical model that is used, along with all significant pre-operative risk factors, is also presented. The last section presents risk-adjusted mortality rates for hospitals in New Jersey for 1998.
The patients represented in this report are the 8,377 patients who underwent isolated coronary artery bypass graft (CABG) surgery (surgery with no other major heart surgery during the same admission) in New Jersey in 1998. As indicated earlier, these operations are performed by using segments of veins or arteries from other parts of the patient's body to create a detour around the blocked portion of the patient's coronary artery. The total number of these patients who died during or after surgery in the same admission was 218 and the statewide in-hospital mortality rate was 2.60 percent. This population includes Minimally Invasive Coronary Artery Bypass (MIDCAB) procedures which were first performed in New Jersey in 1996.
The Commissioner's Clinical Review Panel reviewed cases submitted by hospitals for exclusion as salvage cases and patients who appear to have died from complications of a second unrelated operation performed within the index hospitalization after full recovery from a successful cardiac operation. As a matter of public health policy, the Panel recommended removal of some cases from the report so as not to unduly harm hospitals where such patients are treated.
Rationale for using 1998 Data
Data for the year 1998 were used for this study because they represent the most recent data available. In the two previous reports the most recent two-year periods were used as the time interval for the report. Prior to 1998, roughly one-half of the hospitals used one data entry system and the other half used another data entry system for collecting data. Because these two data systems resulted in inconsistent definitions for several of the data elements, those data elements could not be included as candidates in the statistical model that was developed. However, in 1998 all hospitals used a single system (the Society of Thoracic Surgeons system) so all of the data elements can be included as candidates in the statistical model. Since several data elements that were candidates for the 1998 statistical model were not available in 1997, it would be difficult to combine data from the two years for the purposes of assessing hospital performance. Thus, the 1998 database contains more data elements that can be used in the risk-adjustment process than the 1997 database, and some of these data elements prove to be significant predictors of mortality. After consulting with the Clinical Review Panel, the decision was made to release 1998 data for this report.
Assessing Hospital Performance For CABG Surgery By Calculating Risk-Adjusted Mortality Rates
Hospital performance for CABG surgery is assessed by comparing risk-adjusted patient outcomes in each hospital with outcomes throughout the state. Ideally, "patient outcomes" include such things as whether the patient died during or after surgery, what complications of surgery they suffered, their long-term survival, and their satisfaction with the care they received. This report concentrates on in-hospital mortality, which is arguably the most important outcome and also easiest to collect.
In-hospital mortality is affected by the pre-operative severity of illness of CABG surgery patients, as well as by the quality of surgery and hospital care that patients receive. Consequently, if some hospitals treat sicker patients, they will have higher mortality rates even if their quality of care is comparable to hospitals with patients who are not as sick. Thus, it is important to try to adjust for differences in patient severity of illness when reporting outcomes. The following is a description of how this is done when reporting hospital outcomes.
Surgeon outcomes are not released in this report. Since only one year's worth of data was used, many surgeons did not have the high enough volume of cases needed to yield meaningful risk- adjusted rates with small confidence intervals.
Data Collection and Data Quality Audit
All 14 hospitals in New Jersey in which cardiac surgery is performed collect detailed patient-specific information related to the patient's demographics, pre-operative risk factors, complications of surgery and discharge status (died in the hospital, discharged alive). This process began in 1993 with a pilot data collection from a few hospitals. Full participation by all 13 cardiac centers at the time began in 1994. In January 1998, Saint Francis Medical Center of Trenton began performing cardiac surgery. This is the first report to include Saint Francis Medical Center, which brings the total number of the state's cardiac centers to 14.
Data submitted to the Department was verified by an independent auditor by comparing a sample of cases against medical records. When discrepancies were found between the data and the medical records, the hospitals were asked to produce corroborating medical record documentation of their coding or to recode the cases. A verification against hospital discharge records was also conducted to assure that all cases and deaths were reported. Error trapping and frequency reports were also run several times throughout the data cleaning process and shared with the hospitals for correction. Hospitals provided final verification and sign-off of data prior to the final model being run.
Computing the Observed Mortality Rates for Hospitals
The observed mortality rate for each hospital can be easily computed by dividing the number of patients who died in the hospital during or after CABG surgery by the number of patients who underwent CABG surgery (the number who died in the hospital plus the number who were discharged alive). Unfortunately, this number can be very misleading as a measure of hospital performance because it does not account for how sick the patients were prior to surgery. If one hospital had considerably sicker patients than another hospital, it would be expected that its mortality rate would be somewhat higher. Consequently, a fairer way of comparing the performance of the two hospitals would take into account both their observed mortality rates and a measure of how sick their patients were prior to surgery.
Computing the Predicted Mortality Rates for Hospitals
The measure of how seriously ill each CABG patient is prior to his/her surgery is called the predicted probability of death, and represents the chance that the patient will die during or after surgery but before discharge from the hospital. The predicted probability of death is obtained using a statistical model (called logistic regression) that identifies which of a group of proposed patient risk factors are significantly related to a patient's chance of dying during or after CABG surgery, and then assigns statistical weights to those risk factors. The weights are used in a formula that generates the patient's predicted probability of death.
If a patient's predicted probability of death is 0.05, this means that it is estimated that there are five chances in 100 (or 1 chance in 20) that the patient will die in the hospital prior to discharge.
The estimate of the average risk for all of a hospital's patients is obtained by summing the predicted probabilities of death for all of the patients and dividing by the number of patients. This number, which is called the hospital predicted or expected mortality rate, is an estimate of what the hospital's mortality rate would have been if the hospital's performance had been identical to the statewide performance for those patients.
The statistical methods used to predict mortality on the basis of the significant risk factors are tested to determine if they are sufficiently accurate in predicting mortality for patients who are extremely ill prior to undergoing the procedure, as well as for patients who are relatively healthy. These tests have confirmed that the model is reasonably accurate in predicting how patients of all different risk levels will fare when undergoing coronary bypass surgery.
Computing the Risk-Adjusted Mortality Rate
Hospital performance is assessed by comparing what actually happened (the observed mortality rate) with what was predicted to happen based on how severely ill the hospital's patients were (the expected mortality rate). First, the observed mortality rate is divided by the hospital's expected mortality rate. If the resulting ratio is larger than one, the hospital has a higher mortality rate than expected on the basis of its patient mix; if it is smaller than one, the hospital has a lower mortality rate than expected from its patient mix. The ratio is then multiplied by the statewide mortality rate of 1998 (2.60%) to obtain the hospital's risk-adjusted mortality rate.
The risk-adjusted mortality rate represents the best estimate, based on the associated statistical model, of what the hospital's mortality rate would have been if the hospital had a mix of patients identical to the statewide mix. Thus, the risk-adjusted mortality rate has, to the extent possible, ironed out differences among hospital's inpatient severity of illness, since it arrives at a mortality rate for each hospital on an identical group of patients.
Interpreting the Risk-Adjusted Mortality Rate
If the risk-adjusted mortality rate is lower than the statewide mortality rate, the hospital has a better performance than the state as a whole; if the risk-adjusted mortality rate is higher than the statewide mortality rate, the hospital has a poorer performance than the state as a whole.
The risk-adjusted mortality rate is used in this report as a measure of quality of care provided by hospitals. However, there are reasons that a hospital's risk-adjusted mortality rate may not be indicative of the quality of care being provided.
For example, extreme outcome rates may occur due to chance alone. This is particularly true for low-volume hospitals, for whom very high or very low mortality rates are more likely to occur than for high-volume hospitals. In order to minimize misinterpretation due to chance variation, coronary bypass surgeon data have not been reported for 1998. Another attempt to prevent misinterpretation of differences caused by chance variation is the use of expected ranges (confidence intervals) in the reported results. The interpretations of those terms are provided later when the data are presented.
Differences in hospital coding of risk factors could be an additional reason that a provider's risk-adjusted rate may not be reflective of quality of care. However, the Department of Health and Senior Services monitors the quality of coded data by reviewing patients' medical records to ascertain the presence of key risk factors. The Department also contracts with an independent audit firm to verify hospital data.
Another reason that risk-adjusted rates may be misleading is that overall pre-procedural severity of illness may not be accurately estimated because important risk factors are missing. This is not considered to be an important factor, however, because the New Jersey system does contain a large number of risk factors that have been demonstrated to be related to patient mortality in various national and international studies. Despite the possible limitations in overall predictive power, statistical tests do indicate that the model is a reasonable predictor of in-hospital mortality for New Jersey patients.
A final reason why these data may not provide a definitive measure of provider quality care is that patient mortality is not the only way of measuring quality; for instance, complications of surgery, patient quality of life following surgery, and patient satisfaction are also important markers of quality. Although the risk-adjusted mortality rates presented here may not be a totally accurate depiction of the quality of care, the New Jersey Department of Health and Senior Services believes that this information is a valuable aid in choosing providers for coronary artery bypass graft surgery. Perhaps even more importantly, the Department feels that the information provided here can serve as a guide and an impetus for hospitals to improve the quality of care they provide to CABG surgery patients.
1998 Risk Factors for CABG Surgery
The significant pre-operative risk factors for CABG surgery in 1998 in New Jersey are presented in Table 1, along with coefficients for the statistical model, p-values, and odds ratios. The coefficients can be used to compute a given patient's probability of death given the patient's risk factors. The p-values state the level of significance for each of the risk factors in Table 1. Note that the smaller the p-value, the more significant the risk factor is as a predictor of in-hospital mortality. Also, note that the only risk factors used in the statistical model were ones that were highly significant predictors (i.e. had p-values of .05 or smaller).
For all risk factors except age, ejection fraction and renal failure, the odds ratios represent the odds of a patient with the risk factor dying in the hospital divided by the odds of a patient without the risk factor dying in the hospital. Roughly speaking, this is the number of times more likely a patient with the risk factor is to die in the hospital than a patient without the risk factor, all other risk factors being identical. For example, the odd ratio for congestive heart failure is 1.611. This means that a CABG surgery patient with congestive heart failure has odds of dying in the hospital during or after surgery that are 1.611 times the odds of a patient without congestive heart failure, assuming the two patients are identical with respect to the other risk factors presented in Table 1.
For age, the odds ratio represents the number of times more likely a patient in a certain age group is to die in the hospital than a patient whose age is below 65 years. For example, a CABG patient between 75 and 79 years of age has odds of dying in the hospital that are 2.964 times the odds of a patient who is less than 65 years old with the same other risk factors.
Similarly, for ejection fraction, the odds ratios are relative to the reference group of patients, which are the patients with an ejection fraction that is 50 percent or greater or has not been reported. Thus, for example, the odds of a patient with an ejection fraction of 29 percent or less, dying in the hospital are 2.581 times the odds of a patient with an ejection fraction 50 percent or greater dying in the hospital. For renal failure, the odds ratio are relative to the reference group of patients without renal failure. Thus, the odds of a patient with renal failure on dialysis dying in the hospital are 7.600 times the odds of a patient without renal failure dying in the hospital, all other risk factors being the same.
1998 Hospital Outcomes for CABG Surgery
Table 2 presents the 1998 isolated CABG surgery results for the 14 hospitals in which this procedure was performed in New Jersey. For each hospital, the table contains the number of isolated CABG operations performed in 1998, the number of in-hospital deaths, the observed mortality rate, the expected mortality rate based on the statistical model presented in Table 1, the risk-adjusted mortality rate, and a 95 percent confidence interval for the risk-adjusted rate.
Confidence intervals for the risk-adjusted mortality rate indicate which hospitals had significantly more or fewer deaths than expected given the risk factors of their patients. Hospitals with significantly higher rates than expected after adjusting for risk are those with confidence intervals entirely above the statewide rate. Conversely, hospitals with significantly lower rates than expected, given the severity of illness of their patients before surgery, have confidence intervals entirely below the statewide rate. It should be noted that, in general, hospitals with higher volumes have smaller confidence intervals than hospitals with lower volumes.
As indicated in Table 2, the overall mortality rate for the 8,377 CABG operations performed at the 14 hospitals was 2.60 percent. Observed mortality rates ranged from 0 percent to 6.76 percent. The range in expected mortality rates, which measure patient severity of illness, was from 2.13 percent to 3.64 percent.
The risk-adjusted mortality rates, which are used to measure performance, ranged from 0 percent to 6.32 percent. Two hospitals, Saint Francis Medical Center in Trenton and Morristown Memorial, had risk-adjusted mortality rates that were significantly lower than the statewide rate. Two hospitals, Newark Beth Israel Medical Center and Cooper Hospital/University Medical Center, had significantly higher risk-adjusted mortality rates than the statewide average.
It is important to note that the statistical techniques used have been designed to compare the patient mortality rate associated with a given hospital with the patient mortality of the aggregate of all other hospitals in New Jersey. Therefore, this method does not allow for direct hospital-to-hospital comparisons.
As pointed out earlier in this report, surgeon volume is not reported in this release. In switching to the standardized reporting system, the Department decided to limit this report to one year's data. For each hospital, there are enough cases in one year for results to be statistically meaningful. This is not the case for the individual surgeons and, therefore, results in a wider confidence interval which would make the interpretation of the outcome statistically unreliable. Next year, the Department plans to release two years of data (1998 and 1999) with both hospital and surgeon outcomes.
1996-1998 Statewide Data
Table 3 is derived from the results of a statistical model based on New Jersey CABG surgery data from three years: 1996-1998. The table presents, for each of the three years, the observed, expected, and risk-adjusted in-hospital mortality rates for all patients undergoing isolated CABG surgery in New Jersey.
As shown in Table 3, the volume of isolated CABG procedures rose in each of the one-year periods. The volume rose very slightly from 8,262 in 1996 to 8,286 in 1997, an increase of less than one percent and by 1.1 percent from 1997 to 1998. The observed mortality rate declined from 3.66 percent in 1996 to 3.08 percent in 1997, and then dropped further to 2.60 percent in 1998. The expected rate rose in each of the one-year periods, with a low of 3.04 percent in 1996 and a high of 3.18 percent in 1998. This rise from 1996 to 1998 represents an increase of 4.6 percent. This increase is not surprising in view of the increase in the number of patients with coronary artery disease who have been undergoing percutaneous coronary transluminal angioplasty, which is an alternative to CABG surgery among lower risk candidates for CABG surgery.
The risk-adjusted mortality rate declined from 3.74 percent in 1996 to 2.55 percent in 1998. The drop represents a total decrease of almost 32 percent.
It is especially notable that the risk-adjusted mortality rate in 1998 (2.55%) was significantly lower than the three-year mortality rate for 1996-1998 (3.11%). This indicates that we continue to show marked improvement in CABG surgery outcomes in New Jersey.
|Table 1: Multivariable Risk Factor Equation for Isolated CABG Hospital Deaths in New Jersey in 1998.|
|Patient Risk Factor||Coefficient||P-Value||Odds Ratio|
|Ages 65 - 69||0.5829||0.0126||1.791|
|Ages 70 - 74||0.7326||0.0012||2.080|
|Ages 75 - 79||1.0865||0.0001||2.964|
|Ages 80 - 84||1.2792||0.0001||3.594|
|Ages 85 and Over||1.4551||0.0004||4.285|
|Congestive Heart Failure||0.4766||0.0037||1.611|
|Renal Failure with Dialysis||2.0282||0.0001||7.600|
|Renal Failure without Dialysis||1.3886||0.0001||4.009|
|Ejection Fraction 1% - 29%||0.9480||0.0001||2.581|
|Ejection Fraction 30% - 49%||0.4413||0.0079||1.555|
|Previous Open Heart Surgery||1.0591||0.0001||2.884|
|Triple Vessel Disease||0.3562||0.0472||1.428|
|Source: New Jersey Open Heart Surgery Database, 1998.|
|Table 2: Hospital Observed, Expected and Risk-Adjusted Mortality Rates for Isolated CABG Surgeries in New Jersey, 1998 (Listed Alphabetically by Hospital).|
|Hospital||Cases||Deaths||OMR||EMR||RAMR||95% CI for RAMR|
|Cooper Hospital/University M.C.||273||12||4.40||2.23||5.12||+||(||2.64||,||8.95||)|
|Deborah Heart and Lung Center||782||18||2.30||2.68||2.24||(||1.33||,||3.54||)|
|General Hospital Center at Passaic||457||9||1.97||2.32||2.21||(||1.01||,||4.20||)|
|Hackensack University Medical Center||788||21||2.66||3.16||2.20||(||1.36||,||3.36||)|
|Jersey Shore Medical Center||515||9||1.75||2.56||1.77||(||0.81||,||3.37||)|
|Morristown Memorial Hospital||1,177||15||1.27||2.13||1.56||-||(||0.87||,||2.57||)|
|Newark Beth Israel Medical Center||414||28||6.76||2.79||6.32||+||(||4.20||,||9.13||)|
|Our Lady of Lourdes Medical Center||942||31||3.29||2.37||3.61||(||2.45||,||5.13||)|
|Robert Wood Johnson University Hosp.||1,054||25||2.37||2.89||2.13||(||1.38||,||3.15||)|
|St. Francis Medical Center||238||0||0.00||3.64||0.00||-||(||0.00||,||1.10||)|
|St. Joseph's Hospital/Medical Center||460||18||3.91||3.08||3.31||(||1.96||,||5.23||)|
|St. Michael's Medical Center||481||11||2.29||2.31||2.58||(||1.28||,||4.61||)|
|OMR: Observed mortality rate.|
|EMR: Expected mortality rate.|
|RAMR: Risk-adjusted mortality rate; RAMR = (OMR/EMR)*Statewide OMR.|
|+ Risk-adjusted mortality rate significantly higher than statewide rate based on 95 percent confidence interval.|
|- Risk-adjusted mortality rate significantly lower than statewide rate based on 95 percent confidence interval.|
|Source: New Jersey Open Heart Surgery Database, 1998.|
After Coronary Artery Bypass Surgery, 1996-1998
**Significantly lower than three-year average mortality rate based on 95% confidence interval.