Translate this page into:
Concordance in the Assessment of Effectiveness of Palliative Care between Patients and Palliative Care Nurses in Malaysia: A Study with the Palliative Care Outcome Scale
Address for correspondence: Prof. Kwee Choy Koh, Department of Medicine, Clinical Campus, International Medical University, Jalan Rasah, 70400 Seremban, Negeri Sembilan, Malaysia. E-mail: kweechoy_koh@imu.edu.my
This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as the author is credited and the new creations are licensed under the identical terms.
This article was originally published by Medknow Publications & Media Pvt Ltd and was migrated to Scientific Scholar after the change of Publisher.
Abstract
Context:
The Palliative Care Outcome Scale (POS) is an easy-to-use assessment tool to evaluate the effectiveness of palliative care. There is no published literature on the use of POS as an assessment tool in Malaysia.
Aim:
To define the concordance in the assessment of quality of life between patients with advanced cancers and their palliative care nurses using a Malay version of the POS.
Settings and Design:
This study was conducted in the palliative care unit of the Hospital Tuanku Ja'afar Seremban, Malaysia, from February 2014 to June 2014.
Subjects and Methods:
We adapted and validated the English version of the 3-day recall POS into Malay and used it to define the concordance in the assessment of quality of life between patients and palliative care nurses. Forty patients with advanced stage cancers and forty palliative care nurses completed the Malay POS questionnaire.
Statistical Analysis Used:
The kappa statistical test was used to assess the agreement between patients and their palliative care nurses.
Results:
Slight to fair concordance was found in all items, except for one item (family anxiety) where there was no agreement.
Conclusions:
The Malay version of the POS was well accepted and reliable as an assessment tool for evaluation of the effectiveness of palliative care in Malaysia. Slight to fair concordance was shown between the patients and their palliative care nurses, suggesting the needs for more training of the nurses.
Keywords
Assessment tool
Malay
palliative care
Palliative Care Outcome Scale
INTRODUCTION
Palliative care is an approach that improves the quality of life of patients and their families associated with life-threatening illness, through the prevention and relief of suffering by means of identification and treatment of pain and other problems. It enhances quality of life, positively influences the course of illness, is applicable early in conjunction with other therapies that are intended to prolong life such as chemotherapy or radiation therapy, and includes investigations needed to better understand and manage distressing sufferings.[1]
Many tools are available for evaluation of the effectiveness of palliative care. One such tool is the Palliative Care Outcome Scale (POS) which is a 10-item multidimensional questionnaire designed to assess the physical, psychosocial, spiritual, and practical aspects of patients with metastatic cancers. Nine items in the questionnaire are scored from “0” (best) to “4” (worst) while the last question allows the patient to document their main concerns. The POS can be completed within 10 min.[23] To date, the POS has been translated into more than 12 languages and used extensively not only in patients with advanced cancers but also in patients with HIV/AIDS and patients with neurological diseases.[45678]
There is no published literature in Malaysia on studies using POS as an assessment tool. Only the Hospice Quality of Life Index and the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire C30 (EORTC-QLQ-C30) have been validated and used to assess symptomology and quality of life in patients with advanced cancers.
We describe the cross-cultural adaptation and validation of the translated POS into Malay and its use as an assessment tool in patients with advanced stage cancers.
SUBJECTS AND METHODS
Translation of Palliative Care Outcome Scale to target language
Before commencement of the study, the POS questionnaire was translated into the Malay language as per the methodology stated by the POS development team. There are six phases in the process: Conceptual definition, forward translation, blind backward translation, expert review, cognitive debriefing, and psychometric testing.[9]
Phase I: Conceptual definition
A brief review of the literature on the health-related quality of life issues in palliative care patients with advanced cancer was done, followed by identification and definition of key concepts on each item of the POS questionnaire with selected groups comprising palliative care patients and medical professionals. The detailed documentation and calculations of conceptual definition are shown in Table 1.
Phase II: Forward translation of the Palliative Care Outcome Scale from English to Malay
Forward translation of the original POS questionnaire required three translators and two main steps. First, the questionnaire was translated into the target language, which is Malay, by the translators who had agreed to participate. They were required to complete the work independently without any discussion among them and no time limit was given. In this particular step, we have gotten the consents of a medical officer working in a peripheral health clinic and a nonmedical profession who was an administration assistant. Both translators were bilingual and came from different backgrounds. This allowed for the best outcomes of the translation with its natural, cultural nuances of the language. To create an audit trail of the adaption and translation process and to smoothen the process of forward translation, a copy of the original POS questionnaire was given to each of the translators. They were requested to document their translational work on a piece of paper in their own free time independently. They were required to record any difficulties or uncertainties they faced throughout the process for further discussion to arrive at a consensus. Once the translations by these two translators were completed, a third translator mediated the differences between the two Malay versions through discussions and produced a final Malay POS version agreed by consensus.
Phase III: (Blind) backward translation from Malay back to English
Two translators worked independently to translate the consensus Malay POS version back to English. One was a gynecologist while the other was a journalist. English was their mother tongue.
They were required to complete it independently without any discussion carried out among themselves. Any difficulties or uncertainties they faced throughout the translation process were required to be recorded for further discussion. A third person was then involved to synthesize both back translations into one and to compare the translation differences alongside with the original POS questionnaire.
Phase IV: Expert review
The purpose of expert review committee was to review, discuss, and resolve any uncertainties between the two backward translations and to produce a prefinal translation of POS in Malay. This was performed by a medical personnel and a nonmedical personnel. However, both these translators were different from the translators involved in the forward translation. The aim of backward translate was to ensure that the newly translated Malay version was understandable by someone who was blinded from the original version and to serve the intention of checking the validity of the Malay version to accurately reflect the original content of the POS questionnaire.
The composition of the expert committee was important to achieve a cross-culture equivalence of the translated instrument. In order for this translated version to be validated and to eliminate any probable ambiguity, an Expert Review Committee was formed by a group of people from various disciplines comprising the team leader from the research team, an oncologist who was a content expert and familiar with POS construct, and all four translators involved throughout the process. A face-to-face meeting was carried out at the International Medical University (IMU) to discuss all the uncertainties recorded. The records of uncertainties from previous phases were made available during the meeting. One prefinal version of POS questionnaire in Malay version was produced at the end of the meeting. All challenging words or phrases and the solutions to the discrepancy by discussion were documented. We did not find gross or glaring differences between the final backward translation version and the original version of POS questionnaire.
Phase V: Cognitive debriefing
The field test of the prefinal Malay version was conducted with patients diagnosed with stage IV metastatic cancer and health-care providers at the Hospital Tuanku Ja'afar Seremban (HTJS) by enlisting five patients and five health-care providers. We had arrived at this target number to ensure that a wide range of responses may be obtained to produce a better-finalized version of POS questionnaire in Malay. Each of them was first requested to complete the questionnaire given and to point out any sentences that were baffling. These participants were then interviewed during which the interviewers would clarify the difficulties faced by the participants when answering the questions. The interviewers explored further by inquiring about the participants' thoughts regarding the questionnaire question by question, including asking for suggestions as to how to rephrase certain questions to facilitate comprehension.
The chosen participants used Malay as their native language or were at least able to speak or converse fluently in Malay. Inclusion and exclusion criteria were formulated to assess the quality of the prefinalized version. The inclusion criteria were metastatic cancer patients, fluent in Malay, and resided in the country, while the exclusion criteria were patients with a cognitive or communicative impairment, or a physical limitation, or an expectation of death within days. The health-care providers selected to participate in this process were nurses who were in charge of the respective metastatic cancer patient. These nurses had extensive experience in palliative care and had been actively working in settings with palliative care and knew the patients well.
Out of the five patients, three of them, who were native Malays, did not face any major disputes in understanding the questions and were able to answer the survey on their own within 8 min. The other two patients needed some clarifications on the meaning and extent of the answer to the questionnaire. Before the pretesting, the interviewers had met to standardize the meaning of respective answers to prevent bias and to ensure accuracy of the results. None of the nurses faced any problems in answering the questionnaire.
Phase VI: Psychometric testing
This phase included testing the translated POS for internal consistency reliability, test-retest reliability, construct validity, criterion validity, and responsiveness to change. The test for internal consistency reliability is described below.
Study design and setting
Our study was conducted at the HTJS, Negeri Sembilan, Malaysia, from February 2014 to June 2014. The HTJS is an 800-bedded tertiary hospital government hospital with 17 clinical specialties, including palliative care, in the state of Negeri Sembilan.
The sample size required for statistical significance using the Cohen's kappa statistical test was forty patients and forty palliative care nurses. Forty patients were randomly selected by their palliative care nurses to complete the Malay POS. The inclusion criteria for patients were patients above the age of 18 years with advanced cancers who were able to provide consent and were well versed in Malay. The palliative care nurses were the personnel taking care of these patients.
Survey tool
The translated Malay version of the POS was used as the survey tool in our study. Informed consent was obtained from both the patients and the palliative care nurses before they were surveyed. Both patients' and palliative care nurses' versions of the POS were the same for all 10-items in the POS questionnaire.
Ethical consideration
Approval for our study was obtained from the IMU's Joint-Committee on Research and Ethics (IMU New Research Project Number: CSc/Sem6(39)2013).
Statistical analyses
The Cohen's kappa statistical test was used to assess agreement between the responses of the patients and palliative care nurses.[1011] All statistical analyses were performed using the IBM Corp. Released 2010. IBM SPSS Statistics for Windows, Version 19.0. Armonk, NY: IBM Corp.
RESULTS
The sociodemographic characteristics of the patients in this study are tabulated in Table 2. The mean age was 62.8 years (median 64 years; range 23–84 years). Most of the patients were married and had a monthly income of less than MYR 499, 69% and 70%, respectively. The most common cancers were breast cancer (27%), colorectal cancer (27%), and lung cancer (14%).
The Cohen's kappa was used to assess inter-rater reliability between patients and their palliative care nurses [Table 3]. The agreement between patients and palliative care nurses for the ten items in the POS is tabulated in Table 4. There were “slight” or “fair” agreements between the responses of patients and their palliative care nurses in all the items in the POS, except for item number 4: “Have any of your family or friends been anxious or worried about you?” where there was no agreement.
DISCUSSION
There was only “slight” agreement between patients and palliative care nurses in this study in majority of the items on the POS such as “other symptoms” (item 2), information given (5), life worthwhile (item 7), self-worth (item 8), and “time wasted on appointments” (item 9) where palliative care nurses reported less issues as compared to the patients. In addition, there is “no agreement” in item 4 (family anxiety) between the patients and palliative care nurses. This is in stark contrast to the outcomes of other studies that had used the POS that showed significant concordance for pain, other symptoms, life worthwhile, and family anxiety although “severe pain” was often underrated by healthcare provider at every evaluation.[51213]
In short, the palliative care nurses in this study appeared to be “out-of-touch” with many, if not all, of the issues that concerned the patients. This raises the concerns of weak assessment of a patient's mental state and high risk of missed detection of potential suicidal patients with advanced cancer. This worrisome finding where caregivers are poor in detecting patient's psychological symptoms has also been reported elsewhere.[13] This may be due to either inexperience or inadequate training received by the care nurses. Hence, adequate training of palliative care staffs in mental state assessment and emotional counseling should be implemented to help improve concordance between palliative care nurses and their patients in palliative care.
Finally, there is suggestion that the interpretation of the Cohen's kappa is too lenient for health-related studies because it implied that a score as low as 0.41 might be acceptable. Instead, McHugh[14] suggested that kappa below 0.60 indicates inadequate agreement among the raters and little confidence should be placed in the study results. In addition, kappa value below zero is an indicator of a serious problem and a negative kappa represents agreement worse than expected, or disagreement. Low negative values (0–−0.10) may generally be interpreted as “no agreement.”[14] Nevertheless, regardless of which model of interpretation was used, it would appear that the use of the Malay version of the POS in our study has revealed significant gaps in concordance between the palliative care nurses and their patients.
Several limitations were identified in our study. The study subjects in this study were limited to the patients admitted in one hospital and a relatively small number of palliative care staff nurses from a small oncology department, which may result in the kappa statistics being affected by prevalence bias and underestimation of true agreement. Responsiveness to change and test-retest reliability could not be performed as follow-up of the patients was needed to complete the second or third assessment while the patients must also remain stable clinically in between the assessments. This was not possible in this cohort of patients who were inpatients with terminal illnesses. Construct validity was an issue because requiring the patients to complete two questionnaires (the Malay POS and the 30-questions EORTC-QLQ-30) at one time proved to be too much for them, especially when they were ill needed external help to complete the questionnaire. At present, only the EORTC-QLQ-30 has been validated for use in Malaysia for the assessment of palliative care.
CONCLUSIONS
The Malay version of the POS was well accepted and understood by the patients and their palliative care nurses with reliable and solid validity. Sight to fair concordance was shown between the patients and their care nurses. This suggests that more training is needed to help the latter to improve concordance with their patients.
Financial support and sponsorship
The study was funded by research grant from the IMU (IMU New Research Project Number: CSc/Sem6(39)2013).
Conflicts of interest
There are no conflicts of interest.
REFERENCES
- Education and training in palliative medicine. In: Doyle D, Hanks G, Cherny N, Calman K, eds. Oxford Textbook of Palliative Medicine (3rd ed). London: Oxford University Press; 2004. p. :1155-8.
- [Google Scholar]
- The relationship between patient characteristics and carer psychological status in home palliative cancer care. Support Care Cancer. 2003;11:638-43.
- [Google Scholar]
- Development and validation of a core outcome measure for palliative care: The Palliative Care Outcome Scale. Palliative Care Core Audit Project Advisory Group. Qual Health Care. 1999;8:219-27.
- [Google Scholar]
- PRISMA. The use of two common palliative outcome measures in clinical care and research: A systematic review of POS and STAS. Palliat Med. 2011;25:304-13.
- [Google Scholar]
- Validation of a core outcome measure for palliative care in Africa: The APCA African Palliative Outcome Scale. Health Qual Life Outcomes. 2010;8:10.
- [Google Scholar]
- APCA. Development of the APCA African Palliative Outcome Scale. J Pain Symptom Manage. 2007;33:229-32.
- [Google Scholar]
- Assessing the symptoms, anxiety and practical needs of HIV/AIDS patients receiving palliative care. Qual Life Res. 1992;1:47-51.
- [Google Scholar]
- Palliative care for people with HIV/AIDS: Views of patients, carers and providers. AIDS Care. 1993;5:105-16.
- [Google Scholar]
- The palliative care outcome scale manual for cross-cultural adaptation and psychometric validation. 2012, POS Development Team
- [Google Scholar]
- Understanding interobserver agreement: The kappa statistic. Fam Med. 2005;37:360-3.
- [Google Scholar]
- Validation and clinical application of the german version of the Palliative Care Outcome Scale. J Pain Symptom Manage. 2005;30:51-62.
- [Google Scholar]
- Caregiver assessment of patients with advanced cancer: Concordance with patients, effect of burden and positivity. Health Qual Life Outcomes. 2008;6:42.
- [Google Scholar]
