Development of a Consensus Syllabus of Palliative Medicine for Physicians in Japan Using a Modified Delphi Method

Development of a provisional syllabus

To develop a provisional syllabus, we adopted the following procedures. First, the authors established the structure and sections of the syllabus based on discussions and literature review.[121920212223242526272829] Second, one author (A. S.) generated an item pool of learning outcomes under each category of the syllabus based on a literature review. Third, the authors discussed the appropriateness and coverage of the item pool of learning outcomes to reach a consensus regarding their validity, and the provisional syllabus was then formulated.

Expert panel selection

We selected expert and eligible panelists to create a Delphi panel that consisted of Diplomate or Faculty of the Specialty Board of Palliative Medicine and certified by the Japanese Society for Palliative Medicine. We contacted the Japanese Society for Palliative Medicine through mail and asked them to participate in this study and recommend panelists based on the following criteria: (1) physicians with adequate experience as part of a palliative care consultation team; (2) physicians with adequate experience at a palliative care unit; (3) physicians with adequate experience in community-based palliative care; and (4) coordinators of palliative care education at a university-level graduate school of medicine. Each group consisted of five expert panelists, making a total number of 20 expert panelists. All individuals who confirmed that they met the eligibility criteria and expressed a willingness to participate were included in the Delphi study. The study was conducted in accordance with the Declaration of Helsinki and ethical guidelines with regard to clinical research. This study was reviewed by the Institutional Review Board at Hyogo Prefectural Kakogawa Medical Center, which approved it with waiver of informed consent.

Survey process

Google Forms™ was used to conduct anonymous web-based surveys during the period from September 2016 to March 2017. Our Delphi study consisted of three rounds, each lasting 4 weeks with a 4-week gap between the rounds. Nonrespondents were sent weekly E-mail reminders. No financial incentives were provided.

First, each panelist was asked to review existing syllabi and literature to standardize their knowledge regarding learning outcomes in palliative medicine education for physicians.

Second, 4 weeks later, we implemented a first-round survey, mailing a questionnaire with the outline of the provisional learning outcomes to each panelist. Each member was asked to rate the appropriateness of each learning outcome using a nine-point Likert-type scale (inappropriate 1–3, intermediate 4–6, and appropriate 7–9). In cases where panelists were unfamiliar with items due to their specialty, an “incapable of rating” result was also generated. Panelists, who rated a statement with a score of <6, were asked to provide the reason. In addition, each member was asked to rate the difficulty in each learning outcome using a four-point Likert-type scale: 0 (easy), 1 (adequate), 2 (moderately difficult), 3 (too difficult), and the learning outcomes were reformulated to be more achievable as needed. Panelists who rated the outcome difficulty as 2 or 3 were asked to provide the reason. A consensus in this study was defined a priori as agreement (appropriate, 7–9) among a minimum of 75% of the experts. We also collected basic demographic information from the experts including age, gender, type of clinical practice, and years of experience. A summary of the first-round survey was sent to each panelist and author, and disagreements were discussed via E-mail over 2 weeks. We asked the panelists, especially those who would not be able to attend a panel meeting, to provide their opinions.

Third, we contacted 11 palliative care-related organizations [Table 1] through mail-in December 2016 and asked them to participate in the study and to recommend a representative in charge of education in palliative medicine as an external reviewer. We mailed them the provisional syllabus, summary of the first-round survey, and description of each panelist. We requested opinions regarding the provisional syllabus and each learning outcome from the external reviewers.

Fourth, following discussions through E-mail, an expert panel meeting was convened on December 24, 2016, in Tokyo to discuss statements causing disagreement in person. At the meeting, a summary of the first-round survey, discussion through E-mail, and opinions from the external reviewers were distributed. Following the panel meeting, a summary of the meeting and a revised version of the learning outcomes were sent to all panelists to confirm corrections or to determine whether there were additional opinions.

Fifth, we implemented a second-round survey using the same method as in the first-round survey, addressing only the learning outcomes that could not be agreed on in the first-round survey. For learning outcomes considered inappropriate, the relevant panelists were contacted via E-mail individually, and we attempted to reach an agreement.

Sixth, we conducted a third-round survey using the same methods as in the first-and second-round surveys, addressing only the learning outcomes that could not be agreed upon in the first-and second-round surveys. We eliminated learning outcomes that could not be agreed on during the third round.

Statistical analysis

Data analysis was performed using the software Statistical Package for Social Science version 22.0 (SPSS Japan, Tokyo, Japan).

Participant characteristics

The participants’ characteristics are summarized in Table 2. Among 20 experts surveyed, 20 (100%) responded over all rounds. Ten (50%) participated in a panel meeting.

First Delphi round

In the first-round survey, 179 of 179 (100%) learning objectives were judged to be appropriate by more than 75% of the respondents, and 5 of 179 (3%) learning objectives were judged to be too difficult by more than 10% of the respondents.

In the panel meeting, all learning objectives were examined carefully. Subsequently, 3 new learning objectives were added, and 25 learning objectives were excluded, owing to their high difficulty, during the panel meeting. In case of satisfactory statements that included correcting modes of expression, shuffling of learning objectives among courses, and binding similar objectives together, we made revisions based on a discussion among the participants and authors. In addition, we reworded 15 learning objectives judged to be difficult to make them more understandable and achievable. The number of learning objectives was 157 across 22 courses after the panel meeting. Following the panel meeting, a summary of the panel meeting and a revised version of the learning objectives were sent to all panelists to confirm corrections or determine whether there were additional opinions. We revised them based on a discussion among authors, resulting in 18 learning objectives being reworded, and then conducted the second Delphi round.

Second Delphi round

In the second-round survey, all panelists responded with 18 of 18 (100%) learning objectives judged to be appropriate by more than 75% of the respondents. No learning objectives were rated to be unnecessary or unimportant by more than 75% of the respondents. We decided to conclude the Delphi rounds after the second-round survey because most of the stated learning outcomes had achieved consensus. The final version of the syllabus [Appendix] consists of 157 specific behavioral objectives and 22 general instructional objectives across 22 courses [Table 3].

Third Delphi round

We concluded the study with the third-round survey, and no further Delphi round because all of the stated learning outcomes had achieved consensus after the second-round survey.

Financial support and sponsorship

This study was funded by the Japanese Society for Palliative Medicine.

Conflicts of interest

There are no conflicts of interest.