Therapeutic misconception, misestimation, and optimism in participants enrolled in phase 1 trials
Abstract
BACKGROUND:
Ethical concerns about phase 1 trials persist. Important conceptual advances have been made in understanding concepts used to describe misunderstanding. However, a systematic, empirical evaluation of the frequency of misunderstanding incorporating recent developments is lacking.
METHODS:
The authors queried 95 participants in phase 1 trials to provide a more sophisticated estimate of the proportion who had therapeutic misconception (TM), defined as misunderstanding the research purpose or how research differs from individualized care, and therapeutic misestimation (TMis), defined as incorrectly estimating the chance of a research trial benefit as >20% or underestimating risk as 0%.
RESULTS:
Sixty-five of 95 respondents (68.4%) had TM, which was associated in a multivariate analysis with lower education and family income (P = .008 and P = .001, respectively), but TM was not associated with the vulnerability of having hardly any treatment options. Eighty-nine of 95 respondents (94%) had TMis, although only 18% reported this was a factual estimate. Although the risks of investigational agents and those exacerbated by research, such as uncertain outcomes, were mentioned (39% and 41% of respondents, respectively), risks novel to research, such as research biopsies, were rarely mentioned (3% of respondents). Although most of these respondents believed that their chance of benefit was greater and that their risk was lower than the population chance (optimists) (54.6%), a substantial minority of respondents (37.6%) were pessimists.
CONCLUSIONS:
TM continues to be prevalent. Estimates of personal benefit were not usually meant to report facts, it remains unknown whether respondents in the current study had TMis. Although they are not more vulnerable, phase 1 participants need improved understanding of key TM concepts, with attention to risks that are not present in standard of care. Cancer 2012. © 2012 American Cancer Society
INTRODUCTION
Ethical concerns about phase 1 trials have been discussed for more than 15 years1-3 and have gained considerable attention in the last 7 years.4, 5 According to the reports, the crux of the problem is that most patients misunderstand the purpose of a phase 1 trial2, 6, 7 and enroll for personal benefit8-11 when, at best, the chance of benefiting from a phase 1 trial is uncertain12 and likely is small.13 This disconnect between chance of benefit and expectation of benefit has led commentators to worry that phase 1 participants do not understand phase 1 research. Two notions have been used to capture this misunderstanding: therapeutic misconception (TM) (misconstruing research as personal medical care) and therapeutic misestimation (TMis) (incorrectly estimating the chance of benefit or risk). TM and TMis have undergone careful conceptual development in the last decade, and reasons have been offered to explain why they may not be as prevalent or as easily measured as first assumed.5, 12, 14-18 In addition, a third category, therapeutic optimism (TO), which originally was intended to indicate an ethically acceptable reason why phase 1 participants may overestimate personal benefit, recently has been criticized as ethically problematic.19 A systematic, empirical evaluation of the frequency of misunderstanding that takes into account these recent developments is lacking. The objective of our current study was to document the frequency of TM and TMis with more sophisticated measurement based on the new conceptual insights and to search for associations of TM and TMis with TO.
The concept of TM has evolved since it was coined by Appelbaum et al in 1982.20, 21 A consensus panel of experts has suggested the following definition: TM exists if individuals do not understand that the central objective of clinical research is to produce knowledge that is generalizable whether or not the enrolled participants potentially may benefit from the intervention being studied or from other aspects of the trial.22 Two core concepts present in this and other definitions offered are: 1) individuals need to understand that the main purpose of the research protocol is to produce generalizable knowledge, and 2) conducting a research protocol differs from providing individualized care.14, 15, 22, 23
To more clearly describe respondents' views on the risks and benefits of phase 1 trials, we measured TMis separately from TM,17, 22 taking into account 2 important caveats about estimating the risks and benefits of clinical trials. First, there are 2 types of statements about uncertainties: frequency statements reporting the probability of risks or benefits in the general population (eg, “20 of 100 patients will experience tumor shrinkage”) and belief statements about the participant's own probability (eg, “I am 80% certain that my tumor will shrink”).18, 24
Second, belief statements can serve purposes other than reporting facts; they also can be used to express a positive attitude or to demonstrate hope (eg, “I know it is important to have a positive attitude, so I am telling myself confidently that my tumor will shrink”).18 In sum, an analysis of estimates of risk and benefit must differentiate frequency from belief statements and determine the intention of the belief statement.
TO, or being optimistic about the success of a treatment, was introduced as an ethically unproblematic explanation for why patients may overestimate their personal chance of medical benefit17: Patients are optimistic about their prospects, and this reduces anxiety and gives them hope.25, 26 Recently, concerns have been expressed that TO can be unrealistic and may cloud a patient's judgement.19, 27
Given these conceptual advances, the 3 objectives of the current project were: 1) to determine whether misunderstanding, measured with careful attention to recent conceptual advances, is as widespread as feared; 2) to identify the characteristics of participants who suffer from TM or TMis so that these groups can be offered interventions to enhance the consent process; and 3) to search for associations between TM and TMis and TO.
MATERIALS AND METHODS
This research was approved by the Emory University Institutional Review Board, and all respondents consented to participate.
Population
We interviewed and surveyed phase 1 trial participants at an academic center during the first month of their phase 1 trial participation. All patients had an Eastern Cooperative Oncology Group performance score of 0 to 2 and, thus, were ambulatory and “up and about” at least 50% of the time.
Instrument Development
Because no validated instrument measures the current constructs of TM and TMis, we developed 2 new instruments, an in-person, face-to-face, structured interview with open-ended questions and a paper-and-pencil, self-administered, quantitative survey. Using standard instrument-development techniques,28-30 we first conducted a comprehensive literature review to identify domains and relevant past studies that used scales of interest. The resulting questions were reviewed by a focus group of phase 1 investigators; revised; and then tested cognitively with 12 cancer patients who were asked to assess question wording, to demonstrate comprehension by “thinking out loud” as they framed their answers, and to determine whether there were missing categories to query.
The final domains included motivation for participation, perceptions and estimations of risks and benefits, understanding of phase 1 research, and an expansive section on demographics, including attitude questions, measured on a 10-point Likert scale. The attitude questions were derived from 2 sources: 1) We queried strength of and comfort from religious or spiritual belief and strength of optimism based on work by Weinfurt et al29; and 2) we measured trust in medical research, trust in the phase 1 investigator, and how many cancer treatment options respondents believed they had left based on suggestions that trust31, 32 and availability of options33, 34 may contribute to misunderstanding.
Operationalizing Concepts
Therapeutic misconception
To gauge TM, we asked 2 questions: 1) “Is the research study mostly intending to help research and gain knowledge or mostly intending to help you as a person?”; and 2) “Does the research study or your physician decide the treatments?”15, 23 The first question focuses on the main intent of producing generalizable knowledge, and the second focuses on the difference between protocol-driven treatment and individualized care, the 2 core concepts present in most definitions of TM.
Therapeutic misestimation
In an attempt to obtain a complete picture of the respondents' views of risks and benefits, we asked 4 open-ended questions: what was the main reason the respondent decided to participate in the phase 1 study, what benefits the respondents hoped to gain by joining the study, what risks the study entailed, and whether any risks were particularly concerning to the respondents. We also asked respondents to estimate the probability of benefit and risk in the closed-ended survey. To differentiate the 2 uncertainty statements (frequency and belief), we asked the 2 questions suggested by prior research29: 1) To elicit a frequency statement about the probability that the phase 1 population will benefit (population benefit question), respondents were asked, of 100 patients, how many patients on average would benefit by having the growth of their cancer stop or slow down by the end of the study for at least 6 months; and 2) to elicit a personal belief statement about the probability that they would benefit (personal benefit question), respondents were asked about the chance that they personally would benefit and have the growth of their cancer stopped or slowed down by the end of the research study for at least 6 months. We poised similar frequency and belief questions for risk. TMis is best understood as an individual misestimating their own personal chance of risk and benefit,29 so our main analyses focused on estimations of personal benefit and risk.
Finally, because belief statements can serve purposes other than reporting facts, we asked what individuals meant by their estimate of personal benefit with the closed-ended choices of 1) “those are just the facts,” 2) “that is what I hope will happen,” 3) “that is what I fear will happen,” and 4) “I think it is important to have a positive attitude.”
The correct chance of medical benefit for patients was identified as <20% based on a literature review13, 35, 36 and our center's experience. Because the phase 1 trials in which the respondents were enrolled had different levels of risk, some of which posed very minimal risk, the Winship Cancer Institute phase 1 investigative team determined that any probability of risk other than zero should be considered correct, which we considered a generous interpretation. TO was measured in 2 ways: with the Cohen et al scale25 and using the Weinfurt et al attitude question to measure optimism on a 10-point Likert scale.29
Method of Administration
Phase 1 participants who consented to participate in this study were interviewed by the ethics research assistant at the first clinic visit after enrolling on the phase 1 protocol. Open-ended answers were recorded verbatim on the structured interview form. The patient was then given the survey to self-administer, preferably at that clinic visit. However, it was possible to mail the survey. If the survey was not received by the next scheduled clinic visit, then the patient was contacted at subsequent clinic visits over the next month, and the survey was distributed and collected at that time. All interview and survey data were entered into a web-based database by the ethics research assistant and were double checked by the research coordinator.
Statistical Methods
To identify the factors associated with TM and TMis, a 2-sample t test and/or a Wilcoxon sum-rank test was used for the continuous predictors, and a chi-square test was used for categorical predictors. A logistic regression model was used for the multivariable analysis of TM and TMis. Spearman correlation coefficients were calculated for the association between 2 continuous variables. The SAS statistical software package (version 9.2; SAS Institute, Inc., Cary, NC) was used for all data management and analyses.
Open-ended questions querying benefits and risks were coded independently by the phase 1 nurse practitioner and the ethics research assistant. After coding the first 10 respondents, a standard code book was developed for each question and was reviewed and revised by the principal investigator. The 2 coders recoded all the questions using the standard code book, and all differences of opinion were resolved by the principal investigator. In addition, the codes that were developed for the 2 questions about benefits (that is, what benefits the respondents expected from trial participation and the main reason why respondents participated in the phase 1 trial) were grouped by the ethics research assistant into the types of benefits identified in King's seminal article: direct benefit from the trial intervention and collateral benefit resulting from being a research participant, such as extra testing, access to experimental therapies, and the gratification of altruism.37 Although King identifies a third benefit, aspirational benefit, it was not included in our study, because King defines aspirational benefit as a benefit to society, and our study focuses on benefits for the participant. We report altruistic collateral benefit separately from the other collateral benefits to compare our data with previous reports of altruism.8, 9 The principal investigator reviewed and finalized the categorization of benefits into King's categories.
RESULTS
Of the 114 patients who were approached, 14 refused to participate in our study, and 5 did not complete either the interview or the survey, yielding a sample of 95 patients (83%). The median respondent age was 57 years (range, 28-85 years), and the respondents included 56% men, 67% whites, 50% college graduates, and 57% with incomes ≥$60,000. The only significant difference between the respondents and refusers was that refusers more frequently were women (P = .029). Most respondents reported their strength of (81 respondents; 85%) and comfort from (81 respondents; 85%) religious or spiritual beliefs as high. Trust, both in the research (83 respondents; 87%) and in the phase 1 investigator (84 respondents; 88%), also was high. Twenty-eight respondents (29.5%) reported that they had lots of options left in their cancer care, 38 respondents (40%) reported that they had a few options, and 25 respondents (26%) reported that they had hardly any options.
Because similar measures of attitude were highly correlated, we combined them into a single score to simplify the analysis: Strength of and comfort from religious beliefs (Spearman correlation coefficient, 0.92; P < .001) were combined into a single religion score (REL); trust in research and in the investigator (Spearman correlation coefficient, 0.40; P < .001) were combined into a trust score (TRUST); and scores on the Cohen et al optimism scale and on the single strength of optimism question (Spearman correlation coefficient, 0.52; P < .001) were combined into a single optimism score (OPT).
Therapeutic Misconception
Sixty-five respondents (68.4%) had TM, because they did not correctly answer the 2 core questions. The question whether the research study or the physician determines the study treatments (question 2) was more frequently left blank or was answered “do not know” than the question about the main purpose of the study (question 1) (14.7% vs 1%; P = .01) (Table 1).
(n = 95) | Question 1: Main Purpose? No. of Respondents (%) | ||
---|---|---|---|
Question 2: Treatment Is Decided By? | Correct | Incorrect | DNK/Blank |
Correct | 31 (32.6)a | 22 (23.2) | — |
Incorrect | 18 (18.9) | 9 (9.5) | 1 (1) |
DNK/blank | 8 (8.4) | 6 (6.3) | — |
- Abbreviations: DNK, do not know.
- a This result indicates a lack of therapeutic misconception, because both questions were answered correctly.
Respondents with less than a college degree (P = .008), with family incomes <$60,000 (P = .001), and with a higher REL score (P = .031) more frequently had TM. A multivariate analysis identified 2 significant factors associated with TM, less than a college degree (odds ratio = 5.0, P = 0.02) and income less than $60,000 (odds ratio = 10.7, P = 0.001).
Therapeutic Misestimation
Only 3 respondents (3%) correctly answered both the personal risk and benefit question, 3 respondents (3%) left at least 1 estimate blank, and 89 patients (94%) misestimated risk and benefit. Most of the misestimations were overestimations of benefit, which was not surprising given our interpretation of correct risk estimates as anything other than zero. However, 13 respondents (14%) asserted that their personal risk was zero. TMis was associated only with the OPT score (P = .007).
Overall, the estimates of personal benefit were high: Fifty-nine respondents (62%) estimated a ≥70% chance of personal benefit. Only 17 respondents (18%) stated that their estimate of personal benefit was intended to be just the facts, whereas 31 respondents (33%) stated that it was what they hoped would happen, 2 (2%) stated that it was what they feared would happen, and 43 (45%) stated that it was important to have a positive attitude (2 responses were missing). The estimates of personal risk were low: Fifty-nine respondents (62%) estimated that their personal risk was ≤30%.
Benefits of Trial Participation
When respondents were asked to list all the benefits they hoped to gain from trial participation, they mentioned 155 benefits, including direct medical benefits (102 respondents; 66%), altruistic benefits (helping others and helping science; 19 respondents; 19%), and other collateral benefits (access to an academic center, financial help, access to new cutting-edge treatments, and closer monitoring of their care; 24 respondents; 15.5%). Forty-nine respondents (52%) mentioned only direct medical benefits, 6 (6%) mentioned only altruistic benefits, 9 (9%) mentioned only other collateral benefits, and 31 (33%) mentioned more than 1 type of benefit (all but 3 of those 31 included direct medical benefits). In sum, 77 respondents (81%) mentioned a direct medical benefit. When asked to rank benefits, direct medical benefit most frequently was ranked first (85 of 93 respondents; 91%) and second (35 respondents; 38%), and helping others most frequently was ranked third (29 respondents; 31%).
Motivation for Participation
Sixty-three respondents (66%) offered only 1 type of reason for entering the trial: Forty-five respondents (47%) joined the trial to obtain direct medical benefits, 12 (13%) joined to help science or others, 3 (3%) joined to obtain other collateral benefits (access to a cutting-edge treatment, chance to fight their cancer), and 3 (3%) joined based on their physician's recommendation. The remaining 32 respondents (34%) offered more than 1 reason, and the most frequent combination was direct medical benefit and altruism (17 respondents; 18%). Seventy-two respondents (76%) had direct medical benefit as at least 1 reason for entering the trial.
Risks of Trial Participation
In the close-ended survey, 13 respondents (14%) estimated that their personal risk was zero. However, when asked in an open-ended question, 26 respondents (27%) said that there were no risks, and 56 (59%) said that no risks were concerning to them. In total, 90 risks were identified by 69 respondents (73%), and 46 risks were considered concerning by 39 respondents (41%). Codes that emerged from the verbatim description of risks were: 1) risks directly attributed to standard of care interventions/medications; 2) risks directly attributed to the research intervention/medications; 3) risks because of nontherapeutic features of the research protocol, such as, the trial can be stopped at any time or there is a nontherapeutic biopsy; 4) participation can lower quality of life; and 5) risks inherent in standard of oncologic care but arguably exacerbated in research (unknown side effects; uncertainty of outcome; fear it will not work; need to postpone other, perhaps more effective treatment; breach of confidentiality). Table 2 tabulates the frequency with which respondents mentioned these risks and the risks that concerned them.
Category | No. of Respondents (%) | |
---|---|---|
Risks Mentioned by Respondent | Respondent Identifies the Risk as Concerning | |
1. Side effects of the investigational agent | 37 (39) | 31 (33) |
2. Side effects of standard of care medications | 8 (8) | 3 (3) |
3. Risks novel to research due to research design | 3 (3) | 0 (0) |
4. Lowering quality of life | 3 (3) | 4 (4) |
5. Risks of any cancer treatment that may be exacerbated in research | 39 (41) | 8 (8) |
a. Breach in confidentiality | 1 | 0 |
b. Need to postpone other, perhaps more effective, treatment | 3 | 1 |
c. Unknown side effects | 19 | 4 |
d. Uncertainty of outcome | 7 | 2 |
e. Fear treatment will not work | 9 | 1 |
6. None | 26 (27) | 56 (59) |
Therapeutic Optimists
Thirty-nine respondents (41%) were classified as therapeutic optimists according to the Cohen et al scale,25 and most (82 respondents; 86%) ranked their optimism as high. Most respondents (54.6%) estimated their probability of personal benefit higher and their probability of personal risk lower than their estimates for the population. However, there was a significant minority (29 respondents, 36.7%) who either estimated their personal benefit as lower or their personal risk as higher than the population (Table 3).
Frequency (percent) | Higher Personal Benefit Than Population Benefit: No. of Respondents (%) | |
---|---|---|
Lower Personal Risk Than Population Risk | Yes | No |
Yes | 42 (54.6) | 15 (20) |
No | 14 (16.7) | 5 (6.7) |
DISCUSSION
Although the current study sample's demographic characteristics are not representative of the study's catchment area (more white and higher income), they are representative of phase 1 populations in general, with regard to both standard demographics38 and trust.30 On the face of it, even with a more sophisticated analysis of TM and TMis, understanding levels were low. TM was widespread (68%) and TMis (misestimating one's personal potential for benefit and risk) was nearly universal. The concern that phase 1 participants' main motivation for participation is direct medical benefit, when the chance of such benefit is not high, also appeared to be substantiated.
However, the conceptual advances made in the last decade raise serious concerns about interpreting these data. First, although most respondents could not correctly answer both questions posed as the test of TM, more than half of respondents could identify the main purpose as helping research, a higher percentage than reported by Daugherty et al (9 of 27 respondents; 33%)2 but lower than that reported by Jansen et al (51 of 70 respondents; 72.9%),19 both of which posed open-ended questions. Second, 14.7% of respondents did not answer the query whether their physician or the research determined the treatment, although all but 1 respondent answered the question about the main purpose of the trial. This result hints that it may be difficult for participants to provide a confident answer about how the physician is operating in clinical research, supporting Wendler and Grady's contention that misconstruing the patient-investigator relationship as a patient-clinician relationship is a source of misunderstanding.23
The conceptual advances also call into question conclusions about the prevalence of TMis. Seventy-six respondents (80%) stated that they were not making a statement of fact when reporting their estimate of personal benefit, but, rather, were expressing their hope or a positive attitude. Therefore, although most participated for direct medical benefit, we do not know whether they also overestimated their chance of personal medical benefit, because most were not offering factual estimates.
Some have reported that the high estimates of personal benefit indicate that phase 1 participants suffer from unrealistic optimism,19 assuming they will do better than other, similar patients. Arguably, unrealistic optimism can cloud a patient's appreciation of what is entailed by phase 1 participation and can threaten their decision-making capacity. The association we observed between OPT scores and TMis may support this claim. However, we identified an intriguing subgroup (36.7%) that appeared to be comprised of pessimists who estimated their personal benefit lower than the population's chance of benefit or their personal risk higher than the population's chance of risk. A tendency toward pessimism has not been documented before and reveals a more complex picture of some early phase trial participants' thinking about risks and benefits. Perhaps, just as many of the estimates of personal benefit were expressions of hope and a positive attitude, the estimates of risk were expressions of preparation for the worst to happen. A limitation of our study was that we did not specifically query what the respondents meant by their estimates of personal risk, leaving this question open for future research.
Although incorporating the various conceptual caveats into our study design resulted in casting doubt on any conclusions about the rate of TMis, we did glean a fair amount of information about these respondents' views of benefits and risks.
Although most participated to obtain direct medical benefit, only half cited medical benefit as their sole reason for participation, and approximately 33% mentioned multiple benefits, including collateral benefits, supporting Agrawal and Emanuel's4 contention that at least some phase 1 participants have a broad view of benefits.
The 2 main categories of risk identified were the risk of the investigational agents (41%) and the risks inherent in any cancer treatment that may be exacerbated in research (43%), such as the uncertainty of outcome and unknown side effects. It is reasonable to argue23 that the risks of the interventional agents are exactly those that should receive focus. It is noteworthy that many of these respondents, all of whom had exhausted standard therapy, were aware of the risks present in any cancer treatment that could be exacerbated in research, but only 3 mentioned the novel risks of trial participation, such as extra biopsies, pharmacokinetics assessments, and variable and nonindividualized dosing. This result supports the concern that participants either do not understand or do not focus on the key differences between individualized care and research, namely, that being on a clinical trial introduces procedures and practices that are not included in standard care.
More of these participants (13%) were motivated by altruism than previously reported for phase 1 participants,8, 9 although this level was similar to that in the population at large reported by the President's Advisory Committee on Human Radiation Experiments, which also identified, like the current report, a mixture of motives for trial participation.28
The often reported demographics associated with misunderstanding of research surfaced again: TM was associated with lower education and lower family income. However, we queried an extensive set of attitudes as well as standard demographics and uncovered several noteworthy results. The supposed vulnerability to a self-perception of having hardly any cancer treatment options was not correlated significantly with TM. This finding lends support to the contention that phase 1 participants are not more prone to misunderstanding than others because of their lack of treatment options.38 Nor did our data support the theory that trust or optimism may account for TM,32 because neither TRUST or OPT was associated significantly with TM, although OPT was associated with TMis. This results supports data from Jansen et al indicating that an optimistic bias is not associated with misunderstanding.19 Like Weinfurt et al, we also observed a correlation between REL and TM,29 suggesting a connection that deserves further research.
Because we could not identify a single demographic that was associated with both TM and TMis, we suggest that, rather than targeting subpopulations to improve understanding, the 3 concepts that were most frequently misunderstood in this study should be emphasized in the consent process. Greater than 66% of respondents could not identify both the main purpose of research and lack of individualized care; therefore, we believe that each of these should be emphasized. In addition, the largest deficit in the appreciation of risk was an appreciation of the novel risks introduced by phase 1 participation, such as variable dosing and additional, nontherapeutic tests. These risks should be highlighted.
The current study had several limitations. Because it was a single-institution study at an academic phase 1 cancer research center, the results may not be generalizable. The study population was heavily white with higher incomes, although this is typical of phase 1 studies. We do not know whether the estimates of personal risk were intended as statements of fact. However, the study design did take advantage of the many advances in conceptual analyses for TM and TMis and specifically queried estimates of risks as well as benefit—an advantage over much previous research. Finally, because the interviews and surveys were administered up to 1 month after the consent document was signed, faulty recall may be responsible in part for the high level of TM; although, even directly after the consent conversation, patients do not remember that key pieces of information were discussed.39
In conclusion, conceptual advances allowed us to measure TM and TMis more carefully, substantiating that the majority of these respondents suffer from TM. TM was associated with lower income and higher education but not with the perception of few treatment options, undermining the view that phase 1 participants are vulnerable because of limited treatment options. The majority participated in hope of direct medical benefit, although many listed other benefits as well. However, whether or not these participants overestimated their chance of personal benefit is unknown, because more careful measurement revealed that proffered estimates often were not intended to report facts. This careful measurement also resulted in a novel finding. A substantial minority of participants were pessimists and estimated their chance of benefit as lower or their chance of risk higher than that of the population. Although 25% did not list any risks at all, just less than 50% were aware of both the risks of the investigational agent and the risks that may be exacerbated by research, both of which are important to understand. What was missing was attention to the novel disadvantages introduced by research participation, such as nontherapeutic procedures, which were almost never mentioned. Building on the conceptual advances made in the last decade, it is now clearer that TM continues to be a problem, and a major deficit in these respondents' understanding is grasping the novel ways in which research differs from standard of care.
FUNDING SOURCES
This work was supported by a grant from the National Cancer Institute (NCI P01 CA 116676 to F.R.K.).
CONFLICT OF INTEREST DISCLOSURES
The authors made no disclosures.