Editor's note: Lauren Fusfeld is director at research and consulting firm Veranex. She can be reached at lauren.fusfeld@veranexsolutions.com.
For people with Type 2 diabetes mellitus, early identification of individuals at risk of diabetic kidney disease or a rapid decline in kidney function is an important step for optimal patient management and improved patient health outcomes, such as slowing or preventing disease progression. Nevertheless, traditional tests of kidney function are insufficient for predicting kidney disease and decline.
To address this unmet need, a medical technology company developed and validated a blood-based test that incorporates biomarkers to predict the risk of kidney disease/decline. The firm sought to demonstrate to physicians and payers the test’s clinical utility, defined as the extent to which the test affects physicians’ monitoring and treatment decisions.
The key question was the following: Would U.S. physicians change their medical recommendations to align with patient management guidelines if they had access to the results of this blood-based biomarker test?
Finding a remote approach
Traditionally, companies have relied on prospective clinical studies to establish clinical utility. The downside of using a clinical study for initial research, however, is that such studies require significant resources, costs and time. The pandemic made finding a remote approach to assessing clinical utility increasingly important.
The medical technology company engaged our team of researchers to identify an effective and efficient methodology for demonstrating clinical utility. Based on a detailed understanding of the company’s needs, we suggested conjoint analysis, an approach recommended by the International Society for Pharmacoeconomics and Outcomes Research for this type of investigation.1, 2 This study design has been utilized to understand preferences of patients with Type 2 diabetes mellitus and other conditions,3, 4, 5, 6 as well as to assess factors that influence physician testing decisions in other indications.7, 8
Conjoint analysis derives the importance of attributes included in descriptive vignette-based profiles (which could be products or patients) based on participants’ responses to these profiles. In this case, physicians documented decisions regarding monitoring frequency, treatment prescribing and drug dosing after viewing hypothetical patient profiles in a self-administered online survey. This method simulates the real world, in which physicians assess multiple relevant characteristics of each patient when making medical recommendations. By deducing attribute importance implicitly from physician decision-making, conjoint analysis can avoid some of the biases that result from requiring physicians to comment on the importance of these factors directly in an interview or online study.
Based on secondary research and consultation with clinical experts, we identified six factors or attributes likely to impact physician decision-making: the patient’s age; results of the novel blood-based biomarker test; and results of four conventional patient measurements – albuminuria, estimated glomerular filtration rate (eGFR), blood pressure and glycemic control represented by hemoglobin A1c (HbA1c) level.
Patient profiles included one level from each of the six attributes. With four levels for the blood-based biomarker test result (no test result, low-risk result, moderate-risk result and high-risk result) and three levels for each of other five attributes, 972 unique profiles were possible. A significant advantage of the conjoint design is that analysis is possible using a small subgroup of the total collection of profiles. We created a subset of 42 profiles with Sawtooth Conjoint Value Analysis (CVA) software. The design was orthogonal, which means that each pair of levels (across different attributes) was intended to appear approximately the same number of times. To balance the need to minimize standard errors (<0.1) for a conjoint exercise designed for 400 respondents with the desire to reduce respondent fatigue, we determined that the survey software should randomly present each respondent with eight patient profiles from the subset of 42 profiles. The profile selection adhered to a least-fill methodology so that each of the 42 profiles would appear approximately equally.
For each patient, the survey asked physicians to make one monitoring frequency decision (increase monitoring frequency, decrease monitoring frequency or maintain standard monitoring frequency). The survey also asked the physicians to make three binary treatment decisions: 1) increase the dose of lisinopril to 20 mg per day for kidney protection or continue the dose of 10 mg per day for kidney protection; 2) prescribe a sodium/glucose cotransporter-2 (SGLT2) inhibitor that has a diabetic kidney disease indication or not prescribe an SGLT2 inhibitor that has a diabetic kidney disease indication; and 3) replace ibuprofen (which could adversely affect the kidney) with a non-systemic therapy or not replace ibuprofen.
Questions before and after the conjoint section served to provide context for the conjoint responses. Physicians described their typical care of Type 2 diabetes mellitus patients and indicated their overall likelihood of using the blood-based biomarker test. Following the conjoint exercise, respondents also answered a few specific questions about the blood-based biomarker test, including perceived advantages and disadvantages. The survey was intended to take 15 minutes to complete.
Five physicians participated in a pilot test of the survey to identify any elements requiring clarification or modification. During this test, our team watched the respondents take the survey in real time and were available to answer and ask questions as needed. To evaluate test-retest reliability, the physicians viewed a version of the survey that included two identical additional patient profiles (hold-out cases) not included in the original set of 42 profiles. Based on the interviews, we refined the text describing the attribute levels before formally launching the survey. For example, we replaced ranges for each level with point values and added interpretive clinical descriptions of eGFR levels (i.e., normal, mildly or moderately decreased) and albuminuria levels (i.e., mildly, moderately or severely increased).
After reviewing all the survey data to ensure quality control, we examined the responses in the conjoint exercise. Because data were sparse for the option to decrease monitoring frequency (selected for only 5% of patient profiles), we combined the decrease monitoring frequency and maintain standard monitoring frequency options so that monitoring became a binary variable, like the other medical decisions in this study.
Using Sawtooth Menu-based Choice Software, we analyzed physician responses using multivariable logistic regression models for the monitoring frequency decision and for each of the three treatment decisions. Each of the four models generated relative utilities for the attribute levels. The difference between the minimum and maximum utility for an attribute produced an initial estimate of the importance of the attribute in a physician treatment or monitoring decision; we then normalized these importance values so that the sum was 100% for each model. Additionally, to evaluate the decision impact of the blood-based biomarker test result versus no test, we used utilities to create odds ratios with 95% confidence intervals.
A clinical benefit
The conjoint analysis suggests that physicians attribute a clinical benefit to predicting the risk of diabetic kidney disease in Type 2 diabetes mellitus patients before kidney damage occurs, as well as predicting rapid decline in kidney function in people with diabetic kidney disease. For the decision about the frequency of monitoring risk factors for diabetic kidney disease, the blood-based biomarker test result was more important than the other patient attributes; for the three other patient management decisions, the blood-based biomarker test result was second in importance. Specifically, the blood-based biomarker test result was second to HbA1c in importance for the decision about prescribing SGLT2s with a diabetic kidney disease indication, second to blood pressure for the decision about increasing the dose of lisinopril and second to eGFR for the decision to replace ibuprofen with a kidney-sparing medication.
Importantly, this study indicated that physicians are likely to apply the blood-based biomarker test results appropriately. A low-risk test result reduced the likelihood of frequent monitoring and of more aggressive treatment compared with no test, suggesting a low-risk test result could limit unnecessary therapy, minimize side effects and lower treatment costs. Similarly, a moderate- and high-risk result increased the likelihood of monitoring and treatment changes to protect the kidney in patients at higher risk of diabetic kidney disease or rapid decline in kidney function; consequently, the blood-based biomarker test could contribute to a more personalized approach to diabetic kidney disease management. Also noteworthy was the statistical significance of the odd-ratios of the blood-based biomarker test; all were statistically significant apart from the impact of the low-risk and moderate-risk result on increasing the dose of lisinopril.
Lastly, the conjoint analysis allowed us to explore the impact of other patient characteristics on physician decisions. For example, patient age did not have a statistically significant impact on physician decisions regarding monitoring frequency and replacing ibuprofen with non-systemic therapy.
Some limitations
The conjoint analysis approach does have some limitations. First, the findings are limited to the attributes and levels included in this analysis. While we used secondary research and consultation with clinical experts to focus on the most important attributes, we acknowledge that other patient characteristics may also influence physicians’ treatment and monitoring decisions. Second, while the study was designed to represent real-world decision-making, survey results could differ from physician behavior in a clinical setting. As time and funds allow, additional research could be conducted to confirm these initial findings now that the current study has provided proof of concept. In fact, the conjoint analysis could serve as a guide in the design of optimal endpoints in such a prospective study. Likewise, additional research could also measure the health outcomes of Type 2 diabetes mellitus patients whose physicians had access to the novel blood-based biomarker test results, as the conjoint study was not designed to assess the impact of changes in monitoring and treatment on patient outcomes. Finally, as researchers, we seek to share our findings with a broader audience by publishing the results in high-impact journals. Determining the appropriate journal for publication can be a challenge because some journals, especially those with a clinical focus, may have difficulty identifying reviewers who have the appropriate mix of clinical and biostatistical expertise to assess conjoint analysis techniques in clinical utility studies.
Right treatments, right time
Given opportunities for improved diabetic kidney disease risk prediction, physicians could provide the right treatments at the right time, understand when the risk of damaging the kidney exceeds the benefit of certain treatments and tailor monitoring frequency to each Type 2 diabetes mellitus patient.
With conjoint analysis, we were able to accomplish the following study objectives in a cost-effective, time-efficient and robust manner:
- quantify the clinically and statistically significant impact of the novel blood-based biomarker test on patient management decisions;
- objectively demonstrate the test’s value to physicians across a range of patient types by evaluating the relative importance of the blood-based biomarker test’s results compared with standard-of-care test results and other clinical factors.
Conjoint analysis, one of the many tools in a researcher’s toolkit, is a robust and useful methodology for demonstrating clinical utility.
References
1 Bridges, J.F.P., Hauber, A.B., Marshall, D., Lloyd, A., Prosser, L.A., Regier, D.A., et al. “Conjoint analysis applications in health – a checklist: a report of the ISPOR good research practices for conjoint analysis task force.” Value in Health. 2011 June;14(4):403-13.
2 Johnson, F. Reed, Lancsar, E., Marshall, D., Kilambi, V., Mühlbacher, A., Regier, D.A., et al. “Constructing experimental designs for discrete-choice experiments: report of the ISPOR conjoint analysis experimental design good research practices task force.” Value in Health. 2013 January;16(1):3-13.
3 Janssen, E.M., Hauber, A.B., Bridges, J.F.P. “Conducting a discrete-choice experiment study following recommendations for good research practices: an application for eliciting preferences for diabetes treatments.” Value in Health. 2018 January;21(1):59-68.
4 Hazlewood, G.S., Pokharel, G., Deardon, R., et al. “Patient preferences for maintenance therapy in Crohn’s disease: A discrete-choice experiment.” PLOS ONE. 2020;15(1):e0227635.
5 Kromer, C., Schaarschmidt, M.L., Schmieder, A., Herr, R., Goerdt, S., Peitsch, W.K. “Patient preferences for treatment of psoriasis with biologicals: a discrete choice experiment.” PLOS ONE. 2015;10(6):e0129120.
6 Ride, J., Lancsar, E. “Women’s preferences for treatment of perinatal depression and anxiety: a discrete choice experiment.” PLOS ONE. 2016;11(6):e0156629.
7 Vachani, A., Tanner, N.T., Aggarwal, J., et al. “Factors that influence physician decision making for indeterminate pulmonary nodules.” Annals of the American Thoracic Society. 2014;11(10):1586-1591.
8 Szeinbach, S.L., Harpe, S.E., Williams, P.B., Elhefni, H. “Testing for allergic disease: parameters considered and test value.” BMC Family Practice. 2008;9:47.