Fall 2025 (Volume 35, Number 3)

Multicriteria Decision Analysis with 1000Minds for Developing Scleroderma Renal Crisis Classification Criteria

Download PDF

The Scleroderma Clinical Trials Consortium Scleroderma Renal Crisis Working Group was created to develop and validate classification criteria for scleroderma renal crisis (SRC). In accordance with recommendations for rheumatic disease classification criteria development, we designed a multi-phase expert- and data-driven study. In phase 1, we performed a scoping review to generate items to define SRC.¹ In phase 2, we undertook item reduction using consensus techniques (3-round online Delphi exercise and Nominal group discussion) to generate a core set of items to define SRC.²

We secured CIORA funding in 2023 to conduct phase 3 of the study (Principal Investigators: Marie Hudson, Sindhu Johnson and Christopher Denton). The aim was to refine and weight the core set items. Thirty-one case vignettes reflecting the broad range of SRC were submitted by 8 experts from around the world. A first-ranking exercise was performed by an independent panel of 14 experts (see photo below) to capture individual preferences for item importance. The same experts then met in person for a 1.5-day meeting. Items were discussed individually, and a framework and glossary were finalized. Experts discussed the need for 1) entry criteria; 2) exclusion criteria; and 3) sufficient (or absolute) criteria for inclusion. A multi-criteria decision analysis (MCDA) exercise was then conducted to determine the relative weighting of each item using 1000Minds software. The panel was presented with paired scenarios comparing items and asking them to select the option they believed had a higher probability of being classified as SRC. The distribution of choices on each scenario was presented to the group. Where there was no agreement, the reasons for disagreement were discussed. Consensus was considered achieved when all experts indicated agreement or could accept the majority decision. Through iterative discrete pair-wise choices, the decision analytic software was able to assign relative weights to the items.

Finally, the results of the first-ranking and MCDA exercises were analyzed to produce a continuous measure of the relative probability that a case could be characterized as SRC, and a provisional threshold score above which a case could be definitely classified as SRC.

We have now undertaken the 4th and final phase of the project. We are collecting a large international cohort of SRC cases and controls to 1) refine the weights and threshold score in a new derivation cohort, and 2) test the sensitivity and specificity of the final criteria in an independent validation cohort. We expect to produce the first expert- and data-driven classification criteria for SRC. This work will provide a robust framework for future projects to tackle substantial unmet clinical needs for SRC.

Members of the MCDA panel: Dr. Robyn Domsic (USA), Dr. Laura Ross (Australia), Dr. Virginia Steen (USA), Dr. Benjamin Chaigne (France), Dr. Tracy Frech (USA), Dr. Andrea Low (Singapore), Melanie Baniña (study coordinator), Dr. Edward Stern (UK), Alison Hendry (moderator), Kristina Clark (UK), Falguni Desai (patient partner), Dr. Madelon Vonk (Netherlands), Dr. Tatiana Rodriguez (Mexico), Dr. Marie Hudson (Canada), Dr. Patricia Carreira (Spain) and Dr. Lee Shapiro (USA). Absent: Swati Mehta (USA).

References:

1. Hoa S, Stern EP, Denton CP, Hudson M, Scleroderma Clinical Trials Consortium Scleroderma Renal Crisis Working Group Investigators of the Scleroderma Clinical Trials Consortium Scleroderma Renal Crisis Working G. Towards developing criteria for scleroderma renal crisis: A scoping review. Autoimmun Rev. 2017;16(4):407-15.

2. Butler EA, Baron M, Fogo AB, Frech T, Ghossein C, Hachulla E, et al. Generation of a core set of items to develop classification criteria for Scleroderma renal crisis using consensus methodology. Arthritis Rheumatol. 2019;71(6):964-71.