Validation and equivalency of electronic clinical outcomes assessment systems
Keywords:Validation, Equivalence, BYOD, eCOA, ePRO, Cognitive debriefing/usability testing
The US food and drug administration (FDA) has long called for clinical outcomes assessments (COA), such as patient-reported outcomes (PRO), to be ‘fit-for-purpose’ meaning the COA has been validated to support the context of use. The FDA’s recent patient-focused drug development guidance series has renewed the importance of ensuring that COA is ‘fit-for-purpose’ and valid. In addition, the FDA has recommended that COA be collected electronically and that the electronic (eCOA) system and devices also be validated. Advancing technology requires eCOA systems and devices to evolve; eCOA devices may change over time. As bring your own device (BYOD) models gain popularity and acceptance, devices may also be mixed within trials. Changes in eCOA devices or mixing devices may require equivalence testing to prove validity across platforms. The aim of this article is to provide an overview of the different types of validation at both the assessment level and the eCOA system (device) level to help clinical trial sponsors determine the appropriate level of validation or equivalence testing required for COA used in their clinical trials.
US food and drug administration. Guidance for industry: Patient-reported outcome measures: Use in medical product development to support labeling claims. U.S. department of health and human services, food and drug administration, center for drug evaluation and research (CDER), Center for biologics evaluation and research (CBER), Center for devices and radiological health (CDRH); 2009. Available at: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-reported-outcome-measures-use-medical-product-development-support-labeling-claims. Accessed on 28 June 2020.
US Food and drug administration. Discussion document for patient-focused drug development public workshop on guidance 3: select, develop or modify fit-for-purpose clinical outcome assessments. U.S. department of health and human services, food and drug administration, center for drug evaluation and research (CDER), Center for biologics evaluation and research (CBER); 2018. Available at: https://www.fda.gov/media/116277/download. Accessed on 28 June 2020.
Zbrozek A, Hebert J, Gogates G, Thorell R, Dell C, Molsen E et al. Validation of electronic systems to collect patient-reported outcome (PRO) data recommendations for clinical trial teams: Report of the ISPOR ePRO systems validation good research practices task force. Value Heal. 2013;16(4):480-9.
US food and drug administration. FDA patient-focused drug development guidance series for enhancing the incorporation of the patient’s voice in medical product development and regulatory decision making. U.S. department of health and human services, food and drug administration, Center for drug evaluation and research (CDER), Center for biologics evaluation and research (CBER); 2019. Available at: https://www.fda.gov/ drugs/development-approval-process-drugs/fda-patient-focused-drug-development-guidance-series-enhancing-incorporation-patients-voice-medical. Accessed on 28 June 2020.
European medicines agency (EMA). Reflection paper on the regulatory guidance for the use of health related quality of life (HRQL) measures in the evaluation of medicinal products. 2005. Available at: https://www.ema.europa.eu/en/regulatory-guidance-use-health-related-quality-life-hrql-measures-evaluation-medicinal-products. Accessed on 28 June 2020.
European Medicines Agency (EMA). Appendix 2 to the guideline on the evaluation of anticancer medicinal products in man the use of patient-reported outcome (PRO) measures in oncology studies. 2016. Available at: https://www.ema.europa.eu/en/ appendix-2-guideline-evaluation-anticancer-medicinal-products-man-use-patient-reported-outcome-pro. Accessed on 28 June 2020.
Souza AC de, Alexandre NMC, Guirardello E de B. Psychometric properties in instruments evaluation of reliability and validity. Epidemiol e Serv saude Rev do Sist Unico Saude do Bras. 2017;26(3):649-59.
Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value Heal. 2009;12(4):419-29.
Muehlhausen W, Byrom B, Skerritt B, McCarthy M, McDowell B, Sohn J. Standards for instrument migration when implementing paper patient-reported outcome instruments electronically: recommendations from a qualitative synthesis of cognitive interview and usability studies. Value heal. 2018;21(1):41-8.
Byrom B, Gwaltney C, Slagle A, Gnanasakthy A, Muehlhausen W. Measurement Equivalence of Patient-Reported Outcome Measures Migrated to Electronic Formats: A Review of Evidence and Recommendations for Clinical Trials and Bring Your Own Device. Ther Innov Regul Ici. 2019;53(4):426-30.
Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: Report of the ISPOR task force for translation and cultural adaptation. Value heal. 2005;8(2):94-104.
US Food and Drug Administration. Part 11, electronic records; electronic signatures-Scope and application. U.S. department of health and human services food and drug administration, Center for drug evaluation and research (CDER), Center for biologics evaluation and research (CBER), Center for devices and radiological health (CDRH), Center for Food Safety and A; 2003. Available at: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/part-11-electronic-records-electronic-signatures-scope-and-application. Accessed on 28 June 2020.
Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: A meta-analytic review. Value Heal. 2008;11(2):322-33.
Muehlhausen W, Doll H, Quadri N, Fordham B, O’Donohoe P, Dogar N et al. Equivalence of electronic and paper administration of patient-reported outcome measures: A systematic review and meta-analysis of studies conducted between 2007 and 2013. Health qual life outcomes. 2015;13(1):167
Gwaltney C, Coons SJ, O’Donohoe P, O’Gorman H, Denomey M, Howry C et al. Bring your own device (BYOD): The future of field-based patient-reported outcome data collection in clinical trials. Ther innov regul Sci. 2015;49(6):783-91.
Sandborn P. Design for obsolescence risk management. In: Procedia CIRP. Elsevier BV. 2013;11:15-22.
Aladeojebi TK. Planned Obsolescence. Int J Sci Eng Res. 2013;4(6):1504-8.
Amankwah-Amoah J. Integrated vs. Add-On: A multidimensional conceptualisation of technology obsolescence. SSRN. Published online 2017. https://ssrn.com/abstract=2876252. Accessed on 23 March 2020.