About the Pilot Test and ORCHA:
ACP is conducting this pilot test of a limited database of mobile health apps which have been assessed and judged to be high-quality, safe and effective. One of ACP’s goals is to help physicians become more comfortable in using digital health tools with patients, and this pilot test seeks to learn whether this app library is good and a helpful step towards that goal. We would like to understand how a library such as this may be used by internal medicine physicians, and the potential value of this library to ACP members.
The apps in this library were all assessed and scored by ORCHA (the Organization for the Review of Care and Health Apps) against the recently-published Digital Health Assessment Framework developed jointly by ACP, the American Telemedicine Association, and ORCHA. ORCHA has evolved frameworks for the deployment of digital health tools in the UK and across Europe and the Middle East, and their experience formed the basis for adapting global best practices to support US regulations, guidelines and expectations. A series of workshops was conducted with relevant experts to identify the appropriate assessment criteria for the US. ACP members primarily reviewed and revised the clinical assurance and usability criteria, while the ATA focused on gathering input regarding the technical criteria and usability. ACP greatly appreciates the leadership role the ATA played to bring the DHAF into existence.
ORCHA conducts the app assessment and hosts this app library. Working for numerous national health services and other organizations around the world, ORCHA has completed over 17,000 assessments on 7,000 apps.
ACP will be soliciting the feedback of pilot test participants during the test period, and we appreciate you taking the time to answer a few brief surveys when asked.
Digital Health Assessment Framework (DHAF) and Assessment Process: The DHAF is a comprehensive set of criteria intended to identify high-quality, safe and effective digital health tools (or apps).
Key Components of DHAF:Data & Privacy
- Adherence to federal laws designed to protect the rights and freedoms of individuals.
- A deeper dive into laws that are specific to individual products, such as the need for HIPAA compliance and COPPA
Clinical Assurance & Safety
- Is the app safe and effective? Does the content draw on high-quality clinical guidelines (including ACP Clinical Guidelines, USPSTF Guidelines, and similar) or other respected sources?
- For patient safety, is the app a medical device as outline by FDA Digital Health Requirements?
- Find out more about the adapted Evidence Standards Framework here.
Usability & Accessibility
- Design and development; accessibility; usability; user support
- Compliance with design standards
Technical Security & Stability
- App was built in compliance with the secure coding practice guidelines of the OWASP (Open Web Application Security Project)
The ORCHA Assessment Development Team have translated the DHAF evaluation criteria into a set of objective questions that trained ORCHA Assessors are looking to answer.
Our Assessors are guided through each assessment through our online assessment tool and this ensures that all the relevant questions are investigated. Where our Assessors encounter scenarios that the Assessment Development Team haven’t specified, the Assessors refer these back to the domain lead of the Assessment Development Team for guidance.
For the ACP app library, Apps that are classified as Tiers Bii, Ci and Cii all go through an Enhanced Clinical Assessment process conducted by physicians and other healthcare professionals with expertise related to the App category. Through this process the content sources, published evidence and use cases are assessed, and the clinical assessor generates a subjective summary of the app.
