ModuleMD 2023 Real World Testing Plan
Executive Summary
This is the Real World Testing plan for CY 2023 for ModuleMD’s certified EHR solution.
As ONC has stated in its rule, “The objective of real-world testing is to verify the extent to which certified health IT deployed in operational production settings is demonstrating continued compliance to certification criteria and functioning with the intended use cases as part of the overall maintenance of a health IT’s certification.” In accordance with the ONC’s recommendation, ModuleMD intends to conduct Real World Testing to verify that our Certified EHR continues to perform as intended with regard to interoperability and data exchange criteria. This test plan focuses on demonstrating that certified capability is utilized in real world care settings.
Health IT Developers have been given maximum flexibility to develop innovative plans and measures for Real World Testing. As we develop a plan to execute Real World Testing, the overall complexity of the workflows and use cases within the care settings are being considered in which ModuleMD’s products are deployed.
We have included our milestones and timeline for completing Real World Testing below.
Developer Attestation
This Real-World Testing plan is complete with all required elements, including measures that address all certification criteria and care settings. All information in this plan is up to date and fully addresses the health IT developer’s Real World Testing requirements
| Authorized Representative Name: | Miranda Moore |
| Authorized Representative Email: | miranda.moore@modulemd.com |
| Authorized Representative Phone: | 248.434.0444 x1230 |
| Authorized Representative Signature: |  |
| Date: | 10/20/2022 |
General Information
Plan Report ID Number:MMD_RWT_2023
Developer Name: ModuleMD
Product Name(s): ModuleMD WISE™
Version Number(s): 9.1
Certified Health IT Product List (CHPL) ID(s):
| CHPL Product Number | CHPL Product Number | CHPL URL |
| 9.1 | 15.04.04.2980.Modu.09.00.1.171201 | https://chpl.healthit.gov/#/listing/8979 |
Timeline and Key Milestones
| Key Milestone | Date / Time Frame |
| Develop a list of clients to assist with Real World Testing | Q1 2023 |
| Preparation and scheduling testing with clients | Q1 2023 |
| Collection of information as laid out by the plan | Q2 and Q3 2023 |
| CY 2023 Real World Testing plan will be completed according to ONC and ONC-ACB requirements and expectations. | Q4 2023 |
| Document our CY 2023 test results | December 2023 |
| Submit Real World Testing Report to ONC-ACB | January 2024 |
Standards Updates
| Standard (and version) | N/A |
| Updated certification criteria and associated product | N/A |
| Health IT Module CHPL ID | N/A |
| Method used for standard update | N/A |
| Date of ONC-ACB notification | N/A |
| Date of customer notification (SVAP only) | N/A |
| Conformance measure | N/A |
| USCDI-updated certification criteria (and USCDI version) | N/A |
Justification for Real World Testing
Real World Testing is intended to verify that deployed Certified Health IT continues to perform as intended by demonstrating that certified capability for interoperability and data exchange is successfully utilized in the real world.
All measures reasonably align with the elements within the Real World Testing plan, the scope of the certification, the types of settings in which the certified health IT is marketed, and other factors relevant to the implementation of the certified Health IT Module(s). Each element within the plan is relevant to the overall strategy for meeting the Real World Testing condition and maintenance of Certification requirements.
In each measurement description, we elaborate specifically on our justification for choosing this measure and the expected outcomes. Each measurement was chosen to best evaluate compliance with the certification criteria and interoperability of exchanging electronic health information (EHI) within the certified EHR.
Measures Used in Overall Approach
For each measurement/metric, describe the elements below:
- Associated certification criteria
- Description of each measurement/metric
- Justification for each measurement/metric
- Expected Outcomes for each measurement/metric
- Care Settings Targeted will be ambulatory care settings
We will be using reporting methods and screenshots to capture and demonstrate compliance. This methodology uses the logging or reporting capabilities of the EHR to examine functionality performed in the system over a given interval. This measurement was chosen to best evaluate compliance with the certification criteria and interoperability of exchanging electronic health information (EHI) within the certified EHR.
In instances where no or little evidence exists due to lack of adoption of a certified measure or other reasons, we will demonstrate the requirements in a semi-controlled setting in a production site, otherwise known as a “real world” environment.
Care Settings Targeted
ModuleMD is an electronic medical record system that supports medical clinics in an ambulatory care setting. All measures outlined in the Real World Testing are designed for and will be performed within the ambulatory care setting.
Real World Testing: Care Coordination
Measure 1
| Associated Certification Criteria: | 170.315 (b)(1)cures: Transitions of Care |
| Measurement: | Rate of referral patients received by the specialist for all unique patient visits. |
| Description: | During the testing period, a report will be generated which tracks how many patients have been referred by a primary care provider to a Specialist over the course of a given interval and the information portability using CCDA using Direct Message or direct uploads to EHR. This report also considers patient records that have been received by the specialist using EHR’s secured E-Fax. |
| Additional Software: | EMR Direct/phiMail server |
| Justification: | Coordination of care between primary care provider and specialist is important and it involves testing of information portability between two different systems. We will report clearly in the denominator, the number of unique patients who visited the specialist and with a numerator, the number of those patients whose information was incorporated successfully in our system for the continuity of care. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria, supporting CCDA 2.1 with USCDI standards using standard vocabulary sets in transition of care and demonstrate the exchanging of electronic health information (EHI) in the care and practice settings for which it is marketed. |
Measure 2
| Associated Certification Criteria: | 170.315 (b)(2)cures: Clinical Information Reconciliation and Incorporation |
| Measurement: | Rate of unique patient visits for which reconciliations of discrete data into the patient’s medication list, or allergy and intolerances list, or problem list were successful. |
| Description: | During the testing period, we will report clearly in the denominator, the number of unique patients who visited a clinic, and with a numerator, the report data reconciliation per unique patient. |
| Justification: | We will report clearly in the denominator, the number of unique patients who visited a clinic, and with a numerator, the report data reconciliation per unique patient. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria and demonstrate that a user can simultaneously display a patient’s active data, and its attributes, from at least two of the following sources: a patient’s medication list, allergies and intolerances list, and problem list. |
Measure 3
| Associated Certification Criteria: | 170.315 (b)(3)cures: Electronic Prescribing |
| Measurement: | Rate of unique patients who received an electronic prescription ordered by the provider. |
| Description: | During the testing period, a report will be generated which tracks how many patients have received electronic prescriptions ordered by the provider over the course of a given interval. |
| Additional Software: | DrFirst Rcopia |
| Justification: | We will report clearly in the denominator, the number of unique patients who visited the clinic and with a numerator, the number of those patients who received electronic prescriptions ordered by the provider. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria and demonstrate the orders received by Pharmacy and data exchange with Surescripts as a hub. |
Measure 4
| Associated Certification Criteria: | 170.315 (b)(6): Data Export |
| Measurement: | Rate of unique patients whose Clinical Summaries have been exported from the EHR. |
| Description: | During the testing period, a report will be generated which tracks how many clinical summaries of patients have been exported from EHR. |
| Justification: | We will clearly report the list of patients whose clinical summaries have been exported from our EHR. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria, supporting CCDA 2.1 with USCDI standards using standard vocabulary sets in transition of care and demonstrate the exchanging of electronic health information (EHI) in the care and practice settings for which it is marketed. |
Measure 5
| Associated Certification Criteria: | 170.315 (b)(9): Care Plan |
| Measurement: | Rate of unique patients whose Care Plan summaries have been imported into Patient Records with CCDA via Direct Message or direct uploads. |
| Description: | During the testing period, a patient referred by a primary care provider to a specialist will be exporting their care plan document from their system. Using the information portability of CCDA transmitted via Direct message or direct uploads to EHR, the primary care provider will be able to view/incorporate the care plan details into their patient record. |
| Justification: | We will take screenshots of the Care plan document (blurring the details of the patient identity) and show the evidence of the Care Plan and its portability from/to our system. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria, supporting CCDA 2.1 with USCDI standards using standard vocabulary sets in recording the Care plan and demonstrate the exchanging of electronic health information (EHI) in the care and practice settings for which it is marketed. |
Real World Testing: Clinical Quality Measures
Measure 6
| Associated Certification Criteria: | 170.315 (c)(1): Clinical Quality Measures - Record and Export |
| Measurement: | Rate of unique patients who were referred to a specialist and waiting to receive the report back. |
| Description: | During the testing period, a patient referred by a primary care provider to a specialist will be exporting the QRDA Category I file corresponding to CMS50v9 from their system. |
| Justification: | We will show the screenshot of PCP’s system showing the CQM calculator having the patient in the denominator but not present in the numerator yet. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria, supporting QRDA Category 1 export using standard vocabulary sets in recording the specialist provider and primary care provider information. Demonstrate the exchanging of electronic health information (EHI) in the care and practice settings for which it is marketed. |
Measure 7
| Associated Certification Criteria: | 170.315 (c)(2): Clinical Quality Measures - Import and Calculate |
| Measurement: | Rate of unique patients who were referred by a PCP to a specialist and ready to send the report back to the PCP. |
| Description: | Using the information portability of QRDA transmitted via direct uploads to PCP’s EHR, we will be able to view/incorporate the closing of the referral loop with patient details and notes from a specialist. Demonstrate the CQM calculator showing the patient in calculations for CMS50v9: Closing the Referral Loop: Receipt of Specialist Report. |
| Justification: | We will show the screenshot of PCP’s system showing the CQM calculator having the patient in the denominator and also in numerator now as evidence for closing the referral loop. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria, supporting QRDA Category 1 import using standard vocabulary sets in recording the specialist provider and primary care provider information. Demonstrate the exchanging of electronic health information (EHI) in the care and practice settings for which it is marketed. |
Measure 8
| Associated Certification Criteria: | 170.315 (c)(3): Clinical Quality Measures – Report |
| Measurement: | Rate of unique patients who were referred to a specialist and received the report back. |
| Description: | Demonstrate the CQM calculator showing the patient in calculations for CMS50v9: Closing the Referral Loop: Receipt of Specialist Report and export the results in QRDA Category III. |
| Justification: | Using the CQM calculator, we will demonstrate our reporting/exporting capabilities for CMS50v9: Closing the Referral Loop: Receipt of Specialist Report. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria, supporting QRDA Category III export for reporting of the CMS50v9: Closing the Referral Loop: Receipt of Specialist Report. Demonstrate the exchanging of electronic health information (EHI) in the care and practice settings for which it is marketed. |
Real World Testing: Patient Engagement
Measure 9
| Associated Certification Criteria: | 170.315 (e)(1)cures: View, Download, and Transmit to 3rd Party |
| Measurement: | Rate of unique patients who were able to access the patient portal to view, download, and also transmit to a third party. |
| Description: | During the testing period, a report will be generated which tracks how many patients have access to their portal account over the course of a given interval and what information is shared with the patient portal. |
| Justification: | We will report clearly in the denominator, the number of unique patients who are registered patient portal users in a clinic and with a numerator, the number of those patients who have viewed/downloaded/transmitted their clinical summaries to a third party. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria and demonstrate how patients engaged with the usage of the patient portal. |
Real World Testing: Electronic Exchange
Measure 10
| Associated Certification Criteria: | 170.315 (h)(1): Direct Project |
| Measurement: | Count the number of direct messages received and sent in the EHR. |
| Description: | During the testing period, a report will be generated with the count of direct messages sent and received by the clinic in a real world setting. |
| Additional Software: | EMR Direct/phiMail server |
| Justification: | We will report the direct messaging capability as electronic exchange. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria, supporting Direct Project protocols and capabilities. |
Real World Testing: Public Health
Measure 11
| Associated Certification Criteria: | 170.315 (f)(1): Transmission to Immunization Registries |
| Measurement: | Rate of unique patients in the clinic whose immunization records have been transferred to immunization registries successfully. |
| Description: | During the testing period, a report will be generated which tracks how many immunization records have been transmitted to their state immunization registries over the course of a given interval. |
| Justification: | We will report clearly in the denominator, the number of unique patients who have visited the clinic and with a numerator, the number of those patients whose immunization records have been updated in EHR and transmitted to their state immunization registries. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria and demonstrate how many immunization records are transmitted to the registries. |
Measure 12
| Associated Certification Criteria: | 170.315 (f)(2): Transmission to Public Health Agencies – Syndromic Surveillance |
| Measurement: | Rate of unique patients in a clinic whose records have been transferred to an HIE in case of any outbreaks of interest. |
| Description: | During the testing period, a report will be generated which tracks how many patient records with ICD-10s have been transmitted to their registries over the course of a given interval. |
| Justification: | We will report clearly in the denominator, the number of unique patients who have visited the clinic and with a numerator, the number of those patients whose records have been transmitted to their registries. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria and demonstrate how many records are transmitted to the registries. |
Measure 13
| Associated Certification Criteria: | 170.315 (f)(3): Transmission to Public Health Agencies - Reportable Laboratory Tests and Values/Results |
| Measurement: | Rate of unique patients in a clinic whose lab results have been transferred to an HIE. |
| Description: | During the testing period, a report will be generated which tracks how many lab results have been transmitted to their registries over the course of a given interval. |
| Justification: | We will report clearly in the denominator, the number of unique patients who have visited the clinic and with a numerator, the number of those patients whose lab results have been transmitted to their registries. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria and demonstrate how many lab records are transmitted to the registries. |
Real World Testing: Application Programming Interfaces
Measure 14
| Associated Certification Criteria: | 170.315 (g)(7): Application Access - Patient Selection |
| Measurement: | Count the number of API queries for a patient search made in a real world setting. |
| Description: | During the testing period, a report will be generated which tracks how many patient searches have been done by OpenAPI queries in the real world. |
| Justification: | OpenAPI queries result in the selection of a patient and demonstrate the audit log of search or selection of patient. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria and report the OpenAPI queries in selecting patients in a real world setting. |
Measure 15
| Associated Certification Criteria: | 170.315 (g)(9)cures: Application Access - All Data Request |
| Measurement: | Count the number of API queries for generating all data requests made by patients used in a real world setting. |
| Description: | During the testing period, a report will be generated which tracks how many all-data requests have been done by OpenAPI queries. |
| Justification: | OpenAPI queries result in the selection of a patient and demonstrate the audit log of search or selection of patient and make all-data request. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria and synchronize the OpenAPI query for a patient as a subject, to fetch all-data request in real world setting. |
Measure 16
| Associated Certification Criteria: | 170.315 g.10cures: Standardized API for patient and population services |
| Measurement: | Count the number of API queries for the category of clinical data set made in a real life setting. |
| Description: | During the testing period, a report will be generated which tracks and log the OpenAPI queries made for specific data category requests in the real world. |
| Justification: | Demonstrate Audit log report of OpenAPI queries with a patient as a subject and fetch that patient-specific data by category. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria and report the OpenAPI queries with a patient as a subject in real world setting. |