Cell network and WIFI connectivity
Clearly good cell network or WIFI connectivity is of primary importance when thinking about using ePRO in a clinical study. Sites should conduct a feasibility activity in their local vicinity during the study initiation phase to understand if cell network strengths are good enough for ePRO and if not, if there is likely to be a large population of subjects that have home WIFI. For remote locations with possible issues with either the cell network coverage or even with the availability of electricity, these sites should be reconsidered before a decision is made to use ePRO.
However, even with bad cell network coverage, it is possible to conduct a study with ePRO by ensuring the correct messaging appears to the subjects on the device. All ePRO systems allow the subjects to access and complete their diaries on a daily basis without cell or WIFI connectivity. The data entered by the subject is stored on the device until connectivity is established. There are circumstances where the site may need to be made aware of certain responses in real time in order to maintain subject safety. If connectivity is established and the data is sent in real time, the site staff may receive an alert from the system asking them to contact the participant. In order to ensure participant safety is maintained it is good practice to alert the participant to contact the sites / study doctor if certain responses in the diary are provided, such as if the participant required emergency medical attention.
Translations
As mentioned earlier in an earlier instalment, translation is an aspect not typically associated with clinical systems such as EDC. Additionally, because the ePRO devices are in the hands of the participants, the screens, in their local languages, must be submitted to the local ethics committees within the countries that the study is to take place in.
It should not be forgotten that paper PRO is also subject to a translation process. However, the additional complexity with ePRO is the need to apply the translated text to the software in order to generate the questionnaire screens. This involves multiple review and update rounds between the ePRO vendor and the translation vendor which increases time, effort and therefore cost. Once the screens are generated there may be further review rounds during the in-country review between the local sponsor representatives, ePRO vendor and translation vendor. All of these review rounds increase the timescales for an activity that is already time critical.
Often the date for submission to the ethics committee is set during the planning phase of the study by the sponsor study team. If the submission date is missed it could result in a delay to the study start. Consequently, it is important to ensure clear timelines are in place between agreeing the ePRO requirements, which will decide what is displayed on each of the screens, and submitting the screen report to the ethics committees. The ePRO vendor will need to be made aware of these timelines at the earliest opportunity. It can take more than 12 weeks for the completion of the translations resulting in the creation of the finalized screen reports. These timelines are difficult to manage if the process includes an in-country review round which allows for local country representatives from the sponsor to review the translations. It is important to make clear to the reviewers what it is that these stakeholders are reviewing. Translation is not an exact science. There are many ways of writing the same sentence. As the translator is the trained expert in translations it should be left to them to choose the most appropriate wording in the local language which most faithfully represents the English version. The in-country review, if indeed one is required, should only be conducted to ensure the correct screens are displayed in the screen reports. Allowing the in-country reviewer to make suggestions on preferred wording risks multiple back and forth review rounds, increasing timelines and jeopardizing study start dates.
End User Acceptance
As briefly mentioned in part one of this series, possibly the most crucial aspect when introducing a new system or process to a user base is change management. If you do not bring your users along the journey with you, you are less likely to gain their acceptance.
How does this manifest itself? With ePRO there is plenty of opportunity for issues to arise, from delays in getting devices to sites, to errors in the software, to usability and connectivity issues. All of which can affect the investigators ability to get on with their daily work. If you have not put in place a good change management process you will find the investigators very quickly become disenchanted with the system and even the smallest of issues will become magnified, resulting in escalations to the sponsors senior management team.
It is important from the outset to set the stage with the investigators. Why are we using ePRO? What benefits does it bring to the sponsor? It may require more work on behalf of the investigator which will need to be compensated for. It is also about setting expectations. Things will go wrong, issues will need to be resolved, backup processes may need to be utilized, but the investigator must be aware that they have the support when needed and how they can access that support.
When implementing ePRO thought should go into understanding how to improve the investigators experience. For example, investigators can be working on multiple studies at the same time for different sponsors all using ePRO, so adding labels onto the packaging of shipped devices so that investigators can quickly store them together is a quick win.
As a sponsor it may also be necessary to implement an additional layer of support for the study team. When issues arise the investigator will contact the vendor helpdesk. Often the helpdesk are unable to provide an immediate remedy so the investigator will then contact the sponsor’s study team representatives. The additional layer of support sits between the study team and the vendor, collating issues, communicating technical information back and forth in a manner that is easily digestible and holding the ePRO vendor to account. This relieves the frustration experienced within the sponsor’s study team and reduces the likelihood of escalation.
Qualification/Validation
It is important to remember that in accordance with ICH GCP guidance section 5.2.1 [1] the sponsor may transfer trial-related duties to a vendor, however the ultimate responsibility for the quality and data integrity of the trial data always resides with the sponsor. In order to ensure data is collected, stored and transferred in accordance with ICH-GCP guidance the sponsor will need a suitable oversight strategy of the vendor’s processes. This may, at a minimum, include a qualification activity which involves auditing the ePRO vendor on a regular basis. It may also invoke some internal validation work to ensure, as the sponsor, your own organisation’s technical and study specific requirements for clinical systems are met.
It is not recommended for the sponsor to leave the responsibility of vendor oversight to the individual study teams. The study teams may not have the experience or technical know-how necessary to understand the challenges and considerations discussed earlier in this blog.
We would recommend a two stage process.
Stage one is the qualification of the vendors system by a centralized team including stakeholders from the data management and computer system quality departments. Once qualified, the vendor’s system can be classed as a platform that can subsequently be used on multiple studies.
Stage two is to have experienced stakeholders conduct study specific User Acceptance Testing (UAT) in order to test the many scenarios and nuances associated with ePRO and to ensure the system meets the requirements of the study Protocol. Once implemented and running live on a study it is far more difficult to update the system than finding and fixing the issue during the implementation phase.
However, UAT is the very least the Sponsor must do as part of the overall study specific validation activity (Stage two). The Sponsor’s quality management system (QMS) may also mandate that the implementation of any clinical system (which would include ePRO) requires internal validation documentation to supplement the vendor’s validation package. This can include documents such as a sponsor Validation Plan, Requirements Specification, Risk Assessment, Traceability Matrix and Validation Report.
ePRO vendors will often offer to provide a service to create UAT script on behalf of the Sponsor, however this is not recommended as a misunderstanding of requirements by the vendor may also manifest itself in the scripting. This can result in the script passing when executed when in fact it does not meet the requirements of the Sponsor.
For these reasons, having experienced validation professionals generating the deliverables and executing the testing is recommended.
Summary
The two parts of this blog have detailed, at a high level, many of the challenges that can be experienced during the implementation and use of ePRO. These challenges are summarized below:
- Cost – create a robust business case which focuses on the intangible benefits and areas where cost reduction is possible
- Software – try to standardize your requirements wherever possible. Request the ePRO vendor to maintain a library of questionnaires
- Hardware – Work with the ePRO vendor to determined numbers of devices required for per study when using a provisioned model. Wherever possible, use Bring Your Own Device (BYOD)
- Cell network and WIFI connectivity – Clinical sites should conduct a feasibility activity before the study commences to determine levels of cell network connectivity in the region and decide on course of action. Additionally, messaging should be applied to the ePRO device to ensure the subject contacts the clinical site or study doctor if necessary.
- Translations – Ensure clear timelines are communicated with the ePRO vendor at the initial kick off meeting, including deadlines for ethics submission. Rely on the translation vendor to provide quality translations and do not risk missing ethics submissions deadlines by conducting detailed in-country reviews.
- End User Acceptance – Set expectations with all stakeholders upfront, especially study teams and investigator staff. Explain the benefits and the good reasons for using ePRO. Put in place an additional layer of internal support if required.
- Validation/Qualification – Qualify the ePRO vendor and their system. Deploy an experienced internal team to execute UAT, at the very least, and create formal validation deliverables if required by the sponsors QMS.
These challenges, and others, often create frustration for the investigators and study teams, and in the worst case scenarios, studies can be put on hold or stopped completely, costing millions of dollars and preventing a product from going to market.
A mechanism to help mitigate, or at the very least support the investigators and study teams through, many of these challenges is for the sponsor to put in place a centralized ePRO team within their organization. This team, consisting of internal sponsor stakeholders or perhaps supplementing Scimcon resources into the team, would be responsible for the qualification of the ePRO vendor, the strategic implementation of libraries and standards and for supporting the implementation of ePRO study-specifically, providing help with creating validation deliverables and executing UAT, and being a second level of support during the conduct phase of a study. ePRO is fast becoming the standard and it is worthwhile investing early on in an internal team to ensure success.
To discuss how Scimcon can support your organization with the implementation of ePRO, please contact one of our consultants for an informal conversation.
References
[1] ICH harmonised guideline integrated addendum to ICH E6(R1): Guideline for Good Clinical Practice ICH E6(R2) ICH Consensus Guideline, https://ichgcp.net