When it comes to documenting the advantages of using ePRO over paper in clinical trials, the benefits are clear.
With all the advantages to using ePRO over paper it seems to be a no-brainer to use ePRO whenever possible. However, it’s important to be mindful of certain considerations and challenges that come with the implementation of ePRO within your organization before jumping in.
Historically the implementation of electronic clinical systems in general has been challenging. In the majority of cases it requires the move from a paper-based process to an electronic system in an environment where the reliance has always been on paper, hindering the adoption of computer systems that are seen as alien. Taking EDC as an example, the response to an international survey cited that 46% of respondents identified inertia or concern with changing current process, and 40% identified resistance from investigative sites as the major causes for adoption delays [2].
ePRO is not immune to these challenges. In fact, it could be argued that ePRO is even more susceptible. While ePRO suffers from the traditional technical issues and user acceptance that EDC experiences, ePRO is also placed in the hands of potentially thousands of study participants many of whom may have little technical understanding. Additionally, ePRO relies on hardware (mobile device or tablet), cell network or WIFI connectivity, translation into the participant’s local language, multiple userbase (study teams, investigators and participants) and local helpdesk support, all of which comes with their own set of challenges and associated costs and few, if any, of which are encountered with EDC.
ePRO is one of the few electronic systems that directly collects source data and as a result comes under increased scrutiny from a data integrity and quality perspective, especially when used for primary or secondary end point data collection. The system must always be available in order to allow subjects to be activated on ePRO devices. If a participant leaves a clinical site without an active device, this can result in missed data which can be construed as a serious quality issue and perhaps put subject safety at risk.
Before I go into the more detailed challenges associated with ePRO, let’s first consider the financial costs.
On the surface of it, it would appear that implementing ePRO is significantly more costly than paper. The expense of the devices, associated logistics and data usage (monthly SIM costs), the licenses, helpdesk and translations all contribute to costs that range from hundreds of thousands of dollars to multi-million-dollar contracts per study.
When making a business case for ePRO it is important to take into account the hidden costs associated with paper in order to compare the two.
When conducting a full assessment, the gap between the cost of implementing ePRO vs paper reduces significantly. ePRO vendors have attempted to provide examples which result in paper diaries actually contributing more cost to a study budget than ePRO.
The business case for implementing ePRO should not be solely based on raw cost. This will likely result in failure to get agreement at the leadership level. You will find it easier to get acceptance if you can prove that ePRO costs are comparable to paper while also concentrating on the non-tangible benefits as, in the case of ePRO, these are the real reasons for its consideration. Increasing the quality of your data collection results in more confidence in that data, which in turn reduces the likelihood of rejection when submitted to the regulators (predominantly for primary and secondary end point data). Receiving the data in real time and reducing the need for data cleaning can aid the ability to get a product to market quicker by shortening the timelines to close the study, which in turn results in cost avoidance.
Many ePRO vendors will provide a cost calculator; a spreadsheet where the sponsor can plug in parameters associated with their study to provide an estimate of costs before engagement with the vendor. Only a small number of parameters are required to calculate a good estimate with the most important being the length of the study in months and the number of participants. The length of the study drives the helpdesk, data usage and PM costs, whereas the number of participants drives the device, logistics and shipping costs. There are other costs associated with the configuration of the system, translation, shipping, number of sites etc, however these are often negligible in comparison for larger studies.
In summary; it is important to build a business case for ePRO within your organization in order to assist in gaining acceptance at a leadership level. The business case should include areas of efficiency over paper together with examples of ePRO costs using the cost calculators provided by the vendors, as well as emphasizing the other benefits of ePRO, such as subject safety, compared to paper solutions.
In the past, ePRO implementations were customized pretty much from the ground up, coding the study specifics into the vendor’s study builder toolkit. This resulted in a huge effort required to validate the system to ensure errors and bugs were captured before studies went live. Inevitably despite all this testing some issues did make it through to the live study, causing frustration for the participants, investigators and study teams.
Over the past decade the systems have become more sophisticated. Less code is required during the implementation phase, which has been replaced with configuration. Vendors have also introduced library functionality which allow sponsors to define questionnaires up front that can be reused across studies. As the questionnaire is not rebuilt every time there is less opportunity to introduce errors. Additionally, the ability to reuse questionnaires from a library also results in less work by the vendor per study, less validation on behalf of the sponsor and can reduce the time and costs during the implementation phase.
It may also be possible to standardize other areas of functionality, perhaps the workflow as to when questionnaires are made available to the participants, or the alerting system, or the visit schedule. It may not be possible to standardize across therapeutic areas, but within a therapeutic area where multiple studies collect the same data this approach can result in substantially reduced timelines during the implementation phase, while reducing the risk of software errors on the studies.
ePRO can be implemented in a number of different modalities. In this blog, we are concentrating on provisioned devices; devices that are provided by the ePRO vendor at a cost to the study Sponsor, and “bring your own device” (BYOD); where a subject’s own device is used as the ePRO instrument. It should be noted that all studies require provisioned devices to a certain degree in order to cater for cases where a subject either does not own a compatible mobile device or does not own a mobile device at all.
When provisioning devices, ePRO vendors are responsible for the associated logistics such as software installation and shipping. Vendors are generally very knowledgeable when it comes to the customs regulations in many countries, including the average timelines required to get a shipment to a site.
In scenarios where competitive recruitment between sites is employed, it is particularly important to plan ahead. As it may not be possible to predict the number of subjects that will be recruited at a specific site and therefore the number of devices required at that site, it is necessary to purchase a sensible overage of provisioned devices. Although costly it ensures sites will not run out of devices.
With an increasing number of the world’s population now owning smart phones, BYOD was seen as the natural progression for ePRO. It reduces the costs and burden of acquiring devices and the associated logistics and also reduces the monthly costs for data usage. These costs do not completely disappear as a certain level of provisioning is required for those cases where participants don’t own a compatible smart phone. BYOD also reduces risks associated with not having enough devices on site, especially, as mentioned above, with competitive recruitment. However, BYOD does come with own set of unique challenges, mainly associated with data integrity and privacy. Some considerations might be:
There are clearly a lot of benefits of using BYOD over provisioned devices with more and more sponsors feeling comfortable moving into this space, however it is important to consider the implications before doing so.
In the next instalment of this blog, we will discuss some other challenges of implementing ePRO in your organisation, such as connectivity, translations, and end user acceptance testing. Keep an eye on our Opinion page for part two of the series, coming soon.
[1] ‘Electronic for Industry – Electronic Source Data in Clinical Investigations’, FDA 2013 http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm328691.pdf
[2] ‘Welker JA. Implementation of electronic data capture systems: barriers and solutions.’ Contemp Clin Trials. 2007 May;28(3):329-36. doi: 10.1016/j.cct.2007.01.001. Epub 2007 Jan 11. PMID: 17287151.
