Standards

Dating is hard, especially if you’re not aware of the relevant conventions and etiquette. The same could be said for collecting date information in your EDC. As technologists enamored with data above all, those of us here at OpenClinica probably aren’t your best source for romantic advice. But if you’re searching for the most efficient way to capture unambiguous and properly formatted date information in your eCRF, prepare to swoon. Here are four tips for working with dates.

Tip #1: Allow for full and partial dates.

Requiring a full date when only a certain month or year are known to a CRC or participant is a major hazard for analysis. If that full date field is required, it’s quite possible that the user will select a placeholder for day of the month–the 1st or 15th, say–when that piece of information is unavailable to her. The value in the database, then, implies a level of specificity that wasn’t intended.

To avoid this pitfall, ensure that the individual entering the date indicates which portions are known and which are unknown (“UNK”). Then, offer the corresponding field for input.

During analysis, your statisticians will need to convert entered partial dates into imputed ones, using clear and consistent rules. So it’s important that your EDC support aggregating full and partial dates into a single item.

Tip #2: Offer a user-friendly UI that boosts accuracy.

Suppose that a participant knows that she took a dose of medication on the first Monday of the month. You don’t want her to leave the ePRO form in search of a calendar to retrieve that date. Similarly, for a CRC who prefers point-and-click wherever possible, you want a UI that works with her style, to encourage prompt entry. The same is true for a CRC who prefers to type.

Human factors like these impact both data quality and speed. That’s why they are a major design consideration for OpenClinica, and ought to be for you, as well. The datepicker shown here is the clear and flexible standard we rely on, supporting point-and-click, typed entry, and convenient scrolling through months, years, and even decades. Make sure your mechanism offers the same virtues.

Tip #3: Let your forms do the math.

Exclusion criteria for some trials require no history of a certain condition for a specified amount of time; for example, no history of melanoma for the last five years. In this case, performing the “date math” mentally may not pose a big challenge. But the situation becomes more complex when, rather than being disqualified, a potential participant is assigned to a particular cohort based on the date of some clinical event (or even multiple events). Error-free calculation then becomes as difficult as it is paramount. A capable form engine is your saving grace in these situations. Your EDC ought support real-time calculations, and respond in a protocol-compliant way based on the results. Use your system’s calculation and logic functions to hide or require certain fields, enforce eligibility rules, makes assignments, and otherwise ensure proper workflows.

For optimal viewing, expand the video to fill size using the arrows icon in the lower right-hand corner.

The example shown here employs skip logic and date math to assign a cohort based on the date that a CT scan confirmed the absence of melanoma. Three years or more of complete response triggers assignment to cohort A, less than three years to cohort B.

Tip #4: Follow the standards.

The final step when working with dates is to format them in accordance with a recognized standard. CDISC is arguably the most recognizable and robust. Following ISO 8601 standards, CDISC takes YYYY-MM-DD as its date format.

Want to maximize data quality and speed? Focus on your forms. From layout to item order, skip logic to edit checks, there are no “minor factors” when it comes to getting clean data from the start.

Understand these factors by attending our webinar, “Good Form: Designing Your eCRFs for Better Data, Faster,” taking place Monday, December 3 at 11am EST (4pm UTC). You’ll learn to look at your eCRFs through the eyes of your CRCs, monitors, and participants. You’ll see proven best practices in eCRF design that minimize queries and time to entry, while maximizing the integration potential of your data through standardization.

If designing study forms is among your responsibilities, you can’t afford to miss this webinar. Topics include:

The optimal form layout for different users, settings, and data types

Edit check strategies (e.g. how strict is too strict)

When to use pick lists, multi-select, radio buttons, likert scales, and more

In the previous post, I described the difference between efficacy and effectiveness, an increasingly important concept in clinical research and healthcare. After stressing the importance of effectiveness research to health policy planning and patient decision-making, I summarized seven criteria for identifying effectiveness studies. Finally, I asked whether these criteria could be re-purposed beyond a medical intervention to inform how we measure the effectiveness of software systems used to conduct clinical trials.

Is it possible to assess clinical trial software through the lens of effectiveness, as opposed to just efficacy?

I believe that it’s not only possible, but crucial. Why? We all want to reduce the time and cost it takes to deliver safe, effective drugs to those that need them. But if we don’t scrutinize our tools for doing so, we risk letting the status quo impede our progress. When lives are on the line, we can’t afford to let any inefficiency stand.

In this post, I adapt the criteria for effectiveness studies in clinical research into a methodology for evaluating the effectiveness of clinical research software. I limit the scope of adaptation to electronic data capture (EDC) systems, but I suspect that a similar methodology could be developed for CTMS, IVR, eTMF and other complementary technologies. If I open a field of inquiry, or even just broaden one that exists, I’ll consider it time well spent.

Are you measuring all the relevant outcomes of your clinical trial technology?

For pure pathogen-killing power, it’s hard to beat a surgeon’s hand scrub. Ask any clinician, and she’ll tell you how thoroughly chlorhexidine disinfects skin. If she’s a microbiologist, she’ll even explain to you the biocide’s mechanism of action–provided you’re still listening. But how would the practice fare, say, as a method of cold and flu prevention on a college campus? Your skepticism here would seem justified. After all, it’s hard to sterilize a cough in the dining hall.

Efficacy and effectiveness. It’s unfortunate their phonetics are so close, because while the terms do refer to relative locations along a continuum, they’re the furthest thing from synonyms, as the ever accumulating literature on the topic will attest.

In this post and the one that follows, I’d like to offer some clarity on efficacy vs. effectiveness and illustrate the value that each type of analysis offers. If nothing else, what emerges should provide an introduction to the concepts for those new to clinical research. But I have a more speculative aim, too. I’d like propose standards for assessing trial technology through each of these lenses. Why? Because while we’ve been asking whether a particular technology does what it’s explicitly designed to do, as we should and must, we may have forgotten to ask a critical follow-up question: Does it improve the pace and reliability of our research?

In some cases, the display of your OpenClinica eCRF may not be exactly what you had in mind. You may want to highlight key words or phrases, create a bullet point list, or insert a URL or image. Using HTML tags, you can make some simple manipulations to change the look and feel of your case report forms and make them more inviting for data entry.

The HTML tags described in this document can be used in the following columns in the CRF Excel template:

Items Tab: LEFT_ITEM_TEXT

Items Tab: RIGHT_ITEM_TEXT

Items Tab: HEADER

Items Tab: SUBHEADER

Sections Tab: INSTRUCTIONS

What are HTML tags?

HTML, or Hyper Text Markup Language, is a markup language that is commonly used for web page development. HTML is written using “tags” that surround text or elements. These tags typically come in pairs, with a start tag and an end tag:

<start tag>Text to format</end tag>

To insert an HTML tag, simply surround the text you want to format with the desired tag. Below are the HTML tags that work in OpenClinica:

The examples included in the above CRF Excel template will insert an image that already exists in the images directory of your OpenClinica application. To insert a custom image, community users will need to place the image in the following directory of the OpenClinica application:

tomcatwebappsOpenClinicaimages

OpenClinica Enterprise customers can request an image be placed on the application server by reaching out to the OpenClinica Enterprise Support team via the Issue Tracker.

Have you used HTML in your CRFs? Let us know if you have any other suggestions or tips!

IMPORTANT NOTES:

The RESPONSE_OPTIONS_TEXT field is not included in the list above, as HTML tags are currently not supported for response options.

The QUESTION_NUMBER field will display the text properly, but has been known to cause issues when extracting data. Therefore, HTML should not be used in this column.

I recently delivered a webinar titled “Getting Started with eCOA/ePRO,” in which roughly a third of attendees polled cited expense as the number one reason that has prevented them from adopting an ePRO solution. So what does ePRO really cost? Is it worth it? Here I strive to provide a basic, high-level framework for thinking about the return on investment ROI of eCOA vs. paper.

Let’s start by taking a look at the costs that are unique to each approach.

Paper

In a traditional paper based model, you are incurring costs that stem from printing, mailing, data entry, and data cleaning. These are all expenses than can be estimated with a fair degree of accuracy, with the cost of data entry being the most significant of these. To estimate the cost of data entry, see how long it takes to key in a subject completed paper casebook, multiply this by your cost of labor (don’t forget to include overhead!).

ePRO

The cost side for ePRO is similarly straightforward, but the expense elements are different. You’re either building an ePRO system (which will almost carry a highly unpredictable cost) of buying one (much more predictable cost). Assuming you’re buying, here are the costs you may expect to incur:

You should evaluate whether your study and selected ePRO system will allow for patients to use their own devices, or if you will need to provision devices (or some mix thereof). The cost of provisioning devices, especially for a global study can be significant—in addition to the costs of the devices themselves, you will need to consider the costs of data plans, and logistics associated with supporting the devices. I’m a big fan of BYOD (bring your own device) but, depending on the study, it may not be feasible to utilize while maintaining scientific validity of data collected.

Once you’ve mapped out your costs of each route, you can begin to weigh these against the benefits of going eCOA.

Paper vs. eCOA

When you boil it down, people employ ePRO/eCOA to maximize data quality, increase productivity, and/or enable new capabilities that help answer their research questions. ePRO is e-source, so you don’t have worry about administering a paper data entry process. Depending on the study, the cost savings from this alone might justify ePRO.

There are some additional benefits ePRO offers over paper that may be harder to quantify, but nonetheless very real. For example, there are clear data quality benefits to ePRO. The electronic system can ensure a minimum standard of data quality through edit checks and enforced data structures. ePRO data will always be cleaner than the same data captured on paper.

The use of an ePRO system also allows you to know for sure when the data were recorded. For instance, patients can be reminded automatically when their diaries are overdue, and you now only have much stronger assurances that data were collected at the appropriate time (vs paper), you can also more easily monitor the study progress.

Bypassing manual data entry and having the system provide notifications to subjects to ensure data are captured in a timely way might allow for faster and better in-study decision making and even may accelerate study closeout. Also, an increasing amount of evidence exists that mobile-based messaging and communication strategies help increase patient engagement and treatment adherence. And of course, not having to deal with a stack of paper during a site visit might allow the clinician’s interaction with the patient to be higher quality.

Quantifying the benefits of all of these things can be tough, but start with those which are most quantifiable and see if those items alone these alone provide a compelling ROI (from my experience they often do). Then the less tangible benefits become gravy to the ROI argument. When modeling costs over time and a pay-back period, keep in mind that ePRO will typically carry a higher upfront cost than paper, with the cost saving benefits realized downstream over time. With today’s technologies, even most smaller studies should be able to realize a positive payback.

Naturally, there may be additional ROI factors to consider which are specific to your situation. If you have particular thoughts, questions, or experiences on this topic I encourage you to add a comment to this post.

Whenever I teach case report form (CRF) design in the OpenClinica Central User Training class, the thought that always comes to mind is, “so many features, so little time.” OpenClinica has somany features available for building your CRFs, that there is no way to cover every possibility within the limited timeframe of a training class. Fortunately, there are other ways of getting information out to users, so here I am to introduce you to additional features you may want to incorporate into your CRFs.

Some of these features use JQuery, a cross-platform JavaScript library designed to simplify scripting HTML. The JQuery code in the attached CRF template was tested on Firefox 35.0.1; it may function differently in different browsers or different languages. As with anything you set up for your study, if you decide to use any of the features presented here, be sure to test them in your test environment prior to using them in production.

The attached CRF has 10 sections, and each section focuses on different feature sets. To see a list of all the Sections, click the drop-down arrow in the “Jump” box and then click the appropriate Section topic to see the features included within that topic.

Throughout the CRF, each feature is labeled so you can easily recognize it and locate it in the CRF template. For example, “Left Item Text” is not a prompt you would ever include on your CRF, but it is the feature being demonstrated and is labeled as such so you can easily see where Left Item Text appears on any CRF you design. Each Section of the CRF is listed below with a brief description as well as a screenshot so you can see how the features are represented.

1. Text

The Text Section illustrates various text features such as bold, italics, colors, and displaying images or a URL. This section also covers the positioning of left item text, right item text, header, subheader, title, subtitle, and instruction text. The Instruction area of this section includes many text options that you can apply. If you would like to include blue text on your CRF, simply reference “Text color can be changed” in the Instructions cell of the Section worksheet of the attached CRF template and copy the formatting into your CRF.

Click to enlarge

2. Response Options

This Section illustrates the various means of displaying responses on your CRF. It also provides additional features such as the “undo” button for deselecting a radio button, and an example of how to include a Visual Analog Scale. Again, simply reference the associated items in the Response Options section of the attached CRF template and copy the features you’d like into your CRF.

Click to enlarge

3. Layouts

The Layouts Section features different layout options such as column positions, horizontal vs. vertical checkboxes, and grid displays.

Click to enlarge

4. Required Items, Show/Hide, Decimals

This section demonstrates show and hide functionality, shows how real numbers (decimals) are displayed, and illustrates how required items are represented.

Click to enlarge

5. Validations This section demonstrates the various simple validations that are possible within the CRF template (less than, greater than, ranges, etc.). It also includes examples of some regular expressions.

Click to enlarge

6. Calculations I

This is the first of two calculation sections. In this section, you’ll see examples of calculations and group calculations. You’ll have to upload the CRF, associate it with an event, and enter data to see the calculations in action.

Click to enlarge

7. Calculations II

This second calculation section shows additional calculations and includes JQuery code for including a “calculate” button and for doing instant calculations. As above, you’ll have to upload the CRF, associate it with an event, and enter data to see the calculations in action.

Click to enlarge

8. Discrepancy Note*

The Discrepancy Note Section illustrates the effect of a Discrepancy Note Action Rule.

Click to enlarge

9. Show and Insert Actions*

This Section illustrates the effect of Show and Insert Action Rules.

Before the rule is fired: After the rule is fired:

Click to enlarge

10. Email Action*

This Section illustrates the effect of an Email Action Rule.

Click to enlarge

*These last three Sections listed have Rules associated with them, and the Rules must be uploaded in order to experience the full functionality of these Sections. The Rules are also attached to this post. If you want to use these Rules in your Study, be aware that you’ll have to edit them to reference the correct Object IDs for your Study.

So there you have it – one CRF, with everything but the kitchen sink. Check out the features and use them as you like. Just keep in mind that when designing CRFs, keeping it simple is always the best approach. There are times, however, when adding a little style to a form can help clarify things for data entry and may even help reduce discrepancies.

Happy designing!

Laura

Acknowledgements: Special thanks to Gerben Rienk Visser of Trial Data Solutions for many of these ideas, and to OpenClinica Application Support Engineer Jessica Gosselin for creating the Kitchen Sink CRF – we hope you find it useful!

OpenClinica is a clinical trial software platform that aims to provide data capture, data management, and operations management functionality to support human subjects-based research. It can be used for traditional clinical trials as well as a wide variety of other types of human subjects-based research.

Our vision for the product is to provide data capture, data management, and operations management functionality out-of-the-box, in an easily configurable, usable, and highly reliable manner. The underlying platform should be interoperable, modular, extensible, and familiar – so users can solve specific problems, in a generalizable way.

This past spring, the development team here at OpenClinica, LLC held a series of round table discussions about how this vision is reflected in the product. Our goals were to learn critical standards and information models needed for our technology to truly reflect this vision, to develop a consistent, shared vocabulary for the problem domain and the OpenClinica technology, and identify the most urgent opportunities to put these lessons into practice in the product and the community. In particular, we spent a lot of the time in these discussions about how OpenClinica’s use of the CDISC Operational Data Model helps enable this vision.

The discussions were invigorating and thought-provoking. We’ve recorded them to share with the greater community of OpenClinica developers, integrators, and others who want to better understand how the technology works, the design philosophy behind it, and where we’re going in the future. The videos are embedded below.

But before getting to the videos, here’s a bit more background on how we think about OpenClinica as a product and a platform.

First, OpenClinica functionality should be ready out-of-the-box, easily configurable and highly usable. Some of the most important features include:

Data definition and instrument/form design with no or minimal programming

Many of these things have already been implemented, and more are under development.

The core concept around which OpenClinica is organized is the electronic case report form (CRF). In OpenClinica, a CRF is a resource that is essentially a bunch of metadata modeled in CDISC ODM with OpenClinica extensions. It doesn’t (necessarily) have to correspond to a physical or virtual form; it may represent a lab data set or something similar. An OpenClinica Event CRF is that same bunch of metadata populated with actual data about a particular study participant. Thus, it combines the metadata with the corresponding item (field) data, with references to the study subject, event definition, CRF version, and event ordinal that it pertains to. In this conceptual view of the world, CRFs (as well as CRF items, studies, study events, etc.) are resources with core, intrinsic properties and then some other metadata that has to do with how they are presented in a particular representation. Built around these core resources are all the workflow, reporting, API, security, and other mechanisms that allow OpenClinica to actually save you time and increase accuracy in your research.

Second, OpenClinica should be interoperable. The ultimate measure of interoperability is having shared, machine readable study protocol definitions, and robust, real-time, ALCOA-compliant exchange of clinical data and metadata that aligns with user’s business processes. It should be easy to plug in and pull out or mix-and-match different features, such as forms, rules, study definitions, report/export formats, and modules, to transport them across OpenClinica instances or interact with other applications. Establishing well defined methods and approaches for integration into existing health data environments is a key goal of interoperability.

Third, OpenClinica should be modular and extensible. OpenClinica already provides some of the most common data capture and data management components and aims to have a very broad selection of input types, rules, reports, data extracts, and workflows. However OpenClinica developers should also have the freedom to come up with their own twist on a workflow, module, or data review workflow and have it work easily and relatively seamlessly with the rest of OpenClinica. User identification, authentication, and authorization should be highly configurable and support commonly used general purpose technologies for user credentialing and single-sign-on (such as LDAP & OAuth).

The CRF-centric model allows us a great deal of flexibility and extensibility. We can support multiple modalities, with different representation metadata for rendering the same form, or perhaps the shared representation metadata but applied in a different way (i.e. web browser vs. mobile vs. import job). We can address any part of the CRF in an atomic, computable manner. This approach has been successfully applied in the Rule Designer, which takes the ODM study metadata and allows browsing of the study CRFs and items, with the ability to drag and drop those resources to form rule expressions. Features such as rules and report/export formats are represented as XML documents. These documents define how the features behave in standardized ways so that one rule can, say, be easily replaced with another rule without having to modify all the code that makes use of the rule.

Finally, OpenClinica aims to be familiar in the sense of allowing data managers, developers, statisticians to work in a design/configuration/programming environment that they already know. Programmers don’t all have the same experience, and it would be somewhat limiting to force OpenClinica developers to all use the same language (Java) that OpenClinica was written in. We are constantly looking at ways to make it possible (not to mention reliable and easy!) for users and developers to interact with and extend OpenClinica in a programmatic way. This can mean anything from data loading to more meaningful integrations of applications common to the clinical research environment. As proponents of open, standards-based interoperability, our starting point is always to develop interfaces for these interactions based on the most successful, open, and proven methods in the history of technology – namely the protocols that power the World Wide Web (such as HTTP, SSL, XML, OAuth 2.0). They are relatively simple, extensively documented, widely understood, and well-supported out of the box in a large number of programming and IT environments. On top of this foundation, we rely heavily on the wonderful work of CDISC and the CDISC ODM to model and represent the clinical research protocol and clinical data.

One of the strengths of open source is the ability to open up the code base and learn by reading and doing, that is, the transparency of the code base allows everyone to get involved. However, the barrier to entry can be the complexity of the code itself; without a qualified guide, you can get ‘lost in the code jungle’ pretty quickly.

Welcome to our code

With that in mind, we are starting today to author blog posts about the OpenClinica code base, including topics like how the code is organized, what the code does, and so on. A lot more detail on this can be found on the OpenClinica Developer Wiki, but these posts, viewed as a whole, can be seen as a gentle introduction, before interested parties start to dive deeper.

When we began to design OpenClinica, we had very few requirements, but the desire to create a fully-featured database for clinical data, aligned with open standards, making use of the best technology available. Call it the ‘tyranny of the blank page’, if you will. Every start-up faces it. Where do you start? What’s the plan? How do you build it, and what do you build first?

Luckily for us, we could use an open standard to base our schema, and our code, on top of; the CDISC ODM.

What’s a CDISC ODM?

The Operational Data Model, or ODM for short, is a standard published by the Clinical Data Interchange Standards Consortium (CDISC), and is “designed to facilitate the archive and interchange of the metadata and data for clinical research”, as it states in their website. This is a standard which is designed to a) hold metadata about a Study and all Events contained within a given Study, and b) hold Clinical Data which has been collected for a given Study. All of this information is held in XML, which is a very useful format for exchanging between sites, labs and institutions.

In the above image, you can see an XML file on one side using CDISC ODM and on the other side, an OpenClinica database. Inside the database are tables that map directly to different objects described in the XML. You’ll notice that the tables associated with study metadata also have a column called ‘oc_oid’, which are the Object Identifiers we use in all aspects of the OpenClinica application.

In the second image, you see that the latter half of the XML file (the part contained in the <ClinicalData> tags) also links to specific tables in the OpenClinica database. Since we link back to the Study metadata through those OIDs, we don’t use OIDs in those tables, but instead the conventional methods of primary keys and foreign keys in the database is good enough.

OK, so they map. But where’s the beef?

Of course, the ODM XML in the images is rather simple, and does not capture the full capability of the metadata that can be passed back and forth between different ODM data sources. For a longer example, you can take a look at the following XML, which defines the Rules governing a single Item:

As you start to piece together the XML in the above example, you’ll see that not only can you define the Question in multiple languages, but you can specify which measurement it is using and what kinds of values you can accept. The XML standard is extensible enough to add other pieces of information as well, including coded lists, data types, and so on. More information can be found at XML4Pharma’s page entitled, ‘Using CDISC-ODM in EDC.’

In future posts, we hope to describe more about the code base, and show how it all comes together as a full-featured application. If there are topics that are of specific interest, we hope you’ll comment below and let us know what you’d like to see here in the coming months.