With all of the Apple iPhone hype lately, the Medical Images on an iPhone post caught my eye. Privacy and HIPAA concerns don’t worry me nearly as much as the thought of a Radiologist reading an x-ray while he’s driving to work. Seriously though, improvements in both display resolution and user interface capabilities have come a long way. Apple is not the first to provide this type of functionality. PDA-based medical record applications and image viewers have been around for a long time.

I contacted Frank Jacquette regarding my previous port on this subject (Agile development in a FDA regulated setting). His experience using Agile methodologies for pure software medical device projects does not correspond with my conclusion regarding cost effectiveness and regulatory risks. Frank said:

It has been a long while since I wrote that article, but we’ve applied the same approach to some fairly significant systems and they’ve all come in on time and within budget.

He does agree that the regulatory risk is a legitimate concern, but their experience with clients and regulators has always been positive.

I want to thank Frank for so graciously responding to my inquiry.

He also pointed me to a presentation called Integrating lightweight software processes into a regulated environment (warning: PDF) by Adrian Barnes that I had not seen before. This is a far more detailed look at possible solutions for bridging the gap between “Agile Processes” and “Formal Processes”. The subject progression and graphics are very well done. It’s worth a careful look-through. I’ll let you be the judge, but I think Adrian’s conclusions have the same level of skepticism as mine. I broadly addressed cost effectiveness whereas he specifically deals with risk factors for his bridge solution. He has even less faith on the regulatory risk side: “A brave man would try to convince the FDA that Agile is OK”.

My initial impression is that the current implementation of the Microsoft Health CUI (v1.0.114.000) is strong on depth and weak on breadth. Because of the limited number of components available this software is too early in its implementation of be of much use in a real product. For example, the first thing I would need is a ‘PatientGrid’ component, which doesn’t exist yet. This is just the first CTP, so missing features are be to expected.

Here are the component lists for WinForms and Web applications:

The design guidance documents are the most impressive aspect of this project. Each control has its own complete document that includes sections like ‘How to Use the Design Guidance’ and ‘How Not to Use the Design Guidance’. The higher level terminology and accessibility guidance documents are equally as comprehensive. As a software developer that has had to work from ambiguous requirements specifications, nothing is left to the imagination here. The requirements for some components (e.g. AddressLabel) are written to UK specifications, but that’s be expected since CUI is being developed there.

The Visual Studio integration is good and the source code for the individual components appear to be well constructed and documented.

The CUI Roadmap isn’t very specific, but I like the design guidance driven approach. All of the up-front design work makes me think of my previous post on Agile development. The CUI Delivery Lifecycle is described as iterative, but I doubt it’s actually being developed using one of the Agile methodologies. In any case, I’ll continue to watch the progress of this project and look forward to future releases. It could be my excuse to actually use (instead of just playing with) WPF someday!

The Toolkit controls developed for this release conform to the recommendations contained in the Design Guidance documents. The Toolkit is a set of .NET 2.0 controls that help Independent Software Vendors (ISVs) build safe, consistent user interfaces for healthcare applications. The Toolkit controls have been created for use in both ASP.NET AJAX applications and WinForms.NET. The web versions of the controls are based on the ASP.NET AJAX Toolkit.

I’ll take some time and investigate the controls and library and how well it integrates into an existing .NET 2.0 WinForms application.

I ran across an interesting Agile v. FDA discussion the other day. For those that are not familiar with what a FDA regulated product means, I’ll give a brief overview.

In order to market and sell a medical device in this country — OUS (outside US) medical device regulations are different — you must have FDA approval. Most of the time, this involves a FDA Premarket Notification 510(k) submission. Your company must be registered with the FDA and is subject to periodic on-site visits by FDA inspectors to audit your quality system records. There are two important points here:

Keeping approval: After you receive 510(k) approval the FDA can pull your device off the market (the dreaded “recall”) at any time due to complaints or unsatisfactory audit results.

What this means is that your on-going software development process must adhere to a well defined quality system process. As noted in the discussion, the FDA guidance does not dictate that a particular process must be used. The quality system process itself is designed by you — actually, your entire company — and simply needs to ensure that you are designing and building a quality product. The difficult part is that you have to be able to prove to the outside world that you have actually followed that process.

I’ve spent just about my entire career developing software for devices that were under FDA regulatory control. In the old days (pre ~1996) the FDA did not have a good concept of the software development process as part of it’s quality system regulations and inspectors did not usually scrutinize software design and development. Nowadays, the FDA has a much clearer understanding of the software development process. It’s that understanding that is the one of the central issues with respect to adopting an Agile development process for software in medical devices.

The critical issue for the software development process is that each of the items 3-7 require a formal review and approval step. This is the reason why most companies that develop medical device software have chosen to use a quality system process that follows the Waterfall model for their development.

This sequential approach is a natural fit for allowing you to review and document each step in the process.

Now let’s look at the Agile process. One of the better descriptions of the Agile development process is ‘The New Methodology’ by Martin Fowler. There a number of flavors of Agile (SCRUM, Extreme Programming (XP), etc.) that all try to encompass the Agile Manifesto. One advantage that the Agile process has over the Waterfall approach is it’s ability to adapt to the unpredictability of requirements and changing customer needs. This is handled through the use of Iterations. From the Martin Fowler paper:

The key to iterative development is to frequently produce working versions of the final system that have a subset of the required features. These working systems are short on functionality, but should otherwise be faithful to the demands of the final system. They should be fully integrated and as carefully tested as a final delivery.

Let’s assume that an Agile development process would be able to produce higher quality medical device software and that because of the customer focus of this process the resultant product would better meet market needs. Even with these assumptions, I think there are two major issues that need to be addressed:

I just don’t see how a cost-effective quality system process implementation can be accomplished. Even if (and it’s a big if) the actual overall software development time was shorter, the extra costs incurred by the additional process controls, documentation and testing required for each iteration would far exceed those savings.

The last point Mr. Jacquette makes is the other issue:

Take the time to explain agile methodologies to your regulatory specialists and work hard to gain their understanding and agreement, because the burden of proof will be on them and you if an FDA auditor decides to take a peek under the hood.

This seems like a huge risk to me.

The first question becomes: Is the possibility of an improved product (features and quality) worth the additional cost? I suppose if you had a development team with extensive Agile experience you could make the argument that it would be worth it. If not, I think the ROI (return on investment) analysis would be a difficult one to make.

The second question is a big unknown, which is why I think the risk is high. My experience with FDA auditors is generally good. They are professionals that are focused on getting a very specific job done. Since their interest is the entire quality system, the typical audit is a whirlwind affair as it is. The amount of time spent on design control (auditing the Design History File) is usually minimal. Even if you had received 510(k) approval with an Agile design control process, having to take the time to explain to an on-site FDA auditor (who in all likelihood has never seen your 510(k)) a methodology they probably have never even heard of is reason to worry.

Conclusion:

It seems to me that Agile methodologies have a long way to go before we see them commonly used in medical device software development. I’ve searched around and have found nothing to make me think that there is even a trend in this direction. Maybe it’s that Agile processes are just too new. They seem popular as a presentation topic (I’ve been to several), but I wonder how prevalent Agile is even in mainstream software development?

If you are (or have ever been) part of an Agile development team for a FDA regulated product I’d love to here about your experiences and how you were able to resolve the types of issues presented here.

This is a story that I’m sure is being written over and over again by software designers like myself that need to develop relatively complex applications in a short amount of time. The only realistic way to accomplish this is to bite off major chunks of functionality by leveraging the work of others. Reinventing the wheel, especially wheels you know hardly anything about, can’t be justified. Because of that many of the architectural decisions you have to make revolve around figuring out how to get all of those ‘frameworks’ and libraries and GUI components to work together. But I digress…

This post simply tells the tale of my journey though picking a framework for a project, how the landscape has changed over time, and what that means for the future. In case you don’t want to read on, the ending is happy — I think.

For any project that requires the storage and orderly retrieval of data there’s pretty much no way to get around using a database. Also, if you have a requirement (like I did) that the user is able to select from a variety of SQL back-end databases then the use of some sort of Object-Relational Mapping (ORM) framework software is the only sane way to implement your application.

I’m not going to get into the details of ORM software or try to compare the features of the vast array of frameworks available. One current list of object-relational mapping software includes 34 entries for .NET (and another 27 for Java). One problem with making a selection is that many of these frameworks are constantly evolving and are thus moving targets.

On a side note, Microsoft has squarely thrown their hat into the ORM ring with the introduction of Language Integrated Query (LINQ) which is scheduled to be released with the ‘Orcas’ version of Visual Studio 2008 and .NET Framework 3.5 (see a summary here and a great series of articles on ScottGu’s Blog). LINQ won’t be released until next year so it couldn’t even be considered for my project.

So, the question is: If you’re developing a modest .NET application, how do you go about selecting an ORM framework? I’ll walk you through the selection process I used.

General filtering criteria:

Non-commercial: I had no money to spend. This cut the possibilities about in half.

Open source: Even though I didn’t want to touch the implementation, it was critical for long-term security that I had all the source code and build tools so it could be modified if needed.

SQL back-end selection: The framework needed to support at least four of the most popular SQL databases. I wanted the user to be able to select local and server-based databases that were both free (MySQL and SQLite3) and commercially available (Microsoft Access and MS-SQL Server).

Ease of use: The learning curve for some frameworks is steep. I wanted something that was easy to learn and to get up and running quickly.

Documentation and example code: Having a well documented API is important, but it’s the example code and tutorials that teach you how to use the framework.

Support: This was the most subjective piece of the puzzle. The key is to get a sense of the level of developer and community involvement in the project. This comes from looking at the release frequency and history (in particular, look at the release notes and the number and type of issues resolved) and the level of Forum and/or Wiki activity. It’s pretty easy to detect a dead project. Foreshadow: Just because it’s an active project now doesn’t mean it won’t die in the near future!

To make a long story short, I ended up evaluating two open source ORMs: NHibernate for .NET and Gentle.Net. As noted above, I did this evaluation about a year ago (Summer 2006) and things have changed since since then. In any case, at the time I felt that the two were comparable for the functionality I needed which included just simple class to table mapping. The two major competing factors were ease of use (#4) and support (#6). NHibernate was clearly the more active project, but the Gentle.Net developers (in particular, Morten Mertner) were responsive to questions and quickly resolved issues that came up. I found Gentle.Net much easier to get up and running. In particular, in NHibernate you need to create a matching mapping file for every database class. This was daunting at the start and I felt like the split-file definitions would create a long-term maintenance problem. With Gentle.Net, I was able to use MyGeneration to create the .NET class which included the database field decorators, and that was it. Also, RE: #4 the NHibernate core is a 25MB download while Gentle.Net is less than 7MB. This is not only telling of the relative complexity of the frameworks, but it also means that there would be significantly more baggage that I would have to carry around with NHibernate — and install on customer systems.

Based on that analysis it shouldn’t be a surprise that I picked Gentle.Net. One year later I’m still happy with that decision. Gentle.Net continues to meet my needs and will for the foreseeable future.

OK, I’m happy, so where’s the problem? The problem is that it appears development has stopped on Gentle.Net. I’m using the .NET 1.1 assemblies (1.2.9+SVN) with my .NET 2.0 application. The features in the next generation Gentle.Net (2.0) development looks promising, but at this point it doesn’t look like it will ever be released.

In retrospect, I suppose that even with my concerns, selecting NHibernate might have been the better long-term choice. I think it was primarily the ease of the initial Gentle.Net implementation that had the biggest sway on my decision. Does this mean I made a wrong decision? You may think so, but I don’t. There may be some cognitive dissonance going on here, but I did what I consider a reasonable evaluation and analysis and selected the best solution for my needs under the circumstances.

So here we are today. At some point in the future I will probably need to update or add a new SQL database that Gentle.Net doesn’t currently support. I’ll have to do something. Here are my options:

Add the needed functionality to the existing Gentle.Net code base. This might not be unreasonable to do, but it’s just putting off the inevitable.

Re-evaluate current ORM frameworks and select one like I did last year. Déjà vu all over again… Microsoft side note: I can also wait for .NET Framework 3.5 to be released and use LINQ to SQL. My concern with LINQ is support for non-Microsoft databases. There’s already some effort to develop LINQ providers (MySQL, etc. see here) but how long will it take to get reliable support for something like SQLite3 (or 4 by then) if it happens at all?

Justify the expense and purchase a supported product.

I think my little tale is pretty typical. Any developer that’s using open source components is, or will be someday, in the exact same situation as I am. Doing good software design to mitigate these types of issues is why they pay us the big bucks! 🙂

There was an article about high-tech scales in the this mornings paper that caught my eye. BIA is used to measure body-fat and is similar to ICG. There’s a good explanation of how these devices work here.

One of the critical factors for these measurements is the number of electrodes (2 or 4) and their placement. The impedance model used for the parameter calculations (TBW, FFM, etc.) is highly dependent on the electrode configuration. Reliability and reproducibility have improved over the years which has made these types of devices commercially viable in the consumer market.

There is also active research in multi-frequency BIA (MF-BIA) in order to “accurately monitor acute and chronic changes in hydration during various diseases and their clinical management” (see ref). A commercial MF-BIA product is available from Xitron that uses Bioimpedance Spectroscopy (BIS) for estimating intra and extracellular water (ICW and ECW). These measures can be particularly useful during dialysis treatment.

I’ve been reading technical blogs of all sorts for many years. I’ve finally tired of being just a lurker (join Code Project for a definition) and I’m going to try to contribute to the blogging community. My technical interests include:

Software Development techniques and practices

Microsoft technologies (.NET, WCF, WPF, etc.)

Embedded Real-time systems (WinCE and Linux, DSPs)

Medical Device development

EMR (Electronic Medical Records)

I’ve been a private software consultant in the past and have maintained the RDN Consulting domain. I don’t expect to put content back up any time soon, but since the hosting is paid for for a couple more years this turns out to be a great place to host a blog (WordPress was a snap to install).

My current day job is a Research and Development Staff Engineer at CardioDynamics International. They design, manufacture, and sell Impedance Cardiography (ICG) devices. The technology behind ICG is called thoracic electrical bioimpedance (TEB) and is very cool. My primary responsibility is software design and development for both medical devices and EMR integration solutions. I work with a great group of people.