Medical simulators are complex yet powerful tools that help residents train for (or rehearse) complex surgical procedures and learn basic skills that are common to different surgeries performed in the operating room (OR) [1]. More specifically, simulators make training and skills acquisition more efficient through the use of quantifiable measures. Studies have indicated that medical simulators have led to fewer errors in the OR and have decreased the costs of post-surgical care [2-4].

A typical medical simulator is a collage of software tools from disciplines such as computational mechanics, 3D graphics, medical image processing, computational linear algebra, 3D geometric modeling, computational geometry, haptics, human factors, and computational physiology. As a result, it can be laborious and time consuming for developers with expertise in even a few of these disciplines to build a simulator that can pass validation and be ready for clinical deployment.

In response to this need, we developed the interactive Medical Simulation Toolkit (iMSTK)—a C++ based free and open source toolkit that aids rapid prototyping of real-time multi-modal surgical simulation scenarios. iMSTK systematically embeds commonly used features in separate modules and presents them to developers through a palatable API. It aims to help research and development efforts in academia and industry. Version 1.0 of iMSTK came out last week.

Below are some of the main features that iMSTK offers.

Physics-based simulation: The Simulation module has features such as linear and nonlinear finite element models along with position based dynamics [5] (elastic objects, fluids and membrane).

Collision Detection: The Collision Detection module detects and reports collisions between two geometric entities in a scene. The module contains many narrow phase intersection tests between simpler geometric primitives and advanced collision detection that use spatial hashing.

Collision Handling: The Collision Handling module encodes formulations that act on the collision information to physically resolve the intersection between two objects in a scene. Currently, iMSTK offers constraint- and penalty based collision handling features.

Numerical Solver: The Solver module supports linear and nonlinear numerical solvers that resolve the unknowns at the end of each frame and, thus, form the workhorse of the simulation. Standard direct solvers such as LU and LDLT decomposition as well as iterative solvers such as Conjugate gradient and point relaxation (e.g., Gauss-Seidel and Jacobi) are available.

Virtual Reality: iMSTK features SteamVR support that works with standard VR head-mounted devices such as Oculus and HTC Vive. As it stands, this support is compatible with the VTK backend only.

Build and CI: iMSTK allows automatic CMake-based cross-platform builds where the dependencies are fetched and built automatically. Additionally, default project resources are downloaded and placed in the appropriate directories.

iMSTK also uses a quality control workflow that includes a stringent testing suite, a continuous integration system that is based on CTest and CDash, a web-based code review system and a protocol for accepting patches into the main code repository.

Downloading and Building iMSTK

The iMSTK source code is available from the toolkit’s download page. iMSTK is supported on all major platforms. It can be configured on Linux, macOS or Windows using CMake.

Linux and macOS

To build iMSTK on Linux or macOS, type the following commands from the location where the code is cloned. This will configure the build in a directory adjacent to the source directory. To easily change configuration variables like

For a faster build, use Ninja instead of Unix Makefiles by configuring the

cmake

project with

-GNinja

.

cmake -GNinja ../iMSTK
ninja

This will check out, build and link all iMSTK dependencies. If you want to make changes to iMSTK’s base source code or run examples, build from the Innerbuild directory.

Windows

Run CMake-GUI and follow the instructions from “Running CMake.” You will be prompted to choose a version of Visual Studio. Select Microsoft Visual Studio C++ 15 2017.

CMake will generate an iMSTK.sln solution file for Visual Studio at the top level. Open this file and build all targets to check out, build and link all iMSTK dependencies. If you want to build on multiple cores, add

/MP[N]

to

CMAKE_CXX_FLAGS

in CMake-GUI, where N is optional (representing the number of cores). Without N, the build will use all of the available cores on the device.

Additional Build Options

For additional build options such as enabling the Vulkan rendering backend or using a haptic device, please refer to the instructions on the code repository page.

Hello World Example

Now that you’ve built iMSTK, it’s time to try it out. In this “Hello World” example, a simple elastic object is deformed under gravity and visualized as part of the iMSTK scene. The comments describe the steps.

The output from the “Hello World” example shows object configuration at the start, t=0 sec.

This images displays the configuration at a later time when the object reaches force equilibrium.

Below are the additional runtime user keyboard controls that are available by default.

‘space’: pause or the run simulation

R/r: restart the simulation

D/d: toggle between debug and simulation rendering modes

P/p: display the render frame rate on the screen (see the images above)

You can use ‘debug rendering mode’ to show the camera and lights of the scene, as well as to pan, zoom and rotate the scene with a mouse.

Part II

Part II of this blog series will discuss some of the medical trainer applications that we have developed with iMSTK. These applications cover different medical domains and use iMSTK in combination with 3D Slicer and VTK. We will further discuss upcoming development plans and our efforts to integrate newer software such as Computational Model Builder (CMB) (which helps string together all of the components of a simulation lifecycle) and the Pulse Physiology engine (which models and simulates human physiology using reduced models) to a support wide variety of workflows that are required by modern medical simulators.

Acknowledgment

Research reported in this publication was supported, in part, by the National Institute of Biomedical Imaging And Bioengineering of the National Institutes of Health (NIH) under Award Number R44EB019802; by the National Institute of Diabetes and Digestive and Kidney Diseases of the National Institutes of Health under Award Number R43DK115332; by the Office Of The Director, National Institutes Of Health of the National Institutes of Health under Award Number R44OD018334; and by the National Institute of Dental & Craniofacial Research of the National Institutes of Health under Award Number R43DE027595.

The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH and its institutes.

]]>https://blog.kitware.com/introducing-imstk-part-1/feed/024129CMake 3.13.2 available for downloadhttps://blog.kitware.com/cmake-3-13-2-available-for-download/
https://blog.kitware.com/cmake-3-13-2-available-for-download/#respondThu, 13 Dec 2018 14:10:58 +0000http://blog.kitware.com/?p=24119We are pleased to announce that CMake 3.13.2 is now available for download. Please use the latest release from our download page: https://cmake.org/download/ CMake 3.13.0 ... Read More

CMake 3.13.0 included a change to pass compiler implicit include
directories to the “moc” tool for “AUTOMOC”. This has been reverted
due to regressing existing builds and will need further
investigation before being re-introduced in a later release.

]]>We are pleased to announce the release of Tomviz 1.5.1 (GitHub release page). This release contains a number of bug fixes and stability enhancements found after the 1.5 release, along with a few longer standing issues we tracked down. An issue affecting manual image alignment that could cause crashes when certain offset values were used near the start or end of a tilt series was fixed, and a crash affecting the loading of state files where the data file was not in the expected location was also addressed. Several other minor issues were addressed, such as reducing memory use in one of the reconstruction operators.This is probably our final release in 2018, and we would like to thank the entire community for their feedback and support. Work on the 1.6 release has already begun with better support for data that contains more than one channel, along with improving our support for “live” pipelines where data is being acquired. As always we welcome your feedback, and hope that you find this release useful in your research!

The Tomviz project is developed as part of a collaboration between Kitware and the University of Michigan under DOE Office of Science contract DE-SC0011385. This is a community project, and we are very pleased to take input and contributions from all in the community.

]]>We have a new Open Chemistry DOE SBIR Phase II project developing integration of chemical and materials data within Jupyter coupled with a data server/job execution logic. Previously we wrote up some of the early results in a blog post, and presented the platform at JupyterCon in August of this year. In the last few months we have also been looking at making it easier to try some of the tools being developed for this platform. Providing an easy way to install and use some of them in local JupyterLab notebooks is the focus of this post.

Binary Wheels for Avogadro’s Libraries

The Avogadro libraries are lightweight C++ libraries developed to not only support the Avogadro desktop application, but enable reuse in other contexts. We used PyBind11 to wrap some of the core API for reading/writing chemical files, and calculating electronic structure from the output of quantum codes. Binary wheels can be installed using the Python package manager pip under Python 3 (built using our CI integrated with release tags on GitHub):

pip install avogadro

Once you have the package installed you can use a small subset of the C++ API. We plan on extending this in the future, but it is enough to load files, save files, perform some basic electronic structure calculations, and make some very rudimentary queries on the loaded molecule. Our server process has used this for quite a few years, originally using Boost.Python for wrapping.

Visualizing Electronic Structure in JupyterLab

Another significant part of our project is the extension of the Jupyter Python 3 kernel, and visualization of chemical data in the JupyterLab web client. We use 3Dmol.js or vtk.js as the JavaScript rendering engines for 3D structure, with D3 for charts. A Python module houses the Python API being developed to support the project, and a JupyterLab extension for the visualization components. These can be installed locally using the following commands assuming you have Python 3 and JupyterLab installed already:

If all goes well you can start your JupyterLab server, and if you clone our new examples repository we will even give you a few example files to get started with:

git clone git://github.com/openchemistry/jupyter-examples
jupyter-lab

You can open the example notebooks, and if everything works as expected you will be able to look at the example data, or even import some of your own if you make the files available.

JupyterLab and Binder

We have gone one step further, and added the necessary files to the example repository to enable viewing of these notebooks in Binder. This enables you to experiment with the features described above in a live environment using everything mentioned. It can take a while to launch the containers, but once ready you are able to experiment with no local installation. It is fantastic to be a part of such a vibrant ecosystem of open source tools, and freely available web services that aid in the dissemination of results.

Conclusions

The wrapped Avogadro API is at quite an early stage, and many things you might want to do are not yet supported. Some extra work was done to decouple components so that it is easier to reuse parts of our Python kernel, web visualization, and other pieces independently of the larger project. We hope that you find these pieces useful, and we will continue to work on making it easier to install these open source components independently.

This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Basic Energy Sciences, Small Business Innovation Research (SBIR) program under Award Number DE-SC0017193.

There are two common techniques in computer graphics to render surface geometries: Rasterization and Raytracing. By default, ParaView uses the Rasterization technique with OpenGL to display surfaces in real time but Raytracing is far better at rendering high-quality images.

ParaView 5.6.0 was recently released and includes a lot of improvements regarding OSPRay integration (see full release note here). OSPRay is an open-source raytracing renderer developed by Intel and highly optimized for CPUs. Two engines are available: sciviz and raytracer.

This blog post highlights the one specialized for realistic surface rendering – the raytracer engine – used to generate the following video. It shows a virtual visit of a power plant and shows that we can mix high-quality rendering and scientific visualization:

High-quality virtual tour of a power plant room with ParaView 5.6 and OSPRay. Data are (c) EDF.

According to an engineer of EDF: “A way to make accessible simulation results to non-expert audience is to render generated datasets into their real environment. In this context, OSPRay high-quality rendering is part of valuable tools for EDF to strengthen the comprehension of complex phenomena computed by simulation codes.”

The making of

OSPRay engine supports a lot of materials going from Metal to Glass in order to efficiently render all the materials you can find in the real world.

Examples of materials supported by OSPRay.

In ParaView, materials have to be defined with a JSON file and loaded in ParaView through the “File / Load OSPRay Materials…” menu.

Each material has different specific parameters, documented in the OSPRay documentation. It is important to have in mind that material properties can be uniform on the whole surface or declared through an external image acting as a texture. In the last case, the surface must have texture coordinates (and ParaView has several filters to compute such coordinates if needed). For example, the granular effect of the surrounding surface is created using a normal texture, transforming surface normal direction depending on texture coordinates:

Normal texture used to create a granular effect.

In our video, we used four different types of materials, depicted in this image:

The video contains 600 high definition images (1920×1080 pixels), with a high density of rays (300 samples per pixel) in order to completely eliminate the noise. This represents roughly 600 millions primary rays for each image!

For information, the video took approximately 35 hours to be generated with ParaView 5.6.0 on a distant headless Google Cloud compute node with 72 vCPU without GPU (OSMesa enabled).

]]>One major feature of Tomviz is to display and transform volumetric tomographic materials data. It can read volumes from a wide variety of file formats (TIFF, EMD, MRC, NumPy, Matlab to name a few), but a common need among our users is to create 3D volumes by opening a stack of 2D images in a sequence that is encoded in the file names of the individual slices.

Opening multiple files can be done with a standard file dialog, but this approach has several drawbacks, the most critical of which is a potential crash when creating a volume by opening images with inconsistent sizes. Loading data is one of the first things that a new user wants to do, and to provide the best user experience we added a new a dialog dedicated to opening stacks of TIFF files in Tomviz 1.4.

The image stack dialog provides several ways to open the individual images:

Select multiple files or a directory through a file dialog

Drag and drop files or a directory in the dialog area

Once the user has made their selection the dialog analyzes the individual images to ensure that their sizes are consistent. If an image size doesn’t match it is excluded from the generation of the volume, and the offending image is flagged as shown below. The dialog also offers the ability to remove images from the stack.

The dialog scans the individual file names and guesses the order of the images, using some common naming conventions that have been provided by users. The dialog is also able to determine if an image stack is a tilt series, and extract tilt angles from the file names. These angles will be shown once the tilt series is loaded, and can be used as part of the reconstruction process.

The Tomviz project is developed as part of a collaboration between Kitware and Cornell University under DOE Office of Science contract DE-SC0011385. This is a community project, and we are very pleased to take input and contributions from all in the community.

]]>The company that releases the Visualization Toolkit and 3D Slicer makes a toolkit for surgical simulation.

A member of the iMSTK development team, Samantha Horvath, demonstrates iMSTK in a virtual simulator for kidney biopsy.

Kitware added to its collection of open source toolkits with the first release of the interactive Medical Simulation Toolkit (iMSTK). The toolkit offers manufacturers and researchers all the software components they need to build and test virtual simulators for medical training and planning.

“iMSTK, which we’ve been developing in close collaboration with Professor Suvranu De’s research center — the Center for Modeling, Simulation and Imaging in Medicine at Rensselaer Polytechnic Institute — is meant to empower developers to rapidly prototype virtual simulator applications,” said Andinet Enquobahrie, the director of medical computing at Kitware.

Virtual simulators can help practicing surgeons, medical students, residents and nurses to rehearse or plan surgical procedures such as biopsies, resectioning, radiosurgery and laparoscopy without compromising patient safety in the operating room. Medical simulators can also help accredit potential surgeons in basic skills for laparoscopy, endoscopy or robotic surgery.

To aid the development of medical simulators, iMSTK brings together 3D medical image processing plus modeling and simulation. More specifically, iMSTK works hand-in-hand with 3D Slicer and the Insight Segmentation and Registration Toolkit (ITK) to segment a medical scan, derive 3D models from the segmentations and use the models for interactive physics-based simulation and photorealistic rendering.

In addition to simulation and rendering, iMSTK offers support modules for collision detection and geometric processing. It also offers support for an array of external devices, including HTC Vive, Oculus Rift and the Geomagic Touch haptic device. Behind each of the elements in iMSTK is decades of open source software development at Kitware.

“What separates iMSTK from other open source software for medical simulation is that it provides a plugin-flexible system for VTK and the in-house Vulkan rendering backend,” said Sreekanth Arikatla, the chief architect of iMSTK and a senior R&D engineer at Kitware. “The system can also plug in physics-based modeling libraries for the finite element method, the rigid body simulation and Position Based Dynamics.”

Kitware has worked with surgical training companies, research hospitals and universities to base virtual simulators on iMSTK, including one for kidney biopsy, one for osteotomy and another for basic laparoscopic skills training. The latter drills users in aspects of the Fundamentals of Laparoscopic Surgery program, which consists of navigating a camera, transferring objects onto pegs, cutting patterns, performing loop ligation and tying extracorporeal and intracorporeal suturing. Both the Journal of Biomedical Informatics and Surgical Endoscopy have published results on the use and validation of this simulator. It is part of a near $1.5 million award from the National Institutes of Health, which Kitware received in 2015 with collaborators that include Rensselaer Polytechnic Institute (RPI).

“Kitware has been a powerful ally not only in developing the virtual simulator but also in facilitating the transition of new algorithms from graduate research at RPI’s Center for Modeling, Simulation and Imaging in Medicine to professional-grade software libraries,” said Suvranu De, the center’s director.

Kitware will continue to work with its research partners to add more features to iMSTK such as multithreading for faster simulations, advanced contact algorithms and multiphysics capabilities. They will also more fully integrate iMSTK with Kitware’s Computational Model Builder and Pulse Physiology Engine.

Members of Kitware’s medical computing team will show demos of simulators that use iMSTK at the 2019 International Meeting on Simulation in Healthcare (IMSH) in January. To meet with team members at IMSH, please contact kitware(at)kitware(dot)com, or visit them at booth 740.

Research reported in this publication was supported, in part, by the National Institute of Biomedical Imaging And Bioengineering of the National Institutes of Health under Award Number R44EB019802; by the National Institute of Diabetes and Digestive and Kidney Diseases of the National Institutes of Health under Award Number R43DK115332; by the Office Of The Director, National Institutes Of Health of the National Institutes of Health under Award Number R44OD018334; and by the National Institute of Dental & Craniofacial Research of the National Institutes of Health under Award Number R43DE027595. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

]]>https://blog.kitware.com/new-software-from-kitware-virtualizes-medical-education-and-training/feed/024056Important Update for Online Resources While we Move (Edited with Dates)https://blog.kitware.com/important-update-for-online-resources-while-we-move/
https://blog.kitware.com/important-update-for-online-resources-while-we-move/#commentsWed, 05 Dec 2018 15:54:28 +0000http://blog.kitware.com/?p=24049Edit: We plan to relocate from Friday, December 14th, to Monday, December 17th. The impact on our online resources should be limited to Saturday, December ... Read More

Edit: We plan to relocate from Friday, December 14th, to Monday, December 17th. The impact on our online resources should be limited to Saturday, December 15th.

We are closing in on the move of our headquarters to a new spot in Clifton Park, New York! While we are working to ensure a smooth transition, some of our websites and resources will briefly go offline. These include the following:

kitware.com

certain platform websites such as cmake.org, vtk.org, paraview.org and itk.org

mailman mailing lists

projects and repositories on gitlab.kitware.com

Discourse forums such as discourse.kitware.com, discourse.itk.org, discourse.girder.org and discourse.paraview.org

some parts of kitware.fr

In addition, our phone/voicemail system will be temporarily down.

We will note specific dates for our move and the downtime, once they are finalized. We will also post periodic updates on Twitter, so please follow @Kitware. Thank you for your patience!

If you would like to contact us during the move, please use email (e.g., kitware@kitware.com), as it will stay active. Other resources that will remain uninterrupted include these:

blog.kitware.com

jobs.kitware.com

job postings on hire.withgoogle.com/public/jobs/kitwarecom

the Kitware online store

our books on Amazon

kitware.eu

our Twitter feed

our Facebook feed

our LinkedIn feed

our GitHub repositories

our repos on Docker Hub

software documentation on Read the Docs

If you have not yet seen our new space, please visit our Facebook page. We will continue to add pictures there.

With the move, our headquarters will go from two buildings to two floors in the same building. While we will have more space in terms of square feet, in other ways, we will be brought closer together. We will also have a faster network and a better setup for computing resources. For more information, please read “Kitware Opens Doors with New Headquarters.”

]]>https://blog.kitware.com/important-update-for-online-resources-while-we-move/feed/124049CMake 3.13.1 available for downloadhttps://blog.kitware.com/cmake-3-13-1-available-for-download/
https://blog.kitware.com/cmake-3-13-1-available-for-download/#respondWed, 28 Nov 2018 14:09:15 +0000http://blog.kitware.com/?p=23991We are pleased to announce that CMake 3.13.1 is now available for download. Please use the latest release from our download page: https://cmake.org/download/ Thanks for ... Read More

]]>https://blog.kitware.com/cmake-3-13-1-available-for-download/feed/023991VTK 8.2.0-rc2 ready for testinghttps://blog.kitware.com/vtk-8-2-0-rc2-ready-for-testing/
https://blog.kitware.com/vtk-8-2-0-rc2-ready-for-testing/#respondTue, 27 Nov 2018 15:44:17 +0000http://blog.kitware.com/?p=23984The second release candidate for VTK 8.2.0 is ready to try. You can get it from the downloads page, http://www.vtk.org/download/#candidate or from git with the 8.2.0.rc2 tag. ... Read More

The changes over rc1 consist of a handful of bug fixes, most notably a change to fix python wheel building. Things could turn up in testing of course, but I anticipate that this version will be retagged as 8.2.0 final in about two weeks.

Please try this version of VTK, and report any issues to the bug tracker so that we can try to address them before VTK 8.2.0 final. Kindly set the 8.2 milestone on your issue and merge requests for 8.2 to ensure that they are tracked during the remainder of the release process. Further, please base / rebase all 8.2 target branches off of the v8.2.0.rc1 tag to ensure minimal merge conflicts with the release and master branches.