A short History of the development of 3-D Ultrasound in Obstetrics and Gynecology

Dr. Joseph Woo

Three-dimensional ultrasound comes of age.
Visualization of the fetus in 3-D has always been on the minds of many
investigators, including Tom Brown in Glasgow
in the early 1970s, who had developed an elaborate Multiplanar scanner in 1973, under the Sonicaid Ltd®. With improvements in ultrasonic and computer technology, work on
three-dimensional visualization began to appear in the
early 1980's. Some basic computer algorithms came from the group at Stanford (JF Brinkley, WD McCallum and others) and also from the Holm group at Gentofte, Denmark. Other work came from the domain of cardiologists where initial efforts were directed to acertaining the volume of cardiac
chambers. Real-time scanner probes mounted on articulated arms were often employed where positions
of the probe can be accurately determined. The principle has always been to stack successive parallel image sections together with their positional information into a computer.

Kazunori Baba at the Institute of Medical Electronics, University of Tokyo, Japan, first reported on a 3-D ultrasound system in 1984 and succeeded in obtaining 3-D fetal images by processing the raw 2-D images on a mini-computer in 1986. Their setup was reported in the Acta Obstetrica et Gynaecologica Japonica. Baba, with Kazuo Satoh and Shoichi Sakamoto at the Saitama Medical Center described the improved equipments in 1989 in which they used a traditional real-time convex array probe from an Aloka SSD280 scanner mounted on the position-sensing arm of a static compound scanner (Aloka M8U-10C). The images obtained were processed on elaborate computer systems (see picture with description below). This approach successfully 3-D images of the fetus which were nevertheless inferior to that produced on convenional 2-D scanners. At the same time, to generate each 3-D image it took on an average some 10 minutes for data input and reconstruction making the setup impractical for routine clinical use. Baba published in 1992 in the Japanese language the first book on ultrasonography in Obstetrics and Gynecology which contained chapters on 3-D ultrasound. In the mid 1990s, Baba collaborated with
ALOKA®
with technology developed at the Biomedical Engineering Department of the Tokyo University, and was a driving force in the development of commercial 3-D ultrasound technology in Japan.

Kazunori Baba's 3-D setup in the mid 1980s. A linear array probe was mounted on an articulated arm for position sensing. On the right is the computer setup for making the calculations. (click on the picture for larger view and description)

Another group at the Columbia University led by Donald
King described in 1990 other approaches and computer algorithms for
3-D spatial registration and display of position and orientation of real-time
ultrasound images. HC Kuo, FM Chang and CH Wu at the
National Cheng Kung University Hospital in Taiwan, Republic of China,
reported in 1992 3-D visualization of the fetal face, cerebellum, and
cervical vertebrate using a the Combison 330 from Kretztechnik®, Zipf,
Austria. The Combison 330 which appeared in 1989, was the first commercial 3-D scanner in the market. The Taiwanese group were also the first to describe 3-D visualisation of the fetal heart in the
same year although at that time they were only able to image static parts in
3-D.

In 1987, the Center for Emerging Cardiovascular Technologies at Duke University started a project to develop a real-time volumetric scanner for imaging the heart. In 1991 they produced a matrix array scanner that could image cardiac structures in real-time and 3-D. In 1994, Olaf von Ramm, Stephen Smith and their team produced an improved scanner that could provide good resolution down to 20 centimeters. The team developed state-of-the-art "Medical Ultrasound imaging" integrated circuits (MUsIC) which were capable of processing signals from multiple real-time phased-array images. The microprocessors were developed in collaboration with the Volumetric Medical Imaging Inc. at Durham, North Carolina. The MUsIC 3.2, a 40MHz 1.2µ chip completed in 1994, was the basis for the beam-former in the world's first electronically steered matrix-array 3-D ultrasound imager. This became available commercially from Volumetric Medical Imaging, Inc. in 1997.

The matrix-array transducer, which steered the ultrasound beam in three dimensions, contained 2,000 elements in which 512 were used for image formation. The beam-former produced 4,096 lines running at 30 frames per second. This required as much ultrasound signal processing power as eight top-end 2-D systems, running on microprocessors that execute instructions 30 times the speed of a 2 GHz Pentium V. Due to the relatively small size of the 2-D matrix array probe, it is more suited to cardiac examination rather than for the abdomen. The apparatus is also costly to produce and poses problem in manufacturing and in image quality due to the large amount of crystals and interconnections.

Other pioneering investigators included Ian Kelly and
John Gardener at the Middlessex Hospital in London and the Sturla
Eik-Nes group at Tronheim, Norway, using equipments from Vingmed® which were partly developed at the University's bio-engineering department (see Part 2) . They were able to
demonstrate early gestational age fetuses with their apparatus. Wilfried
Feichtinger at the University of Vienna, Austria reported images of 10
weeks embryos imaged with 3-D transvaginal transducers in 1993.
Kretztechnik®. had in this year marketed their 2nd generation 3-D
scanner the Voluson 530D. Alfred Kratochwil had continued
his support in the development of 3-D technology at Kretztechnic® and was active in the teaching of 3-D sonography after his retirement. The
Ulrike Hamper group at Johns Hopkins reported images of various
congenital malformations with a prototype 3-D scanner. Computation was
based on a 486 computer together with a RISC processor (860/240 mhz).

Thomas Nelson
and Dolores
Pretorius at the University of California, San Diego,
approached the carotid arteries with their prototype 3-D system in 1992
and produced very successful images. The signal chain consisted of a
transducer-array moving along the patient's neck producing sequentially
sampled images which were digitised, acquired and surface-rendered on the
connecting workstation. They collaborated with development by Donal Downey and Aaron Fenster at the Imaging Research Laboratories of the John Robarts Research Institute at the University of Western Ontario, Canada.

Their group continued to make refinements to the
instrumentation and started to publish on fetal visualization in the
following years and continuing on to become one of the most important research
teams in the field of 3-D ultrasound in Obstetrics and Gynecology. In 1996,
Nelson's group and the Medical Imaging group at the university College Hospital in London published
independent researches on 4-D (motion 3-D) fetal echocardiography, using
sonographic cardiac gating methods to remove motion artefacts, which are
present with conventional (static) 3-D methods. A useful feature of 3-D display is the cine loop, in which the rendered 3-D volumes are viewed as they rotate. This capability enhances depth perception and gives a true 3-D perspective of both normal and abnormal structures.

In 1995, Eberhard Merz at the Center for Diagnostic Ultrasound and Prenatal Therapy, University of Mainz, Germany, demonstrated the usefulness of multiplanar orthogonal imaging as well as as surface views and transparent views in the diagnosis and confirmation of fetal surface and skeletal anomalies such as cleft lips and complex multiple malformations. He and his co-workers reported a large series of over 600 cases of fetal diagnosis using 3-D ultrasound. In 1997, his team reported on the diagnosis of facial anomalies using trans-vaginal 3-D scans.

In Obstetrical and Gynecological 3-D imaging, mechanical designs appeared to be the only popular choice. Two-dimensional arrays are mechanically moved to provide the third dimension by sweeping or rotating, using either constrained free-hand adapters or an existing probe alongside with an external motion-sensing system. The most sucessfully deployed transducer design is the mechanically-driven arrays that is built-in into the probe housing from the Austrian manufacturer Kretztechnik®. Their technique was described in the paper "3D ultrasound - the Kretztechnik Voluson approach" in the European Journal of Ultrasound in 1994.

The process of acquisition is microprocessor-controlled and automatic. In the display of the acquired data, the degree of transparency is first chosen which involves applying a mixture of ray-traced, volume-rendered illumination and maximum intensity or summed voxel projection. Perception in 3-D surface is acheived by a combination of depth shading, color-mapping, texture mapping and ray-traced volume rendering. The introduction of Multiplanar reformatting has allowed the generation of any arbituary slice within the data aquired. In obstetrics this is valuable for measurement, and for obtaining re-constructed critical views (such as the 4-chamber view) or scans orthogonal to the face and soft palate. All these are heavily dependent of the software algorithms and processing power of the computers within the machines.

Volume rendering in medical imaging has in fact much of its roots in computer graphics engineering. Volume rendering developed as a separate body of techniques, mainly within the computer graphics literature, before and independent of its application to medical data. One of the earliest pioneers in volume rendering is Marc Levoy at the University of North Carolina (now at Stanford). Volumes are rendered directly from sampled data without first creating an intermediate surface representation. This creates images that represent the underlying data very accurately and can reveal fine details that might be obscured with surface methods. Following his initial paper in 1988: "Display of Surfaces from Volume Data" where he described the classic volume ray tracing algorithm and has been the basis (directly or indirectly) for most commercial 3D ultrasound systems, Levoy has published a number of papers that have continued to break new ground. Volume rendering has made a major impact on the many scientific, engineering and medical disciplines that create and display large multi-dimensional datasets.

Many of the volume rendering agorithms and technology had actually originated from computer scientists at the filmmaking company Pixar Animation Studios, famous for its 3D computer animated films! Initial volume rendering techniques and agorithms weres "invented" by company founders Bob Drebin, Loren Carpenter, and Pat Hanrahan. The algorithm embodied three key ideas: Directional shading based on the gradient in a volume, digital compositing to combine the slices of a volume, and Image warping, also applied to a volume.

Gradient shading of volumes first appeared in a 1986 paper by Karl-Heinz Hoehne, hamburg, who called it graylevel gradient shading. Compositing can be traced through Thomas Porter and Tom Duff's 1984 paper to Edwin Catmull and Alvy Ray Smith's invention of the alpha channel in the mid-70's. Image warping is a special case of texture mapping, which dates back to Edwin Catmull's 1974 PhD thesis at the University of Utah.

Volume rendering approximates the passage of light through a participating media. In this respect, Jim Blinn's 1982 paper on clouds and dusty surfaces, Jim Kajiya's 1984 paper on volume densities, and Holly Rushmeier's 1987 paper with Ken Torrance on zonal radiosity must also be regarded as formative. This relationship between volume rendering and light transport was pointed out to the volume rendering community by Wolfgang Krueger. Another key pioneer was Gabor Herman at the City University of New York, who in a 1979 paper with Ksun Kao Liu proposed using cube-shaped voxels to display computed tomography data ^^.

Medison®, which had acquired Kretztechnik® in 1996 continued to produce more advanced versions of the Voluson series of scanners that produced some of the best 3-D images in the market. Bernard Benoit in Nice, France working in collaboration with Kretztechnik®, published some of the earliest, most stunning and convincing 3-D images in the mid- 1990s using prototype Voluson scanners (see picture of cleft lip on the left). His pictures had been responsible for drawing the attention of many to this new scanning modality.

In the second half of the 1990's at least twenty other important centers worldwide were embarking on distinctive laboratory and clinical research into 3-D ultrasound, usually backed by work done at their own university's medical physics and bio-engineering departments, or commercial enterprises. Many of the innovations relied heavily on software programming. 3-D ultrasound technology would not have been a viable commercial proposition if not for the fact that computer technology was in the mid to late 1990s at a stage where the computations can be done with staggering speed and at an acceptable cost.

In November 1996, with technical assistance from Takashi Okai and Shiro Kozuma from ALOKA®, Kazunori Baba published in the Lancet their initial experience with real-time processable 3-D, which used a simpler algorithm compared to conventional 3-D rendering. Fetal surfaces are demonstrated in near real-time imaging basing on simple 'accoustic impedance thresholding' to identify fetal surfaces in the amniotic fluid. Image quality was very high and required less expensive computers to make the calculations. However the viewing direction is limited to that of the probe so that a desirable direction may not always be possible. Sufficient amniotic fluid is also prerequisite to a good scan. Similarily image clarity depends on the difference between the accoustic impedance of the fetal skin and that of amniotic fluid which made good images difficult to obtain before 20 weeks. Aloka® adapted the technology into their commercial scanners.

The increasing availabililty of 3-D ultrasound has resulted largely from the rapid advancement in computer technology and the decreasing cost of micro-processor electronics. The benefits that 3-D has bought to ultrasound diagnosis has quickly become a matter for debate. The important advantages of 3-D over conventional 2-D ultrasound as it is at the turn of the millenium are its ability to enhance maternal-fetal bonding, improved comprehension of certain fetal anomalies by parents, Improved recognition and better confirmation of certain anomalies such as cleft lips, polydactyl, micrognathia, malformed ears, club foot, vertebral malformations and other anomalies appearing on the 'exterior' of the fetus, consequent to the benefit of volume and surface rendering. The development of transvaginal 3-D probes have further enhanced its value in the early diagnosis of congenital malformations.

In a recent article by Asim Kurjak and his team, "Three-dimensional sonography in prenatal diagnosis: a luxury or a necessity?" (Journal of Perinatology, issue 3, 2000), he concluded,

".... the main advantages of three-dimensional ultrasound in perinatal medicine and antenatal diagnosis include scanning in the coronal plane, improved assessment of complex anatomic structures, surface analysis of minor defects, volumetric measuring of organs, "plastic" transparent imaging of fetal skeleton, spatial presentation of blood flow arborization and, finally, storage of scanned volumes and images. It is our decided opinion that three-dimensional sonography has gained a valuable place in prenatal diagnosis, becoming a necessity for every modern perinatal unit .... ".

Stuart Campbell at the St. Georges Hospital in London was one of the early proponents for the 3-D scan to be an important catalyst for mothers to bond to their babies. What are known as 're-assurance scans' and the rather misnamed 'entertainment scans' have started to develop. The attraction of being able to look at the face of your baby before birth was enthusiastically reported in lay parenting and health magazines. Manufacturers had adopted an unprecedented "profit marketing" and "reverse marketing" strategy to advertise to consumers rather than providers, particularly after the advent of the 4-D (dynamic or motion 3-D) machines.

Barbara Maier and his group (Horst Steiner, Alf Staudach etc.) in a study in Salzburg in 1996 reported that mothers are more incentive to endure pregnancy-related difficulties, reduced anxiety, and improved capacity to cope. Pretorius reported in the same year that improved bonding between the mother and fetus could motivate mothers to refrain from smoking and other harmful behaviors during pregnancy.

Usefulness has been reported for calculating volumes of the gestational sac, fetal lungs and heart from second trimester to term, placental volume, liver volume, and thigh and abdominal volume for the estimation of fetal weight. Andreas Lee with the Kratochwil group evaluated the accuracy of fetal weight estimation using 3-D abdominal and thigh volumes and reported in 1994 similar mean errors as compared to standard 2-D evaluations. Fong-Ming Chang's group at the National Cheng Kung University Hospital in Taiwan, Republic of China reported in 1997 the feasibility of predicting birth weight by upper arm volume measured in 3-D. In 1998, Harm-Gerd Blaas at Tronheim, Norway published 3-D studies of embryos that were less than 10mm and further expanded the usefulness and importance of 3-D sonography as an in-vivo research tool in fetal embryology. 3-D power doppler in the study of placental blood circulation was first described in 1996 by C J Ritchie in a proprietary setup at the Bioengineering Department of the University of Washington, Seattle. Dolores Pretorius published on its usefulness and techniques in 1998. The Croatian group led by Asim Kurjak and Sanja Kupesic expounded this new diagnostic entity further. Their book "Three-Dimensional Power Doppler in Obstetrics and Gynecology
" was published in 2000.

In gynecological applications, Davor Jurkovic at Kings convincingly demonstrated in 1995 the usefulness of 3-D ultrasound in accurately differentiating uterine anomalies such as bicornuate uterus and septate uteri. Similarily the assessment of the endometrial cavity with 3-D sonohysterography and characterization of endometrial masses, adhesions, tubo-ovarian masses, hydrosalpinges, ovarian cysts, small intraovarian tumors and mullerian anomalies have all been quickly and convincingly demonstrated. Diagnostic accuracy of malignant ovarian tumors can be up to 100 percent as reported by Bonilla-Musoles in 1995, who also demonstrated the value of 3-D examination over the convention 2-D transvaginal scans and 2-D sonohysterography in the diagnosis of endometrial lesions. 3-D color power doppler is valuable for visualization of intra-tumor flow and thus is useful in evaluating in particular cervical carcinomas and ovarian carcinomas. It is envisaged that the investigation will lead to greater appreciation of tumor angiogenesis. Another potential benefit of 3-D ultrasound lies in data documentation, storage, and networking. Digitally saved volumes of patient data can be readily transferred to a remote site for interpretation or second-opinion consultation. How much these all add up to make 3-D ultrasound cost-effective and an indispensible tool in Obstetrics and Gynecology will remain to be seen.

3-D picture at the top courtesy of Dr. Bernard Benoit.
Picture of parenting magazines coutesy of Medison®.
Dr. Baba's images of his setup courtesy of Dr. Baba.
^ In Diagnosis and Therapy of Fetal Anomalies. John C Hobbins and Beryl R Benacerraf (eds), Churchill Livingstone, 1989.
^^ Excepted with permission from the homepage of Dr. Marc Leroy, Stanford University: Citation; SIGGRAPH 1996 Computer Graphics Achievement Award
- It is not possible to include all the names who have contributed significantly to the advancement of Obstetrical and Gynecological sonography, some who may have been less well-known than the others and some who may not have published so extensively in the English language. Apologies are extended to those whose contribution has not been fully credited in this article.