Want to write a piece for 3DHEALS Expert Corner? Email us: info@3dheals.com

There’s no question that computer modeling, simulation, and additive manufacturing have transformed clinical medicine around the world. What’s always fascinated me, though, is the variety of ways these technologies have been implemented in different hospitals and even different departments within the same hospital. As a prototyping fellow at Sinai BioDesign, a design and prototyping group within the Mount Sinai Hospital in New York, I’ve seen firsthand almost all of the ways both 3D printing and scanning can be leveraged within a health system. Like many other tools and data streams, there’s no single way 3D medical data is acquired or used within the health system. In each case, though, these printing, scanning, and rendering applications are crucial to clinical care and research. Last week, in part 1 of this  Expert Corner blog, I focused on use cases for 3D modeling and printing. For part 2 this   I’ll be discussing clinical applications of 3D scanning technologies, the other side of the medical 3D coin.

Unlike medical modeling – the 3D rendering of anatomy (oftentimes patient-specific anatomy) – 3D scanning always resides in the digital domain. Nevertheless, there are numerous different technologies employed to capture 3D scan data, each with its own set of pros and cons that, much like the various life cycles of anatomical data covered in last week’s Expert Corner, have utility to different clinical practices. To begin with, it is worth noting the three major groups of 3D scanning technologies: LIDAR, textured light, and photogrammetry.

LIDAR

LIDAR stands for Light Detection And Ranging and is essentially radar or sonar but with infrared light as the signal source instead of radio waves or sound waves. A scan is acquired by rastering an infrared laser light source over an object, capturing its contours and textures. Either the scanning source or the object being scanned must be spatially fixed, though, in order to establish a fiducial reference and coordinate system by which the cloud of 3D points can be plotted as the scan is taken. Together, these points are interpolated to form the surfaces of the scanned object. Depending on the scanning hardware being used, the resolution of LIDAR scanning ranges from several millimeters to a few microns. It’s also worth noting that LIDAR scanning is line-of-sight (LoS), meaning anything the light source can’t reach can’t be scanned. Sometimes a workaround can be achieved by changing the positioning of the scanner or the subject. The highest resolution LIDAR scanners are often employed during the quality assurance portion of the medical device development process to ensure manufacturing is within required tolerances. At lower resolutions, LIDAR scanning is sometimes used to capture complex surfaces of the human body, such as the face.

Textured Light

Textured light scanning works by projecting a series of different sized grids, either in the visible or infrared spectrum, onto the surface of an object. A camera linked to the projection system detects the projected grid and the distortions to the pattern made by the object being scanned are recorded. Based on how the pattern is distorted, an algorithm extrapolates the surface of the object, rendering it in 3D. Like LIDAR scanning, either the scanning unit or the object can be rotated to capture all angles since this scanning approach is also LoS. Textured light is typically regarded as the second-highest resolution 3D scanning technology, after LIDAR. The ultimate determinant of textured light scanning resolution is the combination of the resolution of the projected grid and the resolution of the camera detecting the grid – usually on the order of a millimeter.

It’s also worth pointing out that, for both LIDAR and textured light scanning, there are many materials that are “un-scannable” because of their interaction with the visible or infrared light being employed in the scanning process. Many metallic objects are extremely difficult to scan with LIDAR or textured light because of the way their surface reflects the incident scanning light. In these cases, a temporary matte coating can be applied to improve the optical properties of the material.

Photogrammetry

I often tell people that if there’s a method of 3D scanning that they’ve seen before, it’s photogrammetry. This approach captures a 3D rendering of an object by taking 2D pictures from multiple positions, either by using one camera and capturing one angle at a time, or more commonly through the use of large camera arrays. Photogrammetry is particularly appealing in many fields because (at least in the case of multi-camera arrays) it is an extremely quick method of 3D scanning. In a 360-degree photogrammetry array – on the order of 80 cameras oriented around a platform at many different angles – it takes only as long as it takes to snap a picture to capture a 3D scan. Compare this to LIDAR and textured light, which can take several minutes at their fastest.

Figure 4. An example of a photogrammetry scan with (left) and without (right) the 2D texture mapped to the 3D surface. While some features, such as the eyebrow are still relatively easy to distinguish, details around the eye become much harder to pinpoint.

As is often the case, though, the advantages of photogrammetry come with tradeoffs. Chief among them is the poor resolution of the technique. This often comes as a surprise to many because, at first glance, the 3D models rendered appear very detailed. This detail, though, does not reflect the geometry of the 3D object produced through the scanning technique but rather reflects the 2D texture of the scanned object mapped to a low-resolution. It is this 2D texture – comprised of the individual images of several dozen cameras, often DSLRs – that creates the illusion of a 2D model. Remove that texture (as would be the case for any single-color method of 3D printing) and all your left with is a blob generally shaped like the object you were scanning. Don’t get me wrong, there are certainly times where the geometry captured by photogrammetry is sufficient for the end application of the 3D model, and some geometries capture better than others (think about what a person with their arms raised might look like, versus someone with their arms crossed), but on the whole, this method has a resolution on the order of centimeters, or worse.

Clinical Applications

So why did I go to all that effort to explain these three major groups of scanning technologies? Because different ones are employed by different clinical fields, and for different reasons. At Mount Sinai alone, two or more of these 3D scanning technologies are being used almost daily.

The first example I’d light to highlight is the Fit3D scanner used by Sinai’s Institute for Next Generation Healthcare. This device is part scanner, part scale, and uses a LIDAR scanner and rotating platform to measure a patient’s weight, body fat percentage, and water content, among other metrics (similar to any other high-tech digital scale today) while also capturing a full-body 3D scan. The end result, on the scanning side, is a marble statue-like rendering of the subject, which can subsequently be measured according to metrics like waist-to-hip ratio, height/waist circumference, and other emerging metrics that look at physiological health beyond BMI. For these measurements, a reasonably high-fidelity 3D reproduction of the patient is important, in order to derive accurate measurements from the 3D scan. The texture of the subject (e.g., the color/pattern of their shirt) has no bearing on these metrics. As such, the LIDAR scanning approach is far superior to photogrammetry for the data it provides, even though the scanning process takes about a minute to complete, versus a few seconds.

On the other hand, the lab of Dr. Ethylin Jabs, in Mount Siai’s Genetics Department, uses a scanning system called 3dMD, which uses photogrammetry techniques to capture the faces of patients. Many genetic conditions manifest in the form of facial asymmetries and malformations, which can be captured by the scanning system and subsequently detected by clinicians reading the scans.

Given what I’ve just explained about the various scanning technologies, though, you might be wondering why the Jabs lab uses photogrammetry when this is the lowest resolution form of scanning. The situation only becomes more perplexing when you see that the 3dMD system is a fairly large, elaborate system that’s not very easy to move. Couldn’t a LIDAR or textured light technique capture a better rendering of the face? While the answer to that question is “yes”, it doesn’t fully capture the method in which these scans are being used. It turns out, as I found out several years ago when I showed some different LIDAR and texture light scanners to Dr. Jabs, that there are some very good reasons why her lab uses photogrammetry to capture these 3D scans. First and foremost, many of her patients are infants – a group of people infamous for their inability to hold still. While it might technically be possible to use LIDAR or textured light to capture a scan of an infant, it’s certainly a much longer process than the 1-2 seconds it takes to snap a photo. Photogrammetry lets Dr. Jabs’ team see and process the data for many more patients than any other scanning technique would. Second, the facial metrics being extracted from these scans rely on the accurate identification of facial features (e.g., the outside corners of the eyes), which are rendered much more clearly through the 3D-mapped textured captured by photogrammetry. Thus, even if there is some inaccuracy incurred due to the lower spatial resolution, it’s more than made up for by the precision with which clinicians can identify facial features when extracting anatomical measurements to correlate with genetic data.

Conclusions

Be it 3D printing or 3D scanning, the adoption of 3D visualization and fabrication techniques in hospitals has been transformative in the delivery of patient care and the expansion of personalized medicine. By digitizing and systematizing this information, it has become possible to better prepare for surgery, improve the detection of genetic defects, and even improve the accuracy of measurements as basic as height and weight. We’re still a little ways off from everyone getting a full 3D scan workup as part of their yearly physical, but I think these examples and the spread of scanning technology overall show that we’re well on our way.

About the Author:

Joseph Borrello is currently a biomedical engineer and Ph.D. Candidate at Mount Sinai, working in the labs of Drs. Kevin Costa and Junqian Xu, in addition to managing digital fabrication operations within the Sinai BioDesign innovation team. Previously, he worked at 3D Systems on technical development in the consumer marketing department and as a liaison with engineering project management teams.

He received his bachelors in Biomedical Engineering from Macaulay Honors College at The City College of New York, where I remain active in the Zahn Innovation Center, an on-campus tech startup incubator.

Joseph is also an active member of the New York City startup ecosystem. He is the founder of Proto-Sauce, which is developing new materials for resin-based 3D printing, as well as the CTO of Biosapien, leveraging 3D printing to produce personalized therapeutics. He also tries to summarize as many of the local happenings as he can in his newsletter Magnitude and Direction.

Finally, Joseph is also the editorial assistant for 3DHEALS Lattice newsletter, where he tirelessly curate the best content for healthcare 3D printing and bioprinting community with the 3DHEALS team.

Related Articles:

Clinical Applications of Medical Modeling- Part One

3D Scanning and 3D Printing for Creating Affordable Prostheses

3D Scanning for Prostheses

What Matters When Enabling Your Dental Practice with 3D Printing and 3D Scanning

The post Clinical Applications of Medical Modelling (Part 2) 3D Scanning appeared first on 3DHeals.

There’s no question that computer modeling, simulation, and additive manufacturing have transformed clinical medicine around the world. What’s always fascinated me, though, is the variety of ways these technologies have been implemented in different hospitals and even different departments within the same hospital. As a prototyping fellow at Sinai BioDesign, a design and prototyping group within the Mount Sinai Hospital in New York, I’ve seen firsthand almost all of the ways 3D printing, and more broadly, 3D data can be leveraged within a health system. Like many other tools and data streams, there’s no single way 3D medical data is acquired or used within the health system. In each case, though, these printing, scanning, and rendering applications are crucial to clinical care and research. For part 1 of this Expert Corner blog, I’ll be focusing on use cases for 3D modeling and printing and in next week’s part 2 , I’ll be discussing clinical applications of 3D scanning technologies, the other side of the medical 3D coin.

At Sinai BioDesign, a large portion of our 3D modeling and printing work is focused on pre- and peri-operative applications; providing models of patients’ anatomy for crucial guidance before and during surgical procedures. It often comes as a surprise to people, though, to learn how often we never produce a printed model from the data we acquire. No matter the surgical plan, a 3D model (typically an STL file) is produced from medical imaging data (usually CT and MRI scans). Printing that 3D model, however, is not always the next step in the process.

At Mount Sinai, the decision on whether or not to print a model often falls along surgical departmental lines, with rationales rooted in the nature of different surgeries. Our Department of Neurosurgery, for example, rarely leverages 3D printing in their pre- and peri-operative surgical guidance, opting instead to render the data with a variety of VR and AR tools. For pre-surgical planning, technologies like the Surgical Theater platform are used to produce 3D models of patients’ anatomy, which the surgeon can then manipulate on interactive displays and even “walk” through using VR headsets. While a 3D printed copy of a patient’s anatomy is easier to probe, manipulate, and analyze than the actual patient’s anatomy in the OR, it often doesn’t afford the kind of perspectives and spatial understanding that can be achieved through a VR rendering. Any view that would be blocked by actual anatomies, such as the skull, is also frequently blocked by the printed anatomy. In VR you can shift the transparency of different tissues in much the same way you can adjust the transparency of layers in Photoshop, but you can adjust colors and transparency in a 3D printed model.

Furthermore, the tissues neurosurgeons operate on – mostly brain, blood vessels, and tumors located in and around brain and blood vessels – are very soft and materials with accurate mechanical properties cannot be produced through existing 3D printing technologies. As such, there is little to no tactile information gained through a printed anatomical model and the visual/spatial information is at best as good as what can be obtained through rendering the 3D models in VR. Lastly, Mount Sinai’s Neurosurgery Department frequently employs AR, heads-up display technologies as a means of peri-operatively visualizing anatomical data. This ability to overlay and highlight important anatomical information (say the location of a brain tumor, or important blood vessels that should be avoided) on the live visuals being captured in the OR is arguably even more useful than looking back and forth between a printed model and the patient (at least in the case of neurosurgery).

Although, as the neurosurgery examples show, the lifecycle of an STL in a hospital doesn’t always lead to a 3D printer, there are many cases in which it does, and many departments that almost always print anatomical models for surgical guidance. For purposes of surgical planning, the Ear, Nose, and Throat (ENT) department at Mount Sinai is one of the most avid users of printed anatomical models. Many ENT procedures require the use of endoscopic cameras and surgical tools, typically inserted via the nostrils. Unlike neurosurgery, the tissues involved in ENT procedures are bony and cartilaginous, much stiffer and more reasonably approximated by the thermoplastics and ceramic powders our anatomical models are printed out of. As such, performing a dry run of a planned ENT procedure on the printed copy of a patient’s anatomy comes much closer to approximating the conditions that will be experienced in the OR. Very often, this means using the same types of endoscopes as catheters on the printed model as would be used on the patient while evaluating maneuverability, fit, and overall strategy.

When not intended for patient-specific practice, Sinai BioDesign’s printed anatomical models are frequently used for more generalized practice, especially for neurological procedures such as stereoelectroencephalography (SEEG), which involved the placement of electrodes deep into the brain. As you might imagine, a procedure that involves drilling through the skull and inserting foreign objects into the brain is also a procedure that requires a lot of training and practice. Many of the commercially available practice models do a poor job of replicating actual human anatomy, so we’ve begun producing our own practice models using a combination of 3D printing and casting techniques. The skull and key vasculature to be avoided are printed from a hard, ceramic powder (which happens to feel fairly similar to bone, with respect to drilling) and a cast of a brain (my brain, in fact[1] ) made from a cryogel that replicates the mechanical properties of brain tissue is inserted inside the printed skull. The fully assembled models are then used with the SEEG drilling equipment for training courses for surgeons. These hybrid printed/cast anatomical models have been so successful at replicating human anatomy, at both the physical and mechanical levels, that the production of skulls for SEEG training is likely the single largest application of 3D printing at Mount Sinai.

When considering the impact of 3D printing in medicine, we often primarily consider the impact of printed objects that are directly integrated into the surgical cycle, be they implants, prostheses, or pre-surgical planning models. It is important to remember, though – and I hope my examples from Sinai BioDesign illustrate this – that there are many ways 3D printing (and the 3D files that precede printing) impact clinical care even when not directly integrated into a surgery. Printed models, like our SEEG dummies, are helping democratize training for complex surgical procedures and creating more realistic practice environments. Furthermore, the digital models that produce all of these printed objects don’t even need to be physically manifested to have crucial clinical utility. In cases such as neurosurgery, where a printed model doesn’t necessarily add much to the pre-surgical planning process, VR pre-surgical “walkthroughs” and AR peri-operative guidance can decrease procedure durations and improve outcomes. 3D medical technologies will likely never be used in the same way everywhere, but I think it’s clear all of them will play a crucial role somewhere.

About the Author

Joseph Borrello

Joseph Borrello is currently a biomedical engineer and PhD Candidate at Mount Sinai, working in the labs of Drs. Kevin Costa and Junqian Xu, in addition to managing digital fabrication operations within the Sinai BioDesign innovation team. Previously, he worked at 3D Systems on technical development in the consumer marketing department and as a liaison with engineering project management teams.

He received his bachelors in Biomedical Engineering from Macaulay Honors College at The City College of New York, where he remains active in the Zahn Innovation Center, an on-campus tech startup incubator.

Joseph is also an active member of the New York City startup ecosystem. He is the founder of Proto-Sauce, which is developing new materials for resin-based 3D printing, as well as the CTO of Biosapien, leveraging 3D printing to produce personalized therapeutics. He also tries to summarize as many of the local happenings as he can in his newsletter Magnitude and Direction.

Finally, Joseph is also the editorial assistant for 3DHEALS Lattice newsletter, where he tirelessly curate the best content for healthcare 3D printing and bioprinting community with the 3DHEALS team.

Related Articles:

Smart Spine Surgery- From Planning to 3D Printed Templates

3D Printed Orthopedic Implants in China and the Challenges in Commercialization

Bio Simulator and 3D Printing

The post Clinical Applications of Medical Modeling- Part One appeared first on 3DHeals.