Interview: Dr. Jesse Courtier, Improving Healthcare with 3D Printing and Augmented Reality

Dr. Jesse Courtier is an Associate Professor of pediatric radiology at the UCSF Benioff Children’s Hospital and founder of HoloSurg3D. He has co-developed the medical imaging augmented radiology application RadHA for the Microsoft HoloLens. His interests lie in use of imaging technology for medical education and improved patient care. Dr. Courtier will be a speaker at the #3DHEALS2018 conference on April 20-21st, 2018. 
Jenny: When was the first encounter you had with 3D printing? What was that experience like? What were you thinking at that moment?
Jesse: I’d heard buzz about the potential of 3D printing from news articles, but I first really encountered 3D printing at the Radiologic Society of North America national meeting two years ago. I was really excited and immediately saw the value of using 3D imaging to bridge the “2D-3D gap” in communication between radiologists and surgeons that I’d noted in my day to day work as a pediatric radiologist.
Jenny: What inspired you to start your journey/company/career/research in 3D printing (bio-fabrication/bio-printing)?
Jesse: My earlier research involved diffusion tensor imaging of renal transplants, and from that research, I generated some beautiful tractography images. After RSNA, I was wondering if I could 3D print these fiber tracts. I learned a lot through the process of trying to print these, which I was able to do successfully using the Carbon printer working together with Sculpteo here in the Bay Area.   (below is my first 3d print of renal medullary pyramids derived from an MR diffusion tensor imaging scan).

As I progressed making other models, I started looking further into other 3D display methods and began my journey into the field of Augmented Reality. I found many of the skills I’d acquired in my 3D printing experience directly translated to creating these Augmented Reality models. I further explored the idea of commercialization of this concept through our UCSF Startup 101 course and the idea for HoloSurg3D was born! My co-founder Ben Laguna (a senior radiology resident at UCSF and UPenn Medicine graduate) were fortunate to be supported in our endeavors with excellent advice through UCSF QB3 and a UCSF Catalyst consultation award. We were also fortunate to be selected for the UC Hastings Startup Legal Garage which paired us with the law firm Kilpatrick and Townsend and allowed us to incorporate.

Our overall goal with HoloSurg3D is to improve pre-surgical planning by making patient-specific Augmented Reality models to allow surgeons think through a case 3-dimensionally before it ever happens. We’re effectively bridging the gap between the 2D world of radiology and the 3D real world of surgery.

Our application RadHA (Radiology with Holographic Augmentation) allows users to load 3D Augmented Reality models via the cloud, view and manipulate them using the Microsoft HoloLens platform. We offer an A.R. model creation service tailor-made for our software. We believe the use of this platform will reduce complications and decrease operating room time, letting surgeons plan more realistically and better anticipate potential problems.  We’ve been making our A.R. models here at UCSF Benioff Children’s Hospital for our pediatric surgeons and pediatric neurosurgeons and plan to expand to adult services by the second quarter.

The portability of this solution allows users to use our solution in the reading room, in clinic, at or projected in a surgical planning conference room…really anywhere needed.

Jenny: Who inspired you the most along this journey in 3D printing (bio-printing/bio-fabrication)? This can be a mentor, a patient, a celebrity, anyone basically. You can name more than one as well.
Jesse: Really the inspiration was from the reactions of the clinicians I work with when I demonstrated these 3-dimensional models to them. I’d often noted that it wasn’t until I’d shown a 3D image of the anatomy that I would see the lightbulb go off. The “wow” reactions from surgeons when they immediately see the value of these models is very exciting and inspirational.
Jenny: What motivates you the most for your work?
Jesse: For one, I find a certain beauty in the anatomic structures and in creating these 3-dimensional models. I also, however, feel a sense of accomplishment when I can help to make difficult to understand anatomy clearer to those who need to have a strong grasp of it. Having a perfect 3-dimensional understanding of the anatomy in my head is only useful in as far as I’m able to convey that accurately to the surgeon. As a radiologist, 3D images be they 3d printed or Holographic Augmented Reality models are tools to help me do one of the most important things a radiologist needs to do and that communicates with clinicians
Jenny: What is/are the biggest obstacle(s) in your line of work? If you have conquered them, what were your solutions?
Jesse: In my area, it is finding the time and resources to fully develop my idea. I was fortunate to have great resources and advice from within my University here at UCSF including our QB3, Catalyst program, and Entrepreneurship Center to help take my idea from beyond the “idea stage” into reality.
Jenny: What do you think is (are) the biggest challenge(s) in 3D Printing/bio-printing? What do you think the potential solution(s) is (are)?
Jesse: I think for 3D printing the challenge lies in overall scalability and the need to simplify the segmentation/post-processing of the models. This can be addressed as technology evolves to make these solutions cheaper, easier to use, and more widely available. It is also for this reason that I have pursued Augmented reality solutions as an extension of 3D printing that allows for a wider audience to have access to 3-dimensional models.
Jenny: If you are granted three wishes by a higher being, what would they be?
Jesse: Hmm…tough question. World peace, end to poverty, and an infinite number of additional wishes?
Jenny: What advice would you give to a smart driven college student in the “real world”? What bad advice you heard should they ignore?
Jesse: Would say pursue opportunities that are interesting to you and not overcommit to other things that are less interesting out of a sense of obligation. I think inspiration and motivation come from questions that arise from your own experiences. It is far less interesting and motivating to simply work on a project someone has given you because they think you’d be a good fit for it.
Jenny: If you could have a giant billboard to promote a message to millions and even billions of people in our community (i.e. healthcare 3D printing and bio-fabrication), what message would that be?
Jesse: Continue to innovate, don’t be deterred by those who say you shouldn’t pursue your goals because it will be too difficult (aka naysayers).
Jenny: What were/was the best investment you made in 3D printing?
Jesse: Probably the grant money I used to create my first 3D printed renal tubule model as it led me on my wonderful journey through 3D printing and Augmented Reality.
Jenny: What were/was the worst investment you made in 3D printing/bio-printing/bio-fabrication?
Jesse: Hard to say, haven’t found a regrettable decision yet along this path, but will keep you updated!
Jenny: What was/is the biggest risk you took in your career?
Jesse: Departing from the usual academic path of pursuing research in a very small/obscure niche to instead explore what I truly found interesting and exciting.
Jenny: What do you enjoy in your spare time? What are you passionate about outside of your work/3d printing?
Jesse: I love to spend time with my wife and 5-year-old daughter and enjoying all the SF Bay Area has to offer. I have another soon-to-be-born daughter on the way at the end of this February! When I’m not spending time with them I enjoy practicing Brazilian Jiu-Jitsu. It’s fun and a great way to exercise!
Jenny: What is your favorite quote? Why?
“If you can trust yourself when all men doubt you,
But make allowance for their doubting too;” from Rudyard Kipling’s “If”.
I sometimes think of that when I have to make a tough call on a challenging radiology case, specifically if the clinicians aren’t completely convinced. It’s a gratifying feeling though when you get that call from the surgeon saying, “you were right, it was exactly like you said it would be”. In some ways, I’m hoping with 3D printing and Augmented Reality technologies we’ll be able to much more easily demonstrate what’s in our heads as radiologists so the convincing will be much more effortless!
Jenny: What does the word “3DHEALS” mean to you? =)
Jesse: I think it can mean a number of things but really for me, I think it’s about using 3-dimensional imaging for the better care of our patients. These technologies can help us to better do what we entered medicine to do and that is to help heal the sick.