Radiology, Augmented Reality, LLM: Interview with Prof. Jesse Courtier

Jesse Courtier

Dr. Jesse Courtier is an Associate Professor of Pediatric Radiology at the UCSF Benioff Children’s Hospital and co-Founder of Sira Medical, a startup focusing on the application of Augmented Reality for improved medical education, training, and pre-surgical planning. He co-developed the initial versions of the Sira application for Microsoft Hololens and Apple iOS after seeing the challenging disconnect between medical imaging and surgery, subsequently being awarded the UCSF Innovation Ventures Catalyst Award. Dr. Courtier is a multi-award-winning educator and physician leader. His interests lie in the use of advanced imaging technology for medical education and improved patient care. Dr. Courtier will be speaking at our upcoming webinar.

Jenny: When was the first encounter you had with Augmented Reality? What was that experience like? What were you thinking at that moment?

Jesse: I bought my first HoloLens Developer edition after first seeing the potential of Augmented Reality for Healthcare in a Youtube clip, and learned how to create my first application using online instructional videos. Donning the HoloLens and seeing the first 3D image in Augmented Reality was so transformational for me. I saw the power of this technology and determined to devote my focus to developing it further into a truly valuable product for radiologists and surgeons.  

Jenny: What inspired you to start your journey in Augmented Reality?

Jesse: I attended a series of talks on entrepreneurship in Radiology at our largest meeting in the field, RSNA which made me believe that I could take my idea and turn it into something real. After this, I enrolled in our UCSF Entrepreneurship class, and discussing this idea in the reading room with my co-founder Dr. Ben Laguna, decided to take the next steps of formally beginning on a startup journey. We were awarded a catalyst award through UCSF ‘s Catalyst/Innovation Ventures program. Through that program, we met our advisor and now CEO Rick Beberman. Since then we have made our way through the 500 Global Accelerator and other translation programs and subsequently received venture funding, which allowed us to spin out from UCSF.

Jenny: Who inspired you the most along this journey in Augmented Reality? This can be a mentor, a patient, a celebrity, anyone basically. You can name more than one as well.

Jesse: This was motivated by my experiences as both a former surgical intern and as a practicing radiologist. My amazing clinical colleagues were an inspiration as they encouraged me to further develop and pursue my passion for advanced 3D imaging. I strongly believe this technology can truly improve patient care. 

Jenny: What motivates you the most for your work? 

Jesse: My interest is in improving the ability of my radiologists and surgeons colleagues to care for patients and to make them more effective at their jobs. I hope to positively impact patient care using this technology. 

Jenny: What is/are the biggest obstacle(s) in your line of work? If you have conquered them, what were your solutions? 

Jesse: Some of the obstacles faced in this space revolve around demonstrating the “need to have” of this technology as opposed to “nice to have”. We are addressing this through our research and development of the applications and clinical benefits of Augmented Reality. Navigation of the FDA 510 K clearance process has also been an interesting but valuable learning experience. Our consultants have been very helpful in guiding us through this process.

Jenny: What do you think is (are) the biggest challenge(s) in Augmented Reality? What do you think the potential solution(s) is (are)?

Jesse: Challenges include need for continued evolution of the AR hardware to have even greater portability, computational power, and ease of access. I believe the solutions lie in further development of cloud / edge rendering to allow for high polycount models and incorporation of animation into the 3D models. Development of 5G capabilities will also afford a greater degree of mobility.    

Jenny: If you are granted three wishes by a higher being, what would they be? 

Jesse: Great question! I think on a global level would like to see greater interconnected-ness between societies and cultures, more time with family, and personally even greater ability to focus more on creative activities. I do feel through technologies like augmented and virtual reality we can come closer to achieving goals such as these.  

Jenny: What advice would you give to a smart driven college student in the “real world”? What bad advices you heard should they ignore? 

Jesse: I think taking calculated risks are important for personal and professional growth. It may sound cliché, but I do believe you learn and grow from setbacks and challenges. It’s helpful to develop thick skin. As for bad advice, I think most importantly advice that doesn’t resonate with your morals or character is always something to be set aside and should give you pause.

Advanced Visualization in Healthcare: AR/VR/MR (Course on Demand)

Artificial Intelligence in Healthcare 3D Printing (Course On Demand)

Virtual Reality for Surgical Planning: An Interview With Byeol (Star) Kim

Virtual Reality Software For Molecular Modeling and Structure-Based Drug Design

Medical Simulation Using Augmented Reality, Virtual Reality, 3D Printing

Medical Applications of Augmented Reality

Mixed Reality and 3D Printing, Imperfections to Perfection?

From Academia: Mixed Reality, Augmented Reality, and 3D Printing in Healthcare

From 3D Printing to VR/AR: Simple Connection?