Physics-Driven Mask R-CNN for Physical Needle Localization in MRI-Guided Percutaneous Interventions

Discrepancies between the needle feature position on magnetic resonance imaging (MRI) and the underlying physical needle position could increase localization errors during needle-based targeting procedures in MRI-guided percutaneous interventions. This work aimed to develop a deep learning-based framework to automatically localize the physical needle position using only the needle features on MR images. Physics-based simulations were performed to generate single-slice and 3-slice images with needle features from a range of underlying needle positions and MRI parameters to form datasets for training single-slice and 3-slice Mask Region-Based Convolutional Neural Network (R-CNN) models for physical needle localization. Ex vivo tissue images were combined with simulated needle features for fine-tuning. Next, the physics-driven Mask R-CNN models were combined with a previously developed Mask R-CNN model for needle feature localization to form an automated framework to localize the physical needle. To test the accuracy of the proposed framework, both single-slice and 3-slice MRI data were acquired from needle insertion experiments in ex vivo tissue phantoms. Using the single-slice model, the proposed framework achieved sub-millimeter physical needle localization accuracy on single-slice images aligned with the needle. The fine-tuning step reduced in-plane physical needle tip localization error (mean±standard deviation) to 0.96±0.69 mm in ex vivo tissue data. The 3-slice model further reduced the through-plane physical needle tip localization error to 2.3±1.1 mm in situations where the imaging plane may be misaligned with the needle. The processing time of the framework using both models was 200 ms per frame. The proposed framework can achieve physical needle localization in real time to support MRI-guided interventions. ? 2013 IEEE.