2021 International Conference on Medical Imaging Physics and Engineering and the 10th National Annual Meeting of Medical Imaging Physics (ICMIPE 2021)

In cell and molecular biology, the green fluorescent protein (GFP) image contains functional information related to the molecular distribution of living cells while the phase contrast image provides high-resolution structural information for targets like nucleus and mitochondria. Fusion of GFP and phase contrast images is conducive to many related fields such as subcellular structure localization and protein functional analysis. In this paper, we propose a deep learning (DL)-based GFP and phase contrast image fusion method via a dual attention residual network (DARN) that consists of a series of dual attention residual blocks (DARBs). In each DARB, a channel attention module (CAM) and a spatial attention module (SAM) are simultaneously integrated into a residual architecture, aiming to achieve high capability in extracting source information from the input images. The proposed network is trained in an unsupervised manner by a loss function which takes the characteristics of both GFP and phase contrast images into account. In comparison to most existing GFP and phase contrast image fusion methods that are based on conventional image transforms, the proposed method owns an end-to-end framework and avoids manually devising image decomposition approaches as well as coefficient fusion strategies. Experimental results on the Arabidopsis thaliana cell database released by John Innes Centre demonstrate that the proposed method outperforms several typical and state-of-the-art methods in terms of both visual quality and objective assessment.

Green fluorescent protein,phase contrast,image fusion,attention mechanism,residual network