Diff-FCCF

Please leave us a star ⭐⭐⭐ if you find this work helpful.

🧠✨👀A Novel End-to-End Unsupervised Multi-Task Diffusion Network for Hierarchical Medical Multi-Modality Image Full-Color Channel Fusion

👀Update

• [2025/10] 😊😊😊Release data setting and preprocess for Diff-FCCF.
• [2025/10] 🔥🔥🔥Release all the code for Diff-FCCF.

✨✨✨Model Weight will be released after accepted.

🚀Environment

We test the code on PyTorch 2.6.0 + CUDA 12.9.

1. Create a new conda environment

conda create -n Diff-FCCF python=3.12.7
conda activate Diff-FCCF

2. Install dependencies

pip install -r requirements.txt

🚀Evaluation

You can directly test our model to generate fused images using the following code

python evaluation.py
--pet_folder ["/Dataset/Medical/Train/PET-MRI/pet"] \
--mri_folder ["/Dataset/Medical/Train/PET-MRI/mri"] \
--fusion_folder ["/Dataset/Medical/Train/PET-MRI/fusion_output"] \

🐰 Raw Datset and Preprocessed Dataset Download

Dataset	Dataset	Dataset
Harvard Dataset Download	GFP Download	ADNI Download

The data should organized in the following format:

train
├── HARVARD-CT-MRI                                    
│   ├──CT                                      
│   │   ├──0000.png                          
...                                          
│   ├──MRI                                     
│   │   ├──0000.png                            
...                                           
├── HARVARD-PET-MRI                          
│   ├──PET                        
│   │   ├──0000.png                    
...                                  
│   ├──MRI                             
│   │   ├──0000.png                        
...                                         
├── HARVARD-SPECT-MRI                              
│   ├──SPECT                              
│   │   ├──0000.png                      
...                                          
│   ├──MRI                            
│   │   ├──0000.png                       
...                      
├── GFP                                   
│   ├──g                              
│   │   ├──0000.png                      
...                                          
│   ├──f                            
│   │   ├──0000.png                       
...                                     
├── ADNI                                   
│   ├──PET                              
│   │   ├──0000.png                      
...                                          
│   ├──MRI                            
│   │   ├──0000.png                       
...                                     

2. 🚀Start training

You can use the following code to train the LFDT-Fusion model for different fusion tasks.

python train.py
--pet_dataset_path ["/Dataset/Medical/Train/PET-MRI"] \
--epoch ["\config.json" --epoch]\
--T ["\config.json" --T]\
--lr ["\config.json" --lr]\
You can find their corresponding configuration file paths in './config.json'.

Fusion examples

1. 🖼️Hardvard PET-MRI Fusion

Visual comparison of Diff-FCCF with 6 SOTA methods for MRI and PET image fusion. For a more intuitive comparison, the regions are enlarged as close-ups.

2. 🖼️Hardvard SPECT-MRI Fusion

Visual comparison of Diff-FCCF with 6 SOTA methods for MRI and SPECT image fusion. For a more intuitive comparison, the regions are enlarged as close-ups.

3. 🖼️Hardvard CT-MRI Fusion

Visual comparison of Diff-FCCF with 6 SOTA methods for MRI and CT image fusion. For a more intuitive comparison, the regions are enlarged as close-ups. CT images are preprocessed with rainbow mapping.

4. 🖼️GFP Fusion

Comparison of Diff-FCCF with 6 SOTA methods for PC and GFP image fusion. The intuitive regions are enlarged as close-ups.

5. 🖼️ADNI CT-MRI Fusion

Comparison of Diff-FCCF with 6 SOTA methods for MRI and PET image fusion. The intuitive regions are enlarged as close-ups. PET images are preprocessed with rainbow mapping.

6. 🖼️Hierarchical Condition Enhancement in Inference Phase

Three-stage denoising visualization during inference, presented for the fused image and its corresponding Y, Cr, and Cb channels.

🏷️ License

This repository is released under the MIT license. See LICENSE for additional details.