FPNuNet-workflow

main @ 25463e4

FPNuNet: A Frequency-Aware Prompt-Guided Network for Nuclear Segmentation and Classification in Immunohistochemistry Images

This is the official code repository for "FPNuNet: A Frequency-Aware Prompt-Guided Network for Nuclear Segmentation and Classification in Immunohistochemistry Images".

Introduction

Accurate nuclear segmentation and classification (NuSC) in immunohistochemistry (IHC)-stained images is essential for reliable biomarker quantification, yet existing methods frequently underperform due to domain-specific challenges such as stain heterogeneity and low nuclear contrast. FPNuNet addresses these limitations through a frequency-aware prompt-guided architecture that integrates RGB and Hematoxylin-Eosin-Diaminobenzidine channels via a color fusion stem, processes features through SAM-based structural and ViT-based semantic encoders modulated by lightweight prompt adapters, and employs a progressive frequency-aware residual global fusion neck to aggregate multi-scale features. The network utilizes three collaborative decoder branches to jointly predict binary masks, horizontal-vertical vectors, and nuclear types, enabling robust performance across diverse IHC staining conditions. Experimental evaluation on the CD47-IHCNuSC dataset demonstrates that FPNuNet consistently outperforms state-of-the-art baselines in both segmentation accuracy and classification robustness.

This work is built upon SAM2-PATH arxiv, which is a better segment anything model for semantic segmentation in digital pathology. We extend SAM2-PATH with frequency-aware mechanisms and prompt-guided strategies specifically designed for nuclear segmentation and classification in immunohistochemistry (IHC) images.

Citation

If you use our code or data in your research, please cite our paper:

@article{qin2026fpnunet,
  title={FPNuNet: A Frequency-Aware Prompt-Guided Network for Nuclear Segmentation and Classification in Immunohistochemistry Images},
  author={Lulu Qin, Zhigang Pei, Xudong He, Jiarui Zhou, Xianhong Xu and Zexuan Zhu},
  journal={GigaScience},
  year={2026}
}

⚙️ Usage

Data Preparation

1. Download the CD47-IHCNuSC dataset:

   The dataset can be found in the CD47_IHCNUSC/ directory. For detailed dataset information, please refer to CD47_IHCNUSC/readme_en.md.

   Note: The CD47-IHCNuSC dataset is licensed under CC BY-NC 4.0. Please review the license file in CD47_IHCNUSC/ before use.

2. Prepare data patches:

   The dataset should be preprocessed into HDF5 format. The expected data structure in HDF5 files includes:

   • Image patches
   • Instance segmentation masks
   • Nuclear type labels
   • Horizontal-vertical (HV) vectors

   Organize your processed data as specified in the configuration file:

   dataset_root/
   ├── gt_{name}_train_128x128_64x64_img_inst_type_hv.h5
   └── gt_{name}_valid_128x128_64x64_img_inst_type_hv.h5
   

3. Update configuration file:

   Edit the configuration file in configs/ directory (e.g., CD47_nuclei_HV_h5_128x128.py) to set:

   • dataset_root: Path to your dataset directory
   • checkpoint: Path to SAM2 pretrained weights (e.g., sam_vit_b_01ec64.pth)
   • extra_checkpoint: Path to UNI encoder pretrained weights (e.g., pytorch_model.bin)
   • out_dir: Output directory for saving model checkpoints and logs
   • batch_size: Training batch size (default: 12)
   • num_epochs: Number of training epochs (default: 80)
   • learning_rate: Initial learning rate (default: 5e-4)
   • devices: GPU device IDs for training (e.g., [0] for single GPU, [0, 1] for multi-GPU)

Training

Run the training script with your configuration file:

python main/main_FPNuNet.py --config configs/CD47_nuclei_HV_h5_128x128.py

Training Options

The training script supports various options that can be configured in the config file:

• batch_size: Training batch size (default: 12)
• num_epochs: Number of training epochs (default: 80)
• learning_rate: Initial learning rate (default: 5e-4)
• devices: GPU device IDs (e.g., [0, 1] for multi-GPU training)
• out_dir: Output directory for saving model checkpoints and logs

Training Example

# Single GPU training
python main/main_FPNuNet.py --config configs/CD47_nuclei_HV_h5_128x128.py

# Multi-GPU training (if supported)
# Set devices in config file: devices = [0, 1, 2, 3]

The training process will save model checkpoints and logs to the directory specified in the configuration file. You can monitor training progress using TensorBoard:

tensorboard --logdir /logs

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

The CD47-IHCNuSC dataset is licensed under CC BY-NC 4.0. Please refer to the license file in CD47_IHCNUSC/ for details.

🙏 Acknowledgments

This code is based on:

• SAM2-PATH git link - A better segment anything model for semantic segmentation in digital pathology
• SAM-PATH Miccai conference paper - The original SAM-PATH work
• SAM2 code - Meta's Segment Anything Model 2
• UNI encoder git link - Universal encoder for pathology images

All UNI and SAM2 pretrained weights can be downloaded from their respective repositories. Thanks to the authors for their excellent base code.