Skip to content

FetalMind: Epistemic-aware Vision-Language Foundation Model for Fetal Ultrasound Interpretation

KDD 2026 Oral

Xiao He*,1, Huangxuan Zhao*,†,1, Guojia Wan1, Jiancheng Pan2, Yanxing Liu3, Xin Zou4, Yong Luo5, Yongchao Xu5, Juhua Liu1, Wei Zhou6, Dacheng Tao7, Bo Du†,1
1 Wuhan University; 2 Tsinghua University; 3 University of the Chinese Academy of Sciences; 4 HKUST (Guangzhou); 5 Renmin Hospital of Wuhan University; 6 Wuhan Supercomputing Center; 7 Nanyang Technological University

* Equal contribution    † Corresponding author

ArXiv Code Model Demo Citation
FetalMind motivation and clinically aligned multi-view reasoning workflow
FetalMind aligns variable multi-view ultrasound with structured disease-view knowledge.

Research Overview

Two failure modes motivate a structured, evidence-aware solution.

Research gaps

Why current VLMs fail

Information collapse

About 10K visual tokens are often supervised by only a short diagnostic phrase.

Disease confusion

More than 300 conditions may share similar wording and overlapping anatomical views.

FetalMind response

How the model addresses them

Spatial Alignment

Assigns each image to its anatomical view before multi-image reasoning.

Fetal Token Injection

Encodes view and disease priors as structured tokens.

SED with SVPO

Links diagnoses to salient views and optimizes evidence-aware preferences.

Method Framework

Spatial alignment and token injection. FetalMind first assigns each image to a gestation-specific anatomical view, then maps key view and disease terms to explicit tokens. This organizes a variable number of ultrasound images and reduces confusion between clinically distinct abnormalities with similar descriptions.

Salient Epistemic Disentanglement. An expert-curated disease-view graph identifies the views that carry diagnostic evidence. FetalMind swaps only matched salient views to construct hard preference pairs, while SVPO trains the model to favor evidence-consistent diagnoses and reports.

Overview of FetalMind, FetalSigma-1M, and salient epistemic disentanglement
Figure 3. a, FetalMind aligns with clinical cognition by classifying images into pregnancy-specific views, encoding disease-view keywords as special tokens, and reinforcing their intrinsic associations via salient epistemic disentanglement (SED). b, FetalSigma-1M comprises over 1 million clinical fetal images from 12 centers, each with a complete report and a multi-expert diagnosis, covering all fetal systems in the form of image-report-diagnosis triplets. c, Overview of SED. (1) Salient views are identified from disease-view graphs and treated as perturbation variables, swapped across fetuses with disease replaced. (2) Intersection- and union-based substitution between diseased regions and views. (3) SVPO not only injects disease-view knowledge graphs into MLLMs but also enhances inter-disease discriminability.

FetalSigma-1M

Multi-center supervision across early, mid, and late pregnancy.

1.19M
images
20,566
cases
12
centers
300+
diseases
54
views

Main Results

FetalMind leads across report generation, diagnosis, and gestational stages.

FetalMind performance on report generation, diagnosis, and trimester-level evaluation
Multi-center comparison with medical and general-purpose vision-language models.
81.3
Diagnosis ACC
50.2
Body F1-20
80.7%
Salient attention
98%
Major anomalies

Ablation: removing SED causes the largest overall decline, from 45.2 to 40.5 average score.

Qualitative Comparison

FetalMind correctly identifies skeletal dysplasia while GPT-5 predicts normal
Ground truth
Skeletal dysplasia
FetalMind
Skeletal dysplasia
GPT-5
Normal

Obstetric Ultrasound Report

A guideline-informed template standardizes findings, measurements, impression, and diagnosis.

Generalized obstetric ultrasound report template

Citation

Please cite our paper if you find the dataset or model useful.

@misc{he2025fetalmind,
  title={Epistemic-aware Vision-Language Foundation Model for Fetal Ultrasound Interpretation},
  author={Xiao He and Huangxuan Zhao and Guojia Wan and Jiancheng Pan and Yanxing Liu and Xin Zou and Yong Luo and Yongchao Xu and Juhua Liu and Wei Zhou and Dacheng Tao and Bo Du},
  year={2025},
  eprint={2510.12953},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2510.12953}
}