ReXrank

What is ReXrank?

ReXrank is a public leaderboard for chest X-ray image interpretation, including both radiology report generation (RRG) and visual question answering (VQA) tasks.

ReXrank Challenge V1.0 is a competition in the generation of chest radiograph reports utilizing ReXGradient, the largest private test dataset consisting of 10,000 studies across 67 sites. The challenge attracted diverse participants from academic institutions, industry, and independent research teams, resulting in 24 state-of-the-art models previously benchmarked.

ReXrank Challenge V2.0 is a competition in VQA task utilizing VQA dataset constructed from ReXGradient, including 41,007 VQA pairs with 10,000 radiological studies. We benchmarked 8 state-of-the-art models.

ReXGradient-160K is the largest publicly available multi-site chest X-ray dataset, containing 273,004 unique chest X-ray images from 160,000 radiological studies, collected from 109,487 unique patients across 3 U.S. health systems (79 medical sites). In ReXrank, we use additional private test set ReXGradient, 10,000 studies for benchmarking.

ReXVQA is the largest and most comprehensive benchmark for VQA in chest radiology, comprising 653834 questions paired with 160,000 radiological studies. The dataset is constructed from ReXGradient-160K.

ReXrank Challenge V1.0 Leaderboard (RRG)

Rank	ReXGradient	MIMIC-CXR	IU-Xray	CheXpert Plus
1	Deepwise-RG Deepwise	Deepwise-RG Deepwise	CARE-X Microsoft Research	Deepwise-RG Deepwise
2	CARE-X Microsoft Research	CARE-X Microsoft Research	MoERad-IU IIT Madras	zzy-RGv1-8b Individual
3	MAIRA-2 Microsoft	zzy-RGv1-8b Individual	Cvt2distilgpt2-IU CSIRO	CARE-X Microsoft Research
4	MedVersa Harvard	UniRG-CXR Microsoft Research	BiomedGPT-IU Lehigh University	DD-LLaVA-X SNUH
5	CXRMate-RRG24 CSIRO	MedVersa Harvard	VLCI-IU SYSU	MAIRA-2 Microsoft
6	Cvt2distilgpt2-MIMIC CSIRO	CXRMate-RRG24 CSIRO	CX-Mind SJTU	CheXpertPlus-CheX-MIMIC Stanford
7	zzy-RGv1-8b Individual	Libra University of Glasgow	Libra University of Glasgow	UniRG-CXR Microsoft Research
8	MoERad-IU IIT Madras	CheXpertPlus-CheX-MIMIC Stanford	Deepwise-RG Deepwise	Libra University of Glasgow
9	CheXpertPlus-CheX-MIMIC Stanford	DD-LLaVA-X SNUH	DD-LLaVA-X SNUH	CXRMate-ED CSIRO
10	Cvt2distilgpt2-IU CSIRO	CheXpertPlus-MIMIC Stanford	CXRMate-RRG24 CSIRO	CXRMate-RRG24 CSIRO
11	VLCI-IU SYSU	CXRMate-ED CSIRO	CheXagent Stanford	Cvt2distilgpt2-MIMIC CSIRO
12	Libra University of Glasgow	CheXagent Stanford	Cvt2distilgpt2-MIMIC CSIRO	CheXpertPlus-CheX Stanford
13	CheXpertPlus-MIMIC Stanford	Cvt2distilgpt2-MIMIC CSIRO	CheXpertPlus-CheX-MIMIC Stanford	MedGemma Google
14	CheXagent Stanford	MedGemma Google	UniRG-CXR Microsoft Research	RaDialog TUM
15	CX-Mind SJTU	CheXOne-R1 Stanford	MedVersa Harvard	CheXagent Stanford
16	BiomedGPT-IU Lehigh University	MAIRA-2 Microsoft	CheXpertPlus-MIMIC Stanford	CheXpertPlus-MIMIC Stanford
17	VLCI-MIMIC SYSU	CX-Mind SJTU	zzy-RGv1-8b Individual	CheXOne-R1 Stanford
18	DD-LLaVA-X SNUH	MoERad-IU IIT Madras	MoERad-MIMIC IIT Madras	MedVersa Harvard
19	UniRG-CXR Microsoft Research	Cvt2distilgpt2-IU CSIRO	RadFM SJTU	MedGemma 1.5 4B Google
20	CXRMate-ED CSIRO	VLCI-IU SYSU	MedGemma 1.5 4B Google	VLCI-IU SYSU
21	MedGemma 1.5 4B Google	RaDialog TUM	CheXpertPlus-CheX Stanford	CX-Mind SJTU
22	MedGemma Google	CheXpertPlus-CheX Stanford	MAIRA-2 Microsoft	Cvt2distilgpt2-IU CSIRO
23	RaDialog TUM	VLCI-MIMIC SYSU	CXRMate-ED CSIRO	MoERad-IU IIT Madras
24	MoERad-MIMIC IIT Madras	MedGemma 1.5 4B Google	RaDialog TUM	VLCI-MIMIC SYSU
25	RGRG TUM	MoERad-MIMIC IIT Madras	MedGemma Google	BiomedGPT-IU Lehigh University
26	RadFM SJTU	BiomedGPT-IU Lehigh University	GPT4V OpenAI	RGRG TUM
27	GPT4V OpenAI	RGRG TUM	CheXOne-R1 Stanford	MoERad-MIMIC IIT Madras
28	CheXOne-R1 Stanford	RadFM SJTU	RGRG TUM	GPT4V OpenAI
29	CheXpertPlus-CheX Stanford	GPT4V OpenAI	VLCI-MIMIC SYSU	RadFM SJTU
30	LLM-CXR KAIST	LLM-CXR KAIST	LLM-CXR KAIST	LLM-CXR KAIST

ReXrank Challenge V2.0 Leaderboard (VQA)

Rank	ReXVQA
1	CARE-X Microsoft Research
2	CheXOne-R1 Stanford
3	MedGemma-4B-it Google
4	Eagle2-9B NVIDIA
5	Qwen2.5VL-7B-Instruct Alibaba
6	Qwen2VL-7B-Instruct Alibaba
7	Janus-Pro-7B DeepSeek
8	Phi35-Vision-Instruct Microsoft
9	LLaVA-1.5-7B Meta

ReXrank Challenge V1.0 - RRG Performance on ReXGradient

ReXGradient is a large-scale private test dataset contains 10,000 studies collected from different medical centers in the US.

Rank	Model	CRIMSON	1/RadCliQ-v1	BLEU	BertScore	SembScore	RadGraph	RaTEScore	GREEN
1 2024	CheXagent Stanford	0.339	0.67	0.095	0.301	0.365	0.08	0.428	0.241
2 2024	CheXpertPlus-MIMIC Stanford	0.342	0.777	0.156	0.341	0.44	0.131	0.501	0.52
3 2024	CheXpertPlus-CheX Stanford	0.121	0.783	0.144	0.36	0.428	0.124	0.476	0.411
4 2024	CheXpertPlus-CheX-MIMIC Stanford	0.358	0.827	0.169	0.371	0.44	0.153	0.517	0.489
5 2023	Cvt2distilgpt2-MIMIC CSIRO	0.368	0.862	0.186	0.374	0.458	0.175	0.524	0.514
6 2023	Cvt2distilgpt2-IU CSIRO	0.357	0.834	0.176	0.389	0.403	0.166	0.52	0.47
7 2024	MedVersa Harvard	0.382	0.986	0.205	0.419	0.497	0.2	0.527	0.532
8 2023	RadFM SJTU	0.154	0.759	0.156	0.347	0.394	0.134	0.504	0.406
9 2023	RaDialog TUM	0.269	0.844	0.184	0.38	0.446	0.156	0.522	0.435
10 2023	RGRG TUM	0.246	0.855	0.178	0.37	0.464	0.167	0.54	0.487
11 2023	VLCI-MIMIC SYSU	0.317	0.72	0.158	0.311	0.4	0.122	0.488	0.477
12 2023	VLCI-IU SYSU	0.35	0.895	0.214	0.366	0.466	0.213	0.573	0.536
13 2024	LLM-CXR KAIST	-0.424	0.507	0.044	0.184	0.137	0.031	0.317	0.044
14 2024	GPT4V OpenAI	0.141	0.617	0.074	0.198	0.337	0.135	0.47	0.497
15 2024	BiomedGPT-IU Lehigh University	0.334	0.796	0.159	0.326	0.454	0.164	0.472	0.388
16 2024	MAIRA-2 Microsoft	0.391	0.93	0.2	0.414	0.463	0.185	0.559	0.531
17 2024	CXRMate-ED CSIRO	0.293	0.895	0.205	0.416	0.416	0.187	0.564	0.518
18 2024	CXRMate-RRG24 CSIRO	0.379	0.817	0.151	0.349	0.463	0.152	0.518	0.408
19 2024	Libra University of Glasgow	0.344	0.885	0.165	0.387	0.474	0.168	0.544	0.555
20 2025	MoERad-IU IIT Madras	0.363	1.025	0.227	0.439	0.446	0.247	0.575	0.494
21 2025	MoERad-MIMIC IIT Madras	0.249	0.757	0.145	0.352	0.406	0.116	0.508	0.431
22 2025	DD-LLaVA-X SNUH	0.316	0.888	0.166	0.388	0.469	0.174	0.542	0.504
23 2025	MedGemma Google	0.282	1.01	0.2	0.428	0.479	0.223	0.617	0.566
24 2025	UniRG-CXR Microsoft Research	0.315	1.509	0.284	0.506	0.581	0.301	0.618	0.469
25 2026	CARE-X Microsoft Research	0.438	1.635	0.307	0.529	0.568	0.322	0.653	0.506
26 2025	CheXOne-R1 Stanford	0.127	1.056	0.222	0.45	0.501	0.21	0.535	0.428
28 2025	CX-Mind SJTU	0.339	0.941	0.185	0.435	0.464	0.166	0.574	0.499
29 2026	MedGemma 1.5 4B Google	0.288	0.921	0.196	0.36	0.505	0.215	0.61	0.542
30 2026	Deepwise-RG Deepwise	0.442	2.021	0.351	0.551	0.606	0.356	0.661	0.552
1 2024	CheXpertPlus-MIMIC Stanford		0.791	0.177	0.364	0.431	0.139	0.481	0.523
2 2024	CheXpertPlus-CheX Stanford		0.748	0.165	0.333	0.395	0.148	0.502	0.468
3 2024	CheXpertPlus-CheX-MIMIC Stanford		0.838	0.196	0.389	0.429	0.166	0.5	0.508
4 2024	MedVersa Harvard		0.984	0.172	0.438	0.48	0.188	0.527	0.524
5 2023	RadFM SJTU		0.737	0.132	0.338	0.375	0.131	0.466	0.405
6 2024	GPT4V OpenAI		0.605	0.072	0.214	0.364	0.175	0.456	0.356
7 2025	UniRG-CXR Microsoft Research		1.59	0.3	0.532	0.573	0.3	0.612	0.494
8 2025	CheXOne-R1 Stanford		0.926	0.226	0.388	0.484	0.199	0.515	0.475
9 2026	MedGemma 1.5 4B Google		3.048	0.383	0.59	0.632	0.417	0.675	0.701

ReXrank Challenge V1.0 - RRG Performance on MIMIC-CXR

MIMIC-CXR contains 377,110 images corresponding to 227,835 radiographic studies performed at the Beth Israel Deaconess Medical Center in Boston, MA. We follow the official split of MIMIC-CXR in the following experiments.

Rank	Model	CRIMSON	1/RadCliQ-v1	BLEU	BertScore	SembScore	RadGraph	RaTEScore	GREEN
1 2024	CheXagent Stanford	0.096	0.74	0.154	0.346	0.346	0.147	0.474	0.257
2 2024	CheXpertPlus-MIMIC Stanford	0.107	0.795	0.195	0.368	0.375	0.17	0.485	0.311
3 2024	CheXpertPlus-CheX Stanford	0.026	0.703	0.11	0.32	0.325	0.142	0.469	0.225
4 2024	CheXpertPlus-CheX-MIMIC Stanford	0.132	0.812	0.188	0.374	0.379	0.181	0.49	0.305
5 2023	Cvt2distilgpt2-MIMIC CSIRO	0.09	0.724	0.167	0.336	0.33	0.149	0.432	0.268
6 2023	Cvt2distilgpt2-IU CSIRO	0.042	0.614	0.081	0.304	0.191	0.103	0.448	0.164
7 2024	MedVersa Harvard	0.17	1.086	0.271	0.444	0.465	0.269	0.55	0.374
8 2023	RadFM SJTU	-0.074	0.646	0.13	0.309	0.259	0.108	0.45	0.185
9 2023	RaDialog TUM	0.035	0.787	0.17	0.354	0.385	0.17	0.485	0.273
10 2023	RGRG TUM	-0.057	0.75	0.185	0.345	0.341	0.167	0.491	0.273
11 2023	VLCI-MIMIC SYSU	0.016	0.688	0.187	0.314	0.305	0.14	0.45	0.256
12 2023	VLCI-IU SYSU	0.038	0.606	0.12	0.274	0.212	0.109	0.449	0.21
13 2024	LLM-CXR KAIST	-0.4	0.514	0.052	0.178	0.154	0.046	0.341	0.043
14 2024	GPT4V OpenAI	-0.158	0.559	0.099	0.209	0.214	0.084	0.423	0.161
15 2024	BiomedGPT-IU Lehigh University	-0.018	0.561	0.043	0.213	0.239	0.064	0.36	0.123
16 2024	MAIRA-2 Microsoft	0.075	0.694	0.088	0.308	0.339	0.131	0.517	0.224
17 2024	CXRMate-ED CSIRO	0.102	0.887	0.209	0.394	0.396	0.223	0.531	0.327
18 2024	CXRMate-RRG24 CSIRO	0.153	0.887	0.198	0.38	0.424	0.22	0.521	0.338
19 2024	Libra University of Glasgow	0.152	0.897	0.232	0.402	0.403	0.218	0.523	0.356
20 2025	MoERad-IU IIT Madras	0.043	0.643	0.064	0.321	0.213	0.122	0.455	0.174
21 2025	MoERad-MIMIC IIT Madras	-0.016	0.726	0.163	0.341	0.334	0.143	0.465	0.24
22 2025	DD-LLaVA-X SNUH	0.114	0.801	0.154	0.349	0.402	0.182	0.505	0.301
23 2025	MedGemma Google	0.082	0.744	0.165	0.346	0.339	0.159	0.549	0.293
24 2025	UniRG-CXR Microsoft Research	0.181	1.18	0.259	0.47	0.495	0.268	0.602	0.36
25 2025	CheXOne-R1 Stanford	0.08	1.023	0.215	0.441	0.456	0.234	0.519	0.314
26 2026	CARE-X Microsoft Research	0.225	1.213	0.267	0.489	0.469	0.283	0.607	0.388
28 2025	CX-Mind SJTU	0.062	0.793	0.153	0.394	0.323	0.174	0.543	0.267
29 2026	MedGemma 1.5 4B Google	0.006	0.653	0.117	0.228	0.355	0.152	0.565	0.284
30 2026	Deepwise-RG Deepwise	0.26	1.394	0.264	0.496	0.524	0.316	0.619	0.386
1 2024	CheXpertPlus-MIMIC Stanford		0.802	0.165	0.353	0.382	0.193	0.511	0.377
2 2024	CheXpertPlus-CheX Stanford		0.715	0.127	0.3	0.342	0.173	0.51	0.302
3 2024	CheXpertPlus-CheX-MIMIC Stanford		0.825	0.166	0.362	0.391	0.203	0.52	0.367
4 2024	MedVersa Harvard		0.919	0.193	0.43	0.315	0.273	0.554	0.421
5 2023	RadFM SJTU		0.625	0.081	0.281	0.245	0.111	0.448	0.214
6 2024	GPT4V OpenAI		0.549	0.065	0.204	0.19	0.085	0.429	0.127
7 2025	UniRG-CXR Microsoft Research		1.066	0.205	0.419	0.491	0.266	0.615	0.365
8 2025	CheXOne-R1 Stanford		0.911	0.156	0.372	0.45	0.233	0.528	0.349
9 2026	MedGemma 1.5 4B Google		0.734	0.104	0.287	0.373	0.195	0.578	0.303

ReXrank Challenge V1.0 - RRG Performance on IU Xray

IU Xray contains 7,470 pairs of radiology reports and chest X-rays from Indiana University. We follow the split given by R2Gen.

Rank	Model	CRIMSON	1/RadCliQ-v1	BLEU	BertScore	SembScore	RadGraph	RaTEScore	GREEN
1 2024	CheXagent Stanford	0.633	0.852	0.119	0.367	0.502	0.138	0.503	0.389
2 2024	CheXpertPlus-MIMIC Stanford	0.619	1.022	0.179	0.401	0.601	0.17	0.585	0.661
3 2024	CheXpertPlus-CheX Stanford	0.559	0.917	0.157	0.411	0.495	0.153	0.534	0.541
4 2024	CheXpertPlus-CheX-MIMIC Stanford	0.632	1.174	0.198	0.451	0.593	0.211	0.618	0.648
5 2023	Cvt2distilgpt2-MIMIC CSIRO	0.632	1.165	0.201	0.437	0.614	0.208	0.606	0.682
6 2023	Cvt2distilgpt2-IU CSIRO	0.654	1.268	0.244	0.476	0.548	0.265	0.62	0.686
7 2024	MedVersa Harvard	0.623	1.294	0.215	0.473	0.609	0.234	0.65	0.631
8 2023	RadFM SJTU	0.562	1.218	0.211	0.465	0.58	0.228	0.627	0.615
9 2023	RaDialog TUM	0.532	1.094	0.205	0.445	0.553	0.203	0.586	0.586
10 2023	RGRG TUM	0.481	1.157	0.201	0.429	0.606	0.221	0.62	0.665
11 2023	VLCI-MIMIC SYSU	-0.036	0.933	0.142	0.379	0.482	0.22	0.578	0.474
12 2023	VLCI-IU SYSU	0.654	1.383	0.268	0.456	0.619	0.288	0.679	0.698
13 2024	LLM-CXR KAIST	-0.554	0.482	0.033	0.174	0.062	0.023	0.28	0.025
14 2024	GPT4V OpenAI	0.52	0.664	0.072	0.217	0.408	0.144	0.517	0.651
15 2024	BiomedGPT-IU Lehigh University	0.654	0.994	0.206	0.382	0.542	0.22	0.543	0.523
16 2024	MAIRA-2 Microsoft	0.552	1.256	0.215	0.461	0.606	0.232	0.627	0.194
17 2024	CXRMate-ED CSIRO	0.534	1.274	0.229	0.485	0.557	0.249	0.655	0.685
18 2024	CXRMate-RRG24 CSIRO	0.633	1.557	0.25	0.48	0.642	0.302	0.666	0.68
19 2024	Libra University of Glasgow	0.642	1.183	0.183	0.444	0.614	0.21	0.624	0.698
20 2025	MoERad-IU IIT Madras	0.656	1.96	0.277	0.531	0.641	0.341	0.684	0.665
21 2025	MoERad-MIMIC IIT Madras	0.573	1.022	0.171	0.421	0.559	0.178	0.603	0.584
22 2025	DD-LLaVA-X SNUH	0.635	1.207	0.189	0.445	0.6	0.233	0.636	0.671
23 2025	MedGemma Google	0.524	1.345	0.217	0.476	0.6	0.26	0.678	0.724
24 2025	UniRG-CXR Microsoft Research	0.632	3.907	0.368	0.606	0.705	0.396	0.729	0.664
25 2025	CheXOne-R1 Stanford	0.49	1.515	0.254	0.507	0.611	0.277	0.617	0.585
26 2025	CX-Mind SJTU	0.648	1.385	0.21	0.504	0.613	0.23	0.662	0.644
28 2026	CARE-X Microsoft Research	0.661	1.976	0.279	0.552	0.662	0.301	0.702	0.63
29 2026	MedGemma 1.5 4B Google	0.561	1.339	0.214	0.446	0.643	0.263	0.675	0.706
30 2026	Deepwise-RG Deepwise	0.641	2.198	0.316	0.552	0.668	0.337	0.684	0.64
1 2024	CheXpertPlus-MIMIC Stanford		1.111	0.227	0.449	0.594	0.187	0.57	0.681
2 2024	CheXpertPlus-CheX Stanford		0.995	0.198	0.394	0.55	0.211	0.604	0.706
3 2024	CheXpertPlus-CheX-MIMIC Stanford		1.249	0.244	0.476	0.598	0.232	0.606	0.694
4 2024	MedVersa Harvard		1.452	0.195	0.518	0.601	0.244	0.628	0.658
5 2023	RadFM SJTU		1.22	0.196	0.479	0.556	0.234	0.596	0.644
6 2024	GPT4V OpenAI		0.683	0.079	0.235	0.403	0.16	0.519	0.399
7 2025	UniRG-CXR Microsoft Research		5.138	0.403	0.642	0.697	0.403	0.724	0.689
8 2025	CheXOne-R1 Stanford		1.042	0.254	0.372	0.604	0.221	0.595	0.647
9 2026	MedGemma 1.5 4B Google		1.555	0.233	0.489	0.645	0.288	0.674	0.698

ReXrank Challenge V1.0 - RRG Performance on CheXpert Plus

CheXpert Plus contains 223,228 unique pairs of radiology reports and chest X-rays from 187,711 studies and 64,725 patients. We follow the official split of CheXpert Plus in the following experiments and use the valid set for evaluation.

Rank	Model	CRIMSON	1/RadCliQ-v1	BLEU	BertScore	SembScore	RadGraph	RaTEScore	GREEN
1 2024	CheXagent Stanford	0.102	0.638	0.125	0.276	0.272	0.125	0.434	0.183
2 2024	CheXpertPlus-MIMIC Stanford	0.1	0.679	0.147	0.302	0.298	0.145	0.43	0.238
3 2024	CheXpertPlus-CheX Stanford	0.117	0.807	0.159	0.351	0.379	0.204	0.487	0.237
4 2024	CheXpertPlus-CheX-MIMIC Stanford	0.212	0.831	0.159	0.346	0.407	0.218	0.497	0.274
5 2023	Cvt2distilgpt2-MIMIC CSIRO	0.124	0.636	0.128	0.277	0.271	0.122	0.42	0.215
6 2023	Cvt2distilgpt2-IU CSIRO	0.032	0.587	0.087	0.275	0.158	0.108	0.382	0.147
7 2024	MedVersa Harvard	0.086	0.721	0.139	0.319	0.348	0.153	0.47	0.243
8 2023	RadFM SJTU	-0.179	0.568	0.084	0.223	0.216	0.086	0.396	0.096
9 2023	RaDialog TUM	0.111	0.704	0.133	0.305	0.349	0.143	0.445	0.211
10 2023	RGRG TUM	-0.011	0.674	0.154	0.308	0.273	0.151	0.453	0.216
11 2023	VLCI-MIMIC SYSU	0.018	0.598	0.126	0.237	0.255	0.106	0.384	0.165
12 2023	VLCI-IU SYSU	0.04	0.563	0.11	0.229	0.174	0.098	0.418	0.194
13 2024	LLM-CXR KAIST	-0.433	0.517	0.04	0.158	0.208	0.039	0.321	0.022
14 2024	GPT4V OpenAI	-0.052	0.569	0.098	0.217	0.229	0.086	0.415	0.152
15 2024	BiomedGPT-IU Lehigh University	-0.001	0.566	0.055	0.215	0.251	0.062	0.351	0.118
16 2024	MAIRA-2 Microsoft	0.238	0.782	0.165	0.346	0.359	0.195	0.485	0.273
17 2024	CXRMate-ED CSIRO	0.157	0.732	0.158	0.332	0.32	0.175	0.498	0.265
18 2024	CXRMate-RRG24 CSIRO	0.151	0.812	0.158	0.323	0.415	0.218	0.521	0.276
19 2024	Libra University of Glasgow	0.17	0.718	0.157	0.319	0.323	0.169	0.466	0.253
20 2025	MoERad-IU IIT Madras	0.032	0.595	0.075	0.284	0.175	0.102	0.39	0.127
21 2025	MoERad-MIMIC IIT Madras	-0.033	0.641	0.122	0.268	0.3	0.12	0.434	0.166
22 2025	DD-LLaVA-X SNUH	0.246	0.752	0.085	0.317	0.385	0.172	0.476	0.206
23 2025	MedGemma Google	0.111	0.706	0.147	0.327	0.325	0.137	0.511	0.246
24 2025	UniRG-CXR Microsoft Research	0.192	1.091	0.214	0.416	0.527	0.26	0.581	0.306
25 2025	CheXOne-R1 Stanford	0.094	0.992	0.175	0.396	0.489	0.242	0.522	0.25
26 2026	CARE-X Microsoft Research	0.248	0.951	0.208	0.407	0.42	0.249	0.579	0.306
28 2025	CX-Mind SJTU	0.033	0.664	0.114	0.325	0.244	0.133	0.498	0.179
29 2026	MedGemma 1.5 4B Google	0.063	0.607	0.104	0.18	0.36	0.118	0.524	0.273
30 2026	Deepwise-RG Deepwise	0.335	1.102	0.217	0.42	0.495	0.287	0.575	0.328
1 2024	CheXpertPlus_MIMIC Stanford		0.482	0.103	0.002	0.318	0.049	0.429	0.293
2 2024	CheXpertPlus_CheX Stanford		0.512	0.142	0.02	0.38	0.07	0.492	0.363
3 2024	CheXpertPlus_CheX_MIMIC Stanford		0.511	0.14	0.011	0.388	0.071	0.503	0.382
4 2024	MedVersa Harvard		0.493	0.09	0.013	0.337	0.05	0.452	0.334
5 2023	RadFM SJTU		0.443	0.067	-0.038	0.229	0.027	0.39	0.137
6 2024	GPT4V OpenAI		0.431	0.055	-0.065	0.208	0.028	0.393	0.182
7 2025	UniRG-CXR Microsoft Research		0.702	0.17	0.177	0.517	0.176	0.573	0.348
8 2025	CheXOne-R1 Stanford		0.68	0.137	0.183	0.463	0.174	0.503	0.315
9 2026	MedGemma 1.5 4B Google		0.466	0.083	-0.027	0.294	0.045	0.503	0.274

ReXrank Challenge V2.0 - VQA Performance on ReXVQA

Performance comparison of various vision-language models on medical VQA tasks.

Rank	Model	Overall Accuracy	Differential Diagnosis	Geometric Information	Location Assessment	Negation Assessment	Presence Assessment
1	CARE-X Microsoft Research	0.9083	0.8975	0.7598	0.8193	0.9493	0.8887
2	CheXOne-R1 Stanford	0.8803	0.8447	0.7374	0.8038	0.9110	0.8799
3	MedGemma-4B-it Google	0.8344	0.8190	0.6872	0.7504	0.8979	0.7935
4	Eagle2-9B NVIDIA	0.6939	0.7212	0.5084	0.5761	0.8568	0.5403
5	Qwen2.5VL-7B-Instruct Alibaba	0.6867	0.7451	0.5642	0.5609	0.8802	0.4892
6	Qwen2VL-7B-Instruct Alibaba	0.6835	0.7661	0.6760	0.6160	0.7048	0.6363
7	Janus-Pro-7B DeepSeek	0.6270	0.6923	0.6592	0.6235	0.6764	0.5483
8	Phi35-Vision-Instruct Microsoft	0.6180	0.6990	0.4469	0.5252	0.7527	0.4643
9	LLaVA-1.5-7B Meta	0.2632	0.2628	0.3464	0.2571	0.2576	0.2690