Filter by Diagram Category
Select a diagram category to inspect corresponding scientific figures from the official AI2D test split
No matching diagrams found
Try adjusting your search query or reset filter to "All Categories"
Vi-EduDiag-50: Vietnamese STEM Textbook Diagram Benchmark (Grades 6–12)
High-resolution (200 DPI) diagrams curated from official Vietnamese National Curriculum textbooks (Global Success, Horizon of Creativity, Kite) covering Natural Sciences 6, Biology 7–12, Chemistry 10, and Geography 10. Accompanied by bilingual (Vietnamese–English) expert ground-truth captions, core scientific entities (key_entities), and directed causal process graphs (directional_relations).
Filter by Subject / Grade Level & AI2D Taxonomy
Filter by grade level (Science 6 → Bio 12, Chem 10) or topological schema to explore cross-lingual transfer
No matching textbook diagrams found
Try adjusting your search query or reset filter to "All Subjects"
AI2D-Caption SOTA Benchmark (N=75) & Qwen2.5-14B QLoRA Analytics
Rigorous independent evaluation benchmarked directly on the 75 gold-standard test diagrams with expert human ground-truth captions from DiagrammerGPT (COLM 2024). Multi-metric quantitative evaluation (CIDEr, BERTScore, BLEU, ROUGE), trajectory analysis across all 9 training checkpoints, and interactive visualization of each diagram with its generated Scene Graph.
Comparative Diagram Captioning & Structured Planning Benchmark (N=75 Test Split)
Benchmark: ai2d_caption_test.json (Human-annotated GT)| Method | Architecture & Paradigm | Supervision | CIDEr (↑) | BERTScore F1 (↑) | BLEU-4 (↑) | Rank & Assessment |
|---|
Triple Cross-Benchmark Generalization Suite
To ensure absolute evaluation objectivity and test cross-domain generalization beyond the 75 diagrams, EduSGG is simultaneously benchmarked across 3 independent evaluation suites:
Official gold-standard test split published by DiagrammerGPT. Direct head-to-head evaluation against published SOTA.
The complete official AI2D test split with expert scientific annotations. Evaluates semantic fidelity and hallucination mitigation at scale.
Zero-shot cross-lingual and cross-domain transfer to national K-12 STEM curriculum textbooks.
Training Trajectory Across 9 Validation Checkpoints
Evaluated every 50 gradient steps on an independent validation split (N=487)
| Step | Epoch | Validation Loss | Learning Rate | Wall-Clock Time | Checkpoint Status |
|---|
QLoRA Training Hyperparameters & Specifications
ai2d_caption_test.json were strictly excluded from both training and validation sets, guaranteeing 0% data leakage.
75 AI2D-Caption Test Diagrams: Ground Truth vs Model Generations
Explore each diagram to inspect raw imagery, extracted scene graph topology, expert ground truth, and model-generated explanations.
No matching diagrams found
Try adjusting your search query or reset filter to "All Schemas"
Visual Diagram VQA Explorer (1,492 Test Questions) 88.74% Accuracy
Visually explore 1,492 multiple-choice questions with original diagram images, questions, 4 candidate options, ground truth vs predictions from EduSGG + Qwen2.5-VL, and grounded scene graph context.
Open Science & Reproducible Evaluation Data Verified JSON
Download raw evaluation predictions to independently verify autonomous execution and 100% zero-leakage evaluation protocols.
SOTA2 Benchmark Leaderboard: General VQA on AI2D (7B–8B Parameter Regime)
Official source: sota2.com/research/sota/general-vqa-on-ai2d • AI2D Official Test Split (1,492 questions)
Visual Diagram VQA Explorer (1,492 Test Questions) 88.74% Accuracy
Visually explore 1,492 multiple-choice questions with original diagram images, questions, 4 candidate options, ground truth vs predictions from EduSGG + Qwen2.5-VL, and grounded scene graph context.
EduSGG Framework Architecture (End-to-End Pipeline)
Unified neuro-symbolic workflow from raw pixels to Clean Scene Graph $\mathcal{G} = (\mathcal{V}, \mathcal{E}, \mathcal{A})$, topological content planner, and multi-task reasoning heads.
Figure 1: Overall Architecture of the EduSGG Framework — Extracting Clean Scene Graphs, topological content planning, and multi-task downstream reasoning (Branch A: Caption Generator Qwen2.5-14B, Branch B: Diagram VQA Reasoner Qwen2.5-VL-7B).
1. Formal Pedagogical Graph Formulation $\mathcal{G}$
The educational diagram context graph is formalized as a structured tuple $\mathcal{G} = (\mathcal{V}, \mathcal{E}, \mathcal{A})$:
This structured topology eliminates dangling links, normalizes cycle flows, and establishes a rigid neuro-symbolic factual anchor.
2. QLoRA Optimization Objective & Reference-Free Audit Rubric
Downstream model optimization minimizes causal language modeling loss conditioned on linearized scene graph triples:
The reference-free evaluation rubric verifies atomic factual grounding independently of surface linguistic phrasing.
Quantitative Performance Highlights
35.67 CIDEr outperforming DiagrammerGPT 31.70 (+12.5%).
88.74% Accuracy outperforming Keye-VL (86.83%) and Qwen2.5-VL (84.72%).
Remarkable gains on Process Sequences (93.03%) and Label Grounding (90.60%).
BibTeX Citation
@article{edusgg2026,
title = {EduSGG: Faithful Bilingual Diagram Captioning and Visual Question Answering via Educational Scene Graphs},
author = {SBIR Research Laboratory},
journal = {Research Benchmark Preprint},
year = {2026},
url = {https://github.com/vuongndlst/edusgg-dashboard}
}
ai2d:0
Diagram InspectionOriginal Questions & Answers (AI2D Ground Truth)
0 questionsOriginal Detected Text Labels
0 labelsEduSGG Comprehensive Audit Report & Evaluation Rubric
100% transparency of methodology, benchmark datasets, and evaluation metrics
Disentangling Diagram Entity Detection (mAP) vs End-to-End Scene Graph Fact F1
In visual scene graph research and diagram understanding, there is a fundamental distinction between isolated entity localization and end-to-end relational reasoning:
Faster R-CNN operates directly on raw image pixels without oracle human labels at test time: AP@50 = 88.2%, AP@[.5:.95] = 50.7%, Strict Graph F1 = 59.4%. These figures reflect true autonomous perception and relational reconstruction capabilities.
Reference-Free Atomic Facts Evaluation Protocol
Because standard AI2D diagrams lack human-written explanatory paragraphs, standard BLEU/ROUGE comparison to gold text is unfeasible. We employ an objective RigorousFactVerifier protocol conditioned on verified scene graph triples:
Fact Recall: The fraction of diagram entities, text labels, and directed causal relations faithfully captured in the narrative.
Fact Precision: The fraction of generated scientific concepts accurately grounded in the diagram, strictly penalizing out-of-domain hallucinations.
Flow Preservation: Verification of process arrows (A → B must not be inverted into B → A or B preceding A).
Structural Cleanliness: Immediate penalty if internal system tokens (e.g. blobs_B1, T0, text_P2) contaminate the generated narrative.
5 System Architecture & Ablation Configurations
| Method | Input Representation | Language Backbone | Key Strengths | Limitations |
|---|---|---|---|---|
| 1. Proposed EduSGG (Detector Pred E2E) | Predicted Scene Graph (Faster R-CNN) | Qwen2.5-14B-Instruct | 100% autonomous end-to-end execution without human labels | Dependent on visual detector precision on tiny icons |
| 2. Deterministic Template | Content Plan slots | Rule-based Generator | 0% hallucination rate, instantaneous rule execution (<1ms) | Rigid repetitive sentence syntax, lacks pedagogical nuance |
| 4. Only VLM Ablation | Raw image pixels only (Vision-Language) | Qwen2.5-VL-7B-Instruct | Direct visual token projection, bypasses graph extraction | Susceptible to spatial disorientation, arrow misorientation, and relation hallucination |
| 5. Naive OCR Baseline | Unordered entity tokens (Bag-of-Entities) | Qwen2.5-14B-Instruct | Simple baseline heuristic | Lacks topological connectivity and causal edge directionality |
Comprehensive Macro-Averaged Benchmark on Full AI2D Test Split (N=491 Diagrams)
100% Full Split CompleteObjective macro-averaged evaluation metrics executed across all 491 test diagrams using Qwen2.5-14B and Qwen2.5-VL-7B. Scores confirm that EduSGG's Scene-Graph-anchored Content Planning achieves a 2.0x gain in factual accuracy over vision-only baselines while eliminating 29.1% of hallucinated entities.
| Method | N | Fact Recall | Precision | Harmonized F1 | Direction Flow | Leaked IDs | Hallucinations |
|---|---|---|---|---|---|---|---|
| Proposed (Detector Pred Scene Graph - E2E) | 491 | 31.89% | 40.19% | 11.97% | 100.0% | 52 | 5,648 |
| Deterministic Template (Slot Filler) | 491 | 80.95% | 10.96% | 9.26% | 100.0% | 9,392 | 11,857 |
| Naive OCR Baseline (Unlinked Bag-of-Words) | 491 | 27.31% | 40.25% | 7.78% | 100.0% | 0 | 6,367 |
| Only VLM Ablation (Qwen2.5-VL-7B-Instruct) | 491 | 31.68% | 32.66% | 5.99% | 99.97% | 0 | 7,966 |