Introducing ASCIITermDraw Bench | Testing the ability of VLMs to Generate and Edit ASCII [P]

Short summary

ASCIITermDraw-Bench is a new benchmark evaluating Vision Language Models on their ability to generate and edit ASCII-based diagrams across 80 tasks spanning basic layouts, network topologies, software architecture, and image-conditioned editing. Each task is scored both structurally and semantically via an LLM judge with 95% confidence intervals. Gemma-4-31B-IT currently leads at 73.8%, followed by Qwen3.7-Plus at 70.2%, with the full methodology and examples publicly available on Hugging Face.

• •New 80-task benchmark tests VLMs on ASCII diagram generation and editing across four categories
• •Dual scoring system uses structural verification plus LLM-judge semantic scoring with confidence intervals
• •Gemma-4-31B-IT tops the leaderboard at 73.8%; benchmark is open-source on Hugging Face

Generated with AI, which can make mistakes.