Back to feed
arXiv cs.CL
arXiv cs.CL
7/2/2026
ALEE: Any-Language Evaluation of Embeddings via English-Centric Minimal Pairs

ALEE: Any-Language Evaluation of Embeddings via English-Centric Minimal Pairs

Short summary

ALEE extends embedding evaluation across 275+ languages using Abstract Meaning Representations to generate English minimal pairs paired with translations. Testing reveals substantial performance variations correlated with language prevalence in training data and tokenization differences. The open-source framework enables targeted diagnostics for embedding models in any language.

  • •New evaluation framework for text embeddings across 275+ languages using AMR-generated minimal pairs
  • •Study exposes persistent gaps in cross-lingual semantic representation tied to training data prevalence
  • •Open-source tool available for testing embedding model performance in any language

Generated with AI, which can make mistakes.

#research-breakthrough#open-source#ai-tools
Read full article at arXiv cs.CL

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more

LLMs Silently Correct African American English: Auditing and Mitigating Dialect Bias via Activation Steering
arXiv cs.CL
Benchmarking Frontier LLMs on Arabic Cultural and Sociolinguistic Knowledge: A Cross-Evaluation Framework with Human SME Ground Truth
arXiv cs.CL
Equity with Efficiency: An Empirical Study of Tokenizers for Multilingual Large Language Models
arXiv cs.CL
Bridging Scientific Heritage: An Arabic--Russian Parallel Corpus and LLM Benchmark for Sustainable Knowledge Transfer
arXiv cs.CL