CARPRT: Class-Aware Zero-Shot Prompt Reweighting for Black-Box Vision-Language Models

Short summary

CARPRT is a training-free scoring scheme that adjusts prompt weights per class label in vision-language models, challenging the assumption that prompts are class-independent. It quantifies class-specific relevance by averaging image-text relevance scores and normalizing them into class-specific weights. Evaluations on standard benchmarks show it outperforms class-independent reweighting methods, confirming that modeling prompt-class dependencies matters for zero-shot prediction.

• •CARPRT assigns class-specific prompt weights instead of shared weights across classes
• •Training-free method computes relevance by averaging image-text scores per class
• •Outperforms class-independent reweighting on standard image classification benchmarks

Generated with AI, which can make mistakes.