The Silent Vector Contamination Bug: Why Your Concurrent Embeddings Might Be Lying to You

Short summary

A subtle race condition in OpenVINO's AsyncInferQueue caused concurrent embedding requests to silently swap output vectors between inputs, corrupting a RAG vector store without throwing any exceptions. The bug stemmed from a shared userdata integer index that reset between batch calls, causing overlapping requests to overwrite each other's results. The author provides a cosine-similarity-based cross-contamination test that reliably catches this class of bug in CI.

• •Concurrent OpenVINO inference can silently swap embedding vectors between inputs without errors
• •Root cause: shared userdata index resets between batch calls, causing dictionary key collisions
• •Solution: cosine-similarity cross-contamination test comparing async results against sync baseline

Generated with AI, which can make mistakes.