GPT-2 Small’s embedding geometry around “Trump”: discretized vs. continuous nearest neighbours [P]

Short summary

A visualization analysis of the token 'Trump' in GPT-2 Small's static embedding table using t-SNE projection of 32,070 alphabetic tokens. The study compares discretized vs. continuous nearest-neighbor representations, finding that discretized embeddings yield generic political terms while continuous embeddings surface specific associates like family members, staff, and other presidents. No prompting or generation is involved — all results come directly from learned token embeddings.

• •t-SNE visualization of GPT-2 Small's embedding table for the token 'Trump'
• •Discretized embeddings produce generic political neighbors; continuous embeddings surface specific associates
• •Analysis is purely from static embeddings, no context or generation involved

Generated with AI, which can make mistakes.