Embeddings

PART 3 — Embeddings

An embedding is a lossy compression of semantic meaning into a float32 array.

• Vector: An array like [0.12, -0.9, 0.4, ...] usually containing 768 to 1536 elements.
• Distance Metrics: To find similar text, we compute the distance between arrays using Cosine Similarity or Dot Product.

SDE Takeaway: Think of embeddings as a Locality-Sensitive Hash (LSH). In traditional hashing (SHA-256), a 1-bit change completely changes the hash. In embedding space, semantically similar strings hash to coordinates that are geometrically close to each other. This enables fuzzy, concept-based searching.

---