RAG Challenges & HyDE Solution - Traditional RAG faces challenges due to semantic dissimilarity between questions and answers, leading to irrelevant context retrieval [1] - HyDE addresses this by generating a hypothetical answer using an LLM, embedding it, and using the embedding to retrieve relevant context [2] - HyDE leverages contriever models trained with contrastive learning to filter out hallucinated details in the hypothetical answer [3] HyDE Performance & Trade-offs - Studies indicate HyDE improves retrieval performance compared to traditional embedding models [4] - HyDE implementation results in increased latency and higher LLM usage [4] HyDE Implementation - HyDE involves using an LLM to generate a hypothetical answer (H) for the query (Q) [2] - The hypothetical answer is embedded using a contriever model to obtain embedding (E) [2] - Embedding (E) is used to query the vector database and retrieve relevant context (C) [2] - The hypothetical answer (H), retrieved context (C), and query (Q) are passed to the LLM to produce a final answer [3]

Model Training Strategy - The initial approach of capturing user images and training a binary classifier for face unlock is flawed due to the need for on-device training and the difficulty of obtaining "Class 0" samples [1][2] - A Siamese Network trained via Contrastive learning offers a more suitable solution for face unlock systems [2] - Contrastive learning maps data points to a shared embedding space, where low distance indicates similarity and high distance indicates dissimilarity [3] - The system creates a dataset of face pairs, labeling pairs of the same person as 0 and different people as 1, then trains a supervised model [3] - A neural network generates embeddings for each image, and the distance between embeddings is minimized for similar faces and maximized for dissimilar faces using contrastive loss [4] - The contrastive loss function, L = (1-y)*D^2 + y*max(0, margin-D)^2, guides the model to produce low distances for similar inputs and high distances for dissimilar inputs [5] Face Unlock System Implementation - During setup, the user's facial data generates a reference embedding, and subsequent unlocks compare new embeddings against this reference embedding without further training [6] - New identities can be added by creating additional reference embeddings [6] - During unlock, the incoming user's embedding is compared against all reference embeddings [7]

You're in an ML Engineer interview at Apple.The interviewer asks:"You have to build an ML-based face unlock system for iPhones.How would you train the model?"You: "I will capture user's images & train a binary classifier on them"Interview over.Here's what you missed:There are multiple issues with capturing user's images and training a clasifier.> Firstly, you'd need to on-device training, which can be expensive.> All images provided by the user will be "Class 1" samples. To train a binary classifier, where ...