1. LLM Fundamentals
Q: What is the primary difference between a base model and an instruction-tuned model?
A: A base model is trained purely to predict the next word on a massive text corpus and often just completes a prompt. An instruction-tuned model has been fine-tuned (using techniques like RLHF) to follow instructions, answer questions, and engage in conversation.
Q: What is a “token” in the context of LLMs?
A: A token is the fundamental unit of data processed by an LLM. It can be a whole word, a syllable, or just a few characters. Typically, 1 token is roughly equivalent to ¾ of a word in English.
2. Prompt Engineering & Embeddings
Q: What is Few-Shot Prompting?
A: Few-shot prompting involves providing the model with a few examples of the desired input and output format within the prompt before asking it to perform the task. This significantly improves the model’s performance on specific tasks.
Q: How do embeddings enable semantic search?
A: Embeddings convert text into high-dimensional numerical vectors. Texts with similar meanings are placed close together in this vector space. Semantic search works by converting a search query into a vector and finding the closest text vectors using cosine similarity, allowing it to find concepts rather than just keyword matches.