View Full Databricks Certified Generative AI Engineer Associate Exam Dumps and Practice Test Dumps

 

Question 181. Which technique is most useful for ensuring retrieved documents are relevant before they are provided to an LLM?

1. Increasing temperature
2. Reranking retrieved documents
3. Increasing response length
4. Removing metadata

Correct Answer: 2. Reranking retrieved documents

Explanation:

Reranking improves the relevance of retrieved documents by evaluating the initial search results more carefully and ordering them according to their relationship to the user’s query. In a RAG system, the first retrieval stage may return several documents that are broadly similar but not equally useful. A reranker can examine the query and candidate passages together and place the most relevant information first. This helps the generation model receive better context and can reduce irrelevant or distracting information. Reranking is particularly useful when the retrieval index contains a large and diverse collection of documents where simple similarity search alone may not provide sufficiently precise results.

Question 182. What is a primary benefit of using metadata filters during retrieval in a RAG application?

1. They increase the LLM’s context window
2. They automatically fine-tune the model
3. They restrict retrieval to documents matching specified attributes
4. They eliminate the need for embeddings

Correct Answer: 3. They restrict retrieval to documents matching specified attributes

Explanation:

Metadata filters allow a retrieval system to narrow the search space using attributes associated with documents or chunks. Examples include department, document type, language, date, product, region, or access classification. Instead of searching every indexed item, the system can first restrict candidates to records that satisfy the required conditions. This can improve relevance, reduce unnecessary retrieval, and support access-control requirements. Metadata filtering is especially valuable in enterprise RAG systems where users may need information from a particular business unit or a specific time period. It complements semantic or lexical search rather than replacing those retrieval techniques.

Question 183. A RAG application frequently retrieves outdated policies. What should the development team investigate first?

1. Whether the knowledge base is being updated and re-indexed correctly
2. Whether the temperature is too low
3. Whether the answer contains enough adjectives
4. Whether the model has a larger context window

Correct Answer: 1. Whether the knowledge base is being updated and re-indexed correctly

Explanation:

Outdated retrieval results commonly indicate a problem in the data ingestion or indexing lifecycle. If new or revised policies are not successfully ingested, transformed, and indexed, the retriever may continue returning older versions even when the source system contains current information. The team should therefore verify that document changes are detected, processed, embedded when required, and reflected in the active retrieval index. Metadata such as effective dates and document versions can also help prevent obsolete content from being selected. Changing the generation temperature or increasing the model’s context window does not address the underlying problem when the retriever itself is supplying stale information.

Question 184. Which evaluation approach helps determine whether an LLM answer is supported by retrieved context?

1. Measuring only endpoint throughput
2. Counting the number of prompt tokens
3. Checking the model’s parameter count
4. Evaluating groundedness against the retrieved evidence

Correct Answer: 4. Evaluating groundedness against the retrieved evidence

Explanation:

Groundedness evaluation examines whether the generated response is supported by the evidence supplied to the model. This is especially important for RAG applications because retrieval is intended to provide authoritative or relevant context that constrains generation. A grounded response should make claims that can be justified from the retrieved material rather than introducing unsupported facts. Evaluating groundedness separately from general answer quality helps teams identify hallucinations and weaknesses in the generation stage. A response can be fluent and relevant while still containing unsupported claims, so metrics that specifically assess grounding provide an important dimension of a comprehensive generative AI evaluation process.

Question 185. Why should an evaluation dataset contain examples that resemble real production queries?

1. To make the model’s vocabulary smaller
2. To make evaluation results more representative of expected application behavior
3. To eliminate the need for monitoring
4. To guarantee every answer is correct

Correct Answer: 2. To make evaluation results more representative of expected application behavior

Explanation:

A representative evaluation dataset allows a team to measure system behavior under conditions similar to those expected in production. If evaluation examples are unrealistic, overly simple, or unrelated to actual users, strong evaluation results may not reflect real application performance. Representative datasets can include common requests, difficult queries, ambiguous questions, different document types, and important edge cases. They should also reflect the intended user population and business domain. Although a representative dataset cannot guarantee perfect production behavior, it provides a much stronger basis for comparing prompts, retrieval strategies, models, and application changes before those changes are released.

Question 186. What is a key purpose of tracing in a generative AI application?

1. To increase the model’s training data automatically
2. To replace authentication
3. To provide visibility into requests, retrieval steps, and model calls
4. To permanently increase model accuracy

Correct Answer: 3. To provide visibility into requests, retrieval steps, and model calls

Explanation:

Tracing provides detailed visibility into the sequence of operations performed during an application request. In a RAG system, a trace can help show the incoming query, retrieval operation, selected documents, prompt construction, model invocation, latency, and resulting response. This information is useful when diagnosing unexpected behavior because developers can identify whether a problem originated in retrieval, prompt construction, model generation, or another component. Tracing also supports performance analysis and debugging across complex multi-step workflows. It does not itself improve model accuracy, but it provides the observability required to understand system behavior and make informed improvements.

Question 187. What should an application do when retrieved context does not contain enough information to answer a user’s question reliably?

1. Generate a confident answer from unrelated knowledge
2. Increase temperature
3. Ignore the retrieved context
4. Acknowledge the limitation or request clarification

Correct Answer: 4. Acknowledge the limitation or request clarification

Explanation:

When the retrieved evidence is insufficient, generating a confident answer can introduce unsupported information and increase hallucination risk. A well-designed application should instead communicate the limitation, ask the user for clarification, or indicate that the available sources do not provide enough information. This behavior can be reinforced through system instructions and application-level guardrails. Retrieval thresholds can also help determine when evidence is insufficient for generation. Designing the system to recognize uncertainty is particularly important for enterprise applications, where incorrect answers may lead users to make decisions based on information that was never supported by the organization’s approved knowledge sources.

Question 188. Which change can reduce unnecessary latency when the same expensive operation is repeatedly performed?

1. Introducing appropriate caching
2. Increasing temperature
3. Removing all monitoring
4. Increasing the number of retrieved documents

Correct Answer: 1. Introducing appropriate caching

Explanation:

Caching can reduce latency by storing results of operations that are expensive and likely to be reused. Depending on the architecture, an application may cache embeddings, frequently requested retrieval results, configuration data, or other deterministic intermediate results. When a valid cached result is available, the system can avoid repeating the underlying operation. However, caching must be designed carefully because stale information can be problematic in applications whose data changes frequently. Appropriate cache invalidation and expiration policies are therefore important. Caching does not replace optimization of retrieval or model calls, but it can significantly improve response time and resource efficiency for suitable workloads.

Question 189. What is the main difference between authentication and authorization in an AI application?

1. Authentication determines what information a user can access
2. Authorization verifies the user’s identity
3. Authentication verifies identity, while authorization determines permitted access
4. They are identical security processes

Correct Answer: 3. Authentication verifies identity, while authorization determines permitted access

Explanation:

Authentication and authorization address different security questions. Authentication establishes who a user or service is, commonly through credentials, tokens, or another identity mechanism. Authorization occurs after identity has been established and determines what that authenticated identity is allowed to access or perform. In an enterprise generative AI application, authentication may identify an employee, while authorization may determine whether that employee can retrieve confidential documents from a particular department. Keeping these concepts separate is important when designing secure RAG systems. A system that successfully identifies users but does not enforce appropriate authorization could still expose information to users who should not have access.

Question 190. Why is prompt versioning useful during generative AI development?

1. It allows teams to compare behavior across prompt changes
2. It automatically removes hallucinations
3. It increases the model’s parameter count
4. It eliminates evaluation requirements

Correct Answer: 1. It allows teams to compare behavior across prompt changes

Explanation:

Prompt versioning makes changes to instructions, templates, examples, and formatting requirements traceable over time. When a team modifies a prompt, version control allows developers to identify exactly which prompt produced a particular evaluation result or production behavior. This makes experimentation more reproducible and helps teams compare different prompt versions against a consistent evaluation dataset. If an updated prompt causes unexpected behavior, the previous version can also be identified and restored more easily. Prompt versioning is therefore an important part of disciplined generative AI development, particularly when prompts influence retrieval instructions, response formats, safety requirements, or domain-specific behavior.

Question 191. Which retrieval strategy combines semantic similarity with keyword-based matching?

1. Generative-only retrieval
2. Hybrid search
3. Random sampling
4. Model fine-tuning

Correct Answer: 2. Hybrid search

Explanation:

Hybrid search combines multiple retrieval approaches, commonly semantic vector search and lexical or keyword-based search. Semantic retrieval is useful for finding content that is conceptually related even when the wording differs, while lexical retrieval can perform well when exact terms, product names, identifiers, or specialized terminology are important. Combining these signals can improve retrieval coverage across different query types. The results may then be merged, scored, or reranked before the final context is provided to the language model. Hybrid search is therefore a useful strategy when a single retrieval method does not consistently capture the different ways users express information needs.

Question 192. What is the purpose of an output schema in a generative AI application?

1. To increase the number of model parameters
2. To define the expected structure and fields of the response
3. To replace document retrieval
4. To prevent all possible model errors

Correct Answer: 2. To define the expected structure and fields of the response

Explanation:

An output schema specifies the structure that a generated response is expected to follow. For example, an application might require fields such as a summary, confidence indicator, source identifiers, and recommended actions. Structured output is particularly useful when the model’s response will be consumed by software rather than read only by a human. After generation, the application can validate the response against the schema and handle invalid output appropriately. A schema does not guarantee that the model will always produce correct information, but it provides a predictable interface and makes downstream processing more reliable. This is especially important for production applications integrating LLMs with other systems.

Question 193. What is a major risk of using excessively large chunks in a RAG knowledge base?

1. The embedding model stops functioning
2. The documents become impossible to store
3. Retrieved results may contain too much unrelated information
4. Authentication becomes unavailable

Correct Answer: 3. Retrieved results may contain too much unrelated information

Explanation:

Excessively large chunks can reduce retrieval precision because a single retrieved unit may contain many topics that are not relevant to the user’s question. Although the chunk may match semantically, the additional material can consume valuable context-window capacity and distract the generation model. Large chunks can also make it harder to identify the exact evidence supporting an answer. Chunking should therefore balance sufficient context with focused retrieval. The appropriate size depends on document structure, query patterns, embedding behavior, and the downstream model’s context limitations. Testing different chunking strategies with representative evaluation queries is generally more reliable than choosing a single size without measuring retrieval quality.

Question 194. What does retrieval recall measure in a retrieval evaluation?

1. How many relevant items were successfully retrieved
2. How quickly the model generates tokens
3. How many parameters the model contains
4. How many users accessed the application

Correct Answer: 1. How many relevant items were successfully retrieved

Explanation:

Retrieval recall measures the ability of a retrieval system to find relevant information that should have been retrieved for a query. A low recall value can indicate that important documents or passages are being missed entirely. This differs from retrieval precision, which focuses on how many retrieved items are actually relevant. Both measures are useful because a system may retrieve highly relevant documents but miss other necessary evidence, or retrieve many documents while including substantial irrelevant material. Evaluating recall can help teams investigate embedding selection, chunking, query formulation, filters, indexing, and top-k settings when important information is consistently absent from retrieved results.

Question 195. Which practice improves reproducibility when comparing two RAG system versions?

1. Change several variables simultaneously without recording them
2. Use a consistent evaluation dataset and record system configurations
3. Evaluate only successful production requests
4. Remove all previous experiment results

Correct Answer: 2. Use a consistent evaluation dataset and record system configurations

Explanation:

Reproducible evaluation requires teams to know exactly what was tested and under which conditions. Using the same evaluation dataset when comparing system versions helps ensure that differences in results are associated with the system changes rather than different test questions. Teams should also record important configuration details such as model versions, prompts, retrieval settings, chunking strategy, embedding model, reranking configuration, and relevant application parameters. This creates a reliable basis for analyzing improvements or regressions. Without such records, it can be difficult to determine why a change affected performance or to recreate a previous experiment when investigating unexpected results.

Question 196. Why should sensitive information be protected during generative AI application logging?

1. Logs are never useful for debugging
2. Logging automatically improves retrieval
3. Sensitive data in logs can create additional exposure and compliance risks
4. Models cannot process logs

Correct Answer: 3. Sensitive data in logs can create additional exposure and compliance risks

Explanation:

Application logs can contain prompts, retrieved documents, model outputs, user identifiers, or other information generated during an AI workflow. If sensitive information is recorded unnecessarily, logs can become an additional location where confidential data is exposed or retained. Teams should therefore apply appropriate access controls, data minimization, retention policies, masking, and monitoring to observability systems. Logging remains important for troubleshooting and auditing, but it should be implemented with security and privacy considerations in mind. The goal is to collect enough information to understand system behavior without unnecessarily storing sensitive content or expanding the number of systems that can access it.

Question 197. What is the role of guardrails in a generative AI application?

1. They increase the model’s context window
2. They help constrain unsafe, inappropriate, or unwanted behavior
3. They replace all model evaluation
4. They guarantee factual accuracy

Correct Answer: 2. They help constrain unsafe, inappropriate, or unwanted behavior

Explanation:

Guardrails are controls designed to constrain how a generative AI system behaves. Depending on the application, they can address harmful content, sensitive information, prohibited requests, output formats, unsupported claims, or other defined requirements. Guardrails may be implemented through prompts, input and output checks, policy rules, classifiers, filters, or application logic. They are useful because language models can produce unexpected outputs even when given detailed instructions. However, guardrails should not be treated as a complete guarantee of safety or correctness. They work alongside evaluation, monitoring, access controls, data governance, and other engineering practices to create a more controlled application.

Question 198. A production RAG system suddenly shows a drop in answer quality. What should be examined before changing the model?

1. Retrieval quality and recent data or configuration changes
2. The user’s screen resolution
3. The font used in the application
4. The number of browser tabs open

Correct Answer: 1. Retrieval quality and recent data or configuration changes

Explanation:

A sudden decline in RAG answer quality may originate from changes outside the language model itself. The team should examine retrieval metrics, index freshness, source-document changes, embedding updates, metadata filters, chunking configuration, reranking behavior, and recent application deployments. Comparing current traces and evaluation results with a known-good baseline can help identify where the degradation began. If retrieval quality has declined, changing the generation model may not solve the problem because the model may simply be receiving poor or incomplete context. Investigating recent changes systematically helps isolate the affected component and reduces the risk of introducing unnecessary changes to a production system.

Question 199. Which practice is most useful for identifying whether a RAG improvement actually helped?

1. Relying only on a single anecdotal response
2. Increasing model temperature after every change
3. Comparing evaluation results against a documented baseline
4. Removing the retrieval component

Correct Answer: 3. Comparing evaluation results against a documented baseline

Explanation:

A documented baseline provides a reference point for measuring whether a change produces meaningful improvement. Before modifying a RAG system, teams can record relevant metrics such as retrieval precision, retrieval recall, groundedness, response relevance, latency, cost, and other application-specific measures. After the change, the same evaluation dataset and measurement process can be used for comparison. This makes it easier to distinguish genuine improvements from random variation or improvements in one metric that cause regressions elsewhere. Baseline evaluation also supports controlled experimentation and provides historical evidence that can help teams understand how the system has evolved across different versions.

Question 200. What is an appropriate high-level lifecycle for developing and operating a reliable RAG application?

1. Deploy immediately, then define requirements later
2. Retrieve randomly, generate answers, and never monitor them
3. Fine-tune first and ignore the knowledge source
4. Build the knowledge pipeline, evaluate retrieval and generation, deploy with monitoring, and continuously improve

Correct Answer: 4. Build the knowledge pipeline, evaluate retrieval and generation, deploy with monitoring, and continuously improve

Explanation:

A reliable RAG application requires an end-to-end lifecycle rather than focusing only on the language model. The process typically begins with preparing trustworthy source data, designing ingestion and chunking, generating embeddings, and configuring retrieval. Teams should then evaluate retrieval and generation quality using representative datasets before production deployment. Once deployed, monitoring and tracing can reveal changes in latency, retrieval behavior, groundedness, errors, and user experience. Feedback and evaluation results can guide controlled improvements to prompts, retrieval, data pipelines, models, and guardrails. Versioning these components and maintaining reproducible evaluations makes the overall system easier to operate, troubleshoot, and improve over time.