Reading about Foundry, retrieval and agents is not the same as knowing what fails when those components interact. A candidate may understand the definition of RAG while using a search index that returns outdated documents, or know how to call a model while overlooking that the request contains an unauthorized record. Small, controlled labs expose these mistakes and turn them into evidence that can be reviewed independently of a remembered answer key.
The following original exercises support the practical competencies behind Microsoft AI-103 Exam. They are not live examination tasks. Use only Azure resources and data you are explicitly authorized to use, synthetic information, limited permissions and a clear spending budget. Each exercise should demonstrate an intended behavior, a meaningful failure and a safe cleanup. Avoid creating public, vulnerable or billable resources that have no clear learning purpose.
Establish a repeatable training baseline
Choose one small fictitious business: a repair shop whose employees search approved manuals and prepare service recommendations. Create synthetic documents, tickets and device IDs. Define what the assistant may read and what actions require human approval. Record the Foundry project, deployed model configuration, relevant API or SDK version, source data version and any test application code that affects the result.
For every lab, write five fields before execution: required property, baseline, one planned change, expected observation and teardown. Record errors as well as successes. A green tool call without an explanation of what it proves is not a strong artifact. A failed run is valuable if it identifies a mistaken assumption and can be repeated after a correction.
Use scoped identities and protected configuration instead of copying credentials into notebooks, prompts or screenshots. The platform may support managed identities and keyless authentication for relevant workloads; the appropriate approach depends on the service and deployment. Remove disposable resources and revoke temporary permissions when the test is complete.
Lab 1: make a model request observable
Start with a Python application that sends a harmless synthetic prompt through the supported Foundry interface. Record the model deployment and parameters, request status, latency and returned structure without collecting unrelated secrets. Validate the response fields expected by the application rather than assuming that any nonempty string is an acceptable result.
Use three input cases: one ordinary question with a known answer, one ambiguous question and one clearly unsupported request. A fluent response to the unsupported case is not necessarily correct. Note which evidence the model had and what you expected it to do when no source was provided.
Next simulate an error such as an unavailable deployment, rate limit or invalid response format in the disposable client. The application should classify the error and report incomplete work rather than invent content. Where retries are appropriate, bound them so the test does not accidentally generate uncontrolled calls or cost.
Keep the output small: a sample input, sanitized response, validation result and short explanation of what the run cannot prove about factual accuracy. That baseline becomes important when retrieval and tools are introduced.
Lab 2: expose retrieval and source-freshness mistakes
Prepare a few synthetic repair procedures: one current, one superseded and one restricted to another customer. Include document identifiers, effective dates and access metadata. Index the appropriate records through an authorized retrieval solution and write a question whose answer depends on choosing the current procedure.
Test retrieval relevance and applicability separately. Does the correct passage appear in the returned set? If several passages appear, which one is authoritative? Can the querying identity access only the permitted customer? An answer quoting the superseded manual is not fully grounded merely because the passage exists in the index.
Change one factor at a time: retrieval query, filter, document version or chunking. Compare results and preserve the source identifiers. If the right document is absent, investigate ingestion or retrieval before increasing prompt length. If the document is present but the generated response invents another step, the failure lies elsewhere.
Write at least one negative test for an unauthorized record. The application should not return a private passage even when its text is semantically similar. The enforcement belongs to access-controlled retrieval or data services, not solely to a model instruction to avoid disclosing it.
Lab 3: test a narrow tool under real authorization
Create a read-only tool that retrieves the service status of a synthetic device. Its input includes a validated equipment identifier; its output distinguishes found, not found, permission denied and temporarily unavailable. The host application validates the caller’s scope before accessing the record.
Run a legitimate lookup and an attempt to request an identifier outside the user’s permitted data. The unauthorized request must be rejected by the application or service even if the model generates a perfectly valid tool call. That shows the difference between a tool schema, which helps describe correct input shape, and authorization, which determines who may use the capability.
Next connect a draft-review tool that creates a harmless test record. Introduce a simulated timeout after the record is created and repeat the request. If a duplicate appears, document the missing idempotency control and correct it. This exercise is valuable because a model apology cannot reverse a duplicated external action.
Finally, verify that a tool result containing instruction-like text is treated as untrusted content. A retrieved “ignore the rules” string should not expand tool permissions or authorize a write. The host determines which operations may occur and which require a human reviewer.
Lab 4: extract structured facts from imperfect documents
Provide an invented inspection form with equipment ID, inspector, date, fault category and required follow-up. Use a supported document-understanding or multimodal extraction workflow to produce structured output. Validate the schema and check every populated value against the source rather than accepting a neat JSON object as sufficient proof.
Create an intentionally incomplete form and one with conflicting IDs. The system should represent missing or uncertain fields without fabricating a complete record. Save source references and review flags so a human can inspect the original. A structurally valid response that associates a date with the wrong equipment remains a factual failure.
Compare a layout-aware extraction method with a simpler text extraction only where both are appropriate. Note whether table layout, handwriting or page structure changes the result. The goal is not to demonstrate as many products as possible; it is to choose the approach that preserves the information the business actually needs.
Lab 5: preserve meaning across images and speech
Add a synthetic photograph of a damaged component and a short authorized voice note. For the image, ask the model to describe visible evidence, identify uncertainty and avoid claiming an internal mechanical fault that cannot be seen. For speech, transcribe the note before interpreting its business meaning and preserve the difference between what was said and what the assistant inferred.
Test a blurry image, ambiguous spoken identifier and embedded image text that tries to give the assistant instructions. The application must maintain the same tool and data permissions regardless of what text it discovers in the image. If an important identifier is unclear, a request for human verification is a valid outcome.
Score image understanding and text transcription separately from the final decision. A perfect transcript could still describe a different machine, and an accurate image caption could be too uncertain to support a repair action. Multimodal testing should preserve source identity, uncertainty and approved review responsibility.
Lab 6: turn evaluations into a release gate
Combine examples from the prior labs into a small reusable evaluation set. Include correct answers, stale sources, denied tools, invalid structured output, ambiguous images and a simulated missing dependency. For each example, define what a correct result looks like and which failure is unacceptable. Preserve the expected outcome independently of the model’s own judgment about its response.
Run the set against a known-good configuration. Then change a model version, prompt or retrieval parameter in the disposable environment and compare results. Capture quality, groundedness, relevant safety constraints, latency and cost where useful. A higher average answer score should not conceal a new unauthorized tool call.
Integrate appropriate deterministic checks into a controlled CI/CD workflow: schema validity, allowed tool names, forbidden side effects and required error states. Reserve human review for claims that depend on subject expertise or business interpretation.
Make the release decision explicit. If the new configuration performs worse on an important case, stop promotion and restore the known-good candidate. Successful code build and HTTP health are not sufficient evidence that an AI assistant meets its user-facing obligations.
Lab 7: close the loop with observability and rollback
Deploy the safe test application and create one controlled regression, such as changing the retrieval index to point to an obsolete synthetic procedure. Record request correlation, retrieval version, model configuration, output validation and the resulting quality failure. Use suitable Azure monitoring for platform observations, while retaining AI-specific evaluation evidence for content quality.
Diagnose the earliest failing component and correct only what the evidence supports. Restore the previous source or index configuration and rerun the original and unauthorized cases. Check that an apparently fixed answer did not arrive by permitting a different user’s data or bypassing a validation step.
Finish by deleting disposable resources, confirming temporary credentials and access changes are no longer active, and recording the cost incurred by the experiments. A lab that teaches safe AI engineering should not leave an unexpected open service or persistent billable infrastructure behind.
The useful portfolio is small but defensible: expected behavior, configuration, evidence, failure, correction, negative test and cleanup for each exercise. AI-103 practical skill grows when the candidate can explain why one component failed and how the surrounding controls prevented that failure from becoming an unsupported answer or unauthorized action.