An AI application can appear ready after it answers several demonstration prompts. That is often the easiest part of the engineering problem. Production work begins when source documents change, users have different permissions, images contain uncertain evidence, a tool can trigger business actions and the team must explain why an answer changed after a release. A lifecycle view connects these conditions rather than treating each Azure AI feature as a separate demonstration.
For Microsoft AI-103 Exam, the April 16, 2026 objectives span planning a Foundry solution, implementing generative and agentic behavior, working with vision and language, extracting information, and securing, evaluating and operating the result. This article follows one fictional public-utility maintenance assistant from initial requirements to a verified update. It is not a personal study schedule or a catalogue of AI terms; the goal is to show how design decisions change as the system becomes accountable for real work.
Start with a task whose answer can be checked
A municipal water utility receives reports of leaking meters, damaged inspection boxes and low pressure. Employees submit structured work orders, photographs and short notes. The proposed AI assistant should identify the likely report category, locate an applicable maintenance procedure and prepare a draft action for a dispatcher. It must not close an incident, order a replacement part or reveal unrelated customer addresses without the organization’s authorized workflow.
Define a successful result before selecting a model. For one synthetic report, the application should extract equipment ID and location, cite the current approved procedure, indicate uncertainty and mark whether a human needs to review it. The system can support a dispatcher’s decision; it should not claim to diagnose an unseen physical fault or issue a binding safety instruction from a photograph alone.
Specify which outcomes are deterministic. A ticket number is validated by application code; a customer’s access permission is enforced by the backend; a model may classify the narrative and draft a plain-language explanation. Separating these responsibilities prevents the prompt from becoming an unreliable replacement for a business process.
Choose a Foundry architecture that reflects the data boundary
The utility’s records are divided among public maintenance guides, restricted customer tickets and engineering manuals. The application needs a trusted identity, an approved data retrieval path, a selected Foundry model deployment and an output contract. That system needs security and operational boundaries regardless of whether its conversational interface seems simple.
The current Azure AI architecture concepts help distinguish a model, a retrieval pipeline and the application that governs their use. A model cannot make a document current or authorized merely by describing it confidently. The host system should know which collection was queried, which source version was returned and whether that source belongs to the user’s permitted scope.
Start with a fixed workflow: validate the request, retrieve relevant records, call the model, validate a structured response and present the draft to a dispatcher. An agentic loop is not automatically necessary. Add model-directed tool selection only if the next evidence source genuinely depends on information found during the earlier step and the added flexibility can be bounded and tested.
Build an evidence pipeline before optimizing the answer
Ingest a small collection of synthetic maintenance policies and equipment records with titles, revision identifiers and effective dates. Index them using an appropriate retrieval method, then test questions that have an answer, have no answer and contain two conflicting procedures. The application should identify the supporting document or state that available information is insufficient rather than fabricate the missing policy.
Retrieval errors and generation errors are different. A search index may omit the most relevant document because of ingestion or chunking. A model may receive the correct document but infer a rule it does not contain. Evaluation should distinguish whether the failure occurred before or after evidence reached the model. Increasing prompt length will not necessarily fix missing or incorrectly scoped retrieval.
Access controls must follow the source data. A maintenance record about one customer should not be exposed to a user authorized only for another district. The application should enforce document access in its retrieval process or underlying data service instead of trusting the model’s instruction to avoid disclosing private information.
Preserve source metadata alongside the answer. Without an identifier and version, a later reviewer may be unable to determine whether a wrong recommendation came from an obsolete policy or a model interpretation error.
Add agent tools under explicit business authority
The assistant can initially look up equipment status and prepare a draft work item. Give each capability a clear input schema, permitted action and error response. The model may suggest invoking a tool, but the application should validate that the actual user or workload identity is authorized for the equipment record and intended operation.
For a read-only lookup, returning “record not found,” “not authorized” and “service unavailable” as distinct results helps the model and application reason about the next step. These conditions are not equivalent: the first may justify asking for a corrected identifier, the second should not trigger a broader search for private records, and the third may require a bounded retry or an incomplete status.
If the workflow later permits creating a maintenance-review task, ensure duplicate requests cannot create multiple work orders merely because a tool timed out after performing its action. An idempotency key or other transactional mechanism can protect that boundary. A generated apology for a duplicate is not a substitute for preventing the duplicate effect.
Human approval belongs at the actual side-effect boundary. A dispatcher may approve a draft after examining evidence, but the system should not modify live equipment status before that approval occurs. Include authorization and audit details in the application design rather than relying on a prompt that asks the model to “be careful.”
Introduce multimodal evidence with limits on inference
An incident report may include a photograph of a broken meter cover, a spoken field note and a PDF maintenance form. These inputs can require vision, speech, text analysis and information extraction. The model may identify visible components or summarize a transcript while remaining uncertain about causes that cannot be observed directly.
Give the application one simple structured task per modality. Extract the meter identifier from a synthetic form, summarize a maintenance note and flag a blurry photograph for human inspection. Validate the resulting fields against permitted formats and compare the content with approved source evidence. A schema-valid output still may contain the wrong serial number or an unsupported damage conclusion.
Text embedded inside an image or scanned document should be treated as untrusted content, not as a new instruction source. The application must not expand data permissions or issue external actions because a document contains text directing the assistant to do so.
Measure the quality of extraction separately from downstream decision quality. A correctly extracted field does not make a wrong repair recommendation acceptable, while a partial extraction may still support a useful review task if uncertainty is properly surfaced.
Evaluate the whole workflow against realistic failure cases
Create an evaluation set of synthetic utility reports. Include a straightforward leak description, ambiguous address, conflicting manual versions, missing equipment ID, low-quality photo, unauthorized record and unavailable external tool. For each, define what a safe, complete or partial response should contain. Record which source and model version produced the result.
Some expectations can be enforced mechanically: valid schema, permitted tool names, nonempty record references and absence of unapproved side effects. Others require human subject-matter judgment: was the cited procedure appropriate, was the uncertainty adequately explained and would the draft help a dispatcher make a sound decision?
Do not measure success only through fluency or an average quality score. A system that answers nineteen ordinary questions beautifully but accesses a protected customer record on the twentieth has an authorization failure. A system that refuses every question may reduce risk but fail to provide the intended business value. Evaluation needs both usefulness and clearly enforced boundaries.
When a test fails, label the responsible component—retrieval, source freshness, interpretation, tool authorization, schema, output evaluation or human workflow. Repair that component before building a longer generic prompt.
Deploy the application as a versioned system
Model deployments, prompt templates, retrieval settings, source documents, tool schemas and application code can each alter behavior. Record their versions so that a regression can be reproduced. An application release should include authorized configuration, integration tests, evaluation cases and a way to restore a known-good state if the change worsens results.
CI/CD pipelines can automate deterministic checks and approved deployments, but should not treat a successful build as proof that the assistant remains safe. An updated tool description might cause more action requests even when the code is unchanged. A revised policy document might change answers without any software deployment.
Use appropriate secret management such as Azure Key Vault where the integration requires protected credentials. Prefer scoped identities and avoid placing account keys in source files, log output or model context. The purpose is to keep the application’s technical authority no broader than its operational role.
Operate from evidence rather than a green endpoint
After release, the assistant begins recommending an outdated meter inspection step. A generic health check reports HTTP success and normal latency. Those observations show the service is reachable, not that the business answer is grounded. Investigators need the request trace, retrieved source identifiers, tool results, output validation and model or prompt version involved in the recommendation.
Compare the affected case with an earlier known-good run. Did the retrieval index return an old manual? Did the current source change? Did the model ignore a contradictory document? Was the result evaluated using the correct policy? Each hypothesis leads to a different fix. Azure monitoring can support operational telemetry, but an AI-specific evaluation process must still judge content and tool behavior.
Introduce a controlled regression in a training environment and observe whether alerts, evaluation checks or the dispatcher review identify it. Then roll back the affected configuration or correct the source and repeat both a permitted and prohibited test. Do not restore customer-facing service merely because the technical endpoint has returned to green while the recommendation remains wrong.
The AI-103 lifecycle begins with a bounded user requirement and ends with a system that can be checked, corrected and improved. Planning, retrieval, tools, multimodal processing, evaluation, security and operations are not isolated chapters once customers rely on the application. The engineer’s lasting skill is explaining how those components produce an authorized, grounded result—and what evidence should change the design when they do not.