This would test a different behavior, chiefly preserve the intended Auto Loader checkpoint and schema state across permitted restarts. The checkpoint determines previously discovered input files and prevents unnecessary reprocessing on recovery. The specific evidence needed here is that a malformed fixture is available with its source path and reason while trusted downstream tables exclude it.

This would test a different behavior, chiefly configure Auto Loader with an appropriate cloud-file source and durable schema/checkpoint location. Auto Loader tracks discovered files and incrementally processes newly arrived objects without costly full-directory enumeration. The specific evidence needed here is that a malformed fixture is available with its source path and reason while trusted downstream tables exclude it.

This would test a different behavior, chiefly configure the Auto Loader schema evolution strategy appropriate to the source and production contract. Schema evolution must handle newly observed fields without silently corrupting established column types. The specific evidence needed here is that a malformed fixture is available with its source path and reason while trusted downstream tables exclude it.

Dimensional modeling separates measures from descriptive attributes and makes aggregation grain explicit. That option implements design a star schema with a fact table at a defined grain and separate conformed dimensions. It addresses a different problem; here the distinguishing need is to document the fact-table grain and choose aggregations that respect that grain.

Metric governance requires controlled semantic changes and evidence that historical and new calculations are understood. That option implements version and validate the governed metric definition with representative reconciliation tests. It addresses a different problem; here the distinguishing need is to document the fact-table grain and choose aggregations that respect that grain.

A balance is a point-in-time semi-additive measure and requires snapshot-aware aggregation. That option implements model periodic snapshot facts at the defined observation date rather than summing transactions as if they were balances. It addresses a different problem; here the distinguishing need is to document the fact-table grain and choose aggregations that respect that grain.

This would test a different behavior, chiefly join relevant audit events by object, principal and event time in supported system tables. A timeline connecting policy changes and access requests can establish sequence better than independent screenshots. The specific evidence needed here is that a time-series view identifies idle intervals and confirms lower spend after auto-stop adjustments.

This would test a different behavior, chiefly analyze the supported audit system tables over the relevant principal, object and time range. Centralized audit event records provide historical evidence of actions that a current permission listing cannot show. The specific evidence needed here is that a time-series view identifies idle intervals and confirms lower spend after auto-stop adjustments.

This would test a different behavior, chiefly query the supported Databricks billing and usage system tables with workload or resource attribution. System tables expose account-level metering and usage records that can be aggregated by job or compute. The specific evidence needed here is that a time-series view identifies idle intervals and confirms lower spend after auto-stop adjustments.

This would test a different behavior, chiefly check package layout and wheel metadata, then attach a correctly built wheel to the job task. Python wheel import resolution depends on the packaged module structure and installation into the task environment. The specific evidence needed here is that a clean job cluster imports the required package and records the intended installed version.

This would test a different behavior, chiefly publish the approved versioned wheel to an authorized artifact location and reference it in job dependencies. Managed artifact distribution provides traceability without exposing proprietary packages through public repositories. The specific evidence needed here is that a clean job cluster imports the required package and records the intended installed version.

This would test a different behavior, chiefly inspect the package dependency tree and resolve version conflicts in the declared environment. Conflicting transitive requirements must be reconciled rather than hidden by rerunning the same failing installation. The specific evidence needed here is that a clean job cluster imports the required package and records the intended installed version.

Spark task retries and parallel execution do not preserve a single shared mutable Python process state. That option implements make the transformation deterministic and side-effect-free instead of depending on Python global state. It addresses a different problem; here the distinguishing need is to use a supported Pandas UDF to process batches while explicitly specifying its return schema.

Pure DataFrame transformations are reusable and can be tested independently of notebook state. That option implements implement a pure DataFrame-to-DataFrame function and apply it with DataFrame.transform. It addresses a different problem; here the distinguishing need is to use a supported Pandas UDF to process batches while explicitly specifying its return schema.

Native expressions benefit from Spark optimization and avoid expensive Python boundary serialization for simple operations. That option implements use built-in Spark SQL expressions where possible instead of a Python UDF. It addresses a different problem; here the distinguishing need is to use a supported Pandas UDF to process batches while explicitly specifying its return schema.

CDC target correctness relies on deterministic sequencing and replay-safe incremental state management. That option implements use stable keys, sequence ordering and managed incremental pipeline state when applying AUTO CDC. It addresses a different problem; here the distinguishing need is to use AUTO CDC with stable keys, sequence ordering, and delete handling for SCD Type 1.

CDC delete-handling logic translates source tombstones into target state changes instead of treating them as ordinary updates. That option implements configure AUTO CDC delete expressions for source tombstones on identified keys. It addresses a different problem; here the distinguishing need is to use AUTO CDC with stable keys, sequence ordering, and delete handling for SCD Type 1.

SCD Type 2 preserves prior versions with validity periods while maintaining the current version. That option implements use AUTO CDC configured for SCD Type 2 with the appropriate sequence column. It addresses a different problem; here the distinguishing need is to use AUTO CDC with stable keys, sequence ordering, and delete handling for SCD Type 1.

Execution profiling reveals the costly operators and task distribution that are not evident from business output rows. That option implements open Spark UI and query profile to examine stages, shuffle and task skew. It addresses a different problem; here the distinguishing need is to use the Databricks CLI or SDK with approved permissions to list pipeline metadata and updates.

Operator-level profiling distinguishes inefficient data access from downstream aggregation work. That option implements inspect the SQL query profile for scan volumes, shuffles, joins and spill information. It addresses a different problem; here the distinguishing need is to use the Databricks CLI or SDK with approved permissions to list pipeline metadata and updates.

Programmatic run metadata supports automation and central dashboards without screen scraping. That option implements query the supported Jobs REST API or SDK for run and task state. It addresses a different problem; here the distinguishing need is to use the Databricks CLI or SDK with approved permissions to list pipeline metadata and updates.

This would test a different behavior, chiefly use query profile scan metrics to identify predicate pushdown or data-skipping regressions. Scan-level evidence distinguishes inefficient reads from subsequent aggregation performance. The specific evidence needed here is that the bad fixture is identified and appropriate quarantine handling allows valid partitions to complete.

This would test a different behavior, chiefly inspect Spark UI memory, shuffle spill and executor logs to choose a join or memory fix. Execution metrics identify whether memory pressure comes from wide shuffles, skew or configuration. The specific evidence needed here is that the bad fixture is identified and appropriate quarantine handling allows valid partitions to complete.

This would test a different behavior, chiefly use Spark UI task timelines and shuffle metrics to isolate skewed partitions before applying a targeted fix. Disproportionate task durations and shuffle bytes reveal skew rather than general CPU scarcity. The specific evidence needed here is that the bad fixture is identified and appropriate quarantine handling allows valid partitions to complete.

This would test a different behavior, chiefly choose an output mode supported by the stateful aggregation and its target sink. Append mode cannot emit continually revised aggregates until they are finalized; output mode must match query semantics. The specific evidence needed here is that restarting after a simulated failure does not duplicate external records for the same batch ID.

This would test a different behavior, chiefly configure a durable distinct Structured Streaming checkpoint location for the query. Checkpoint metadata records source progress and state for consistent recovery across restarts. The specific evidence needed here is that restarting after a simulated failure does not duplicate external records for the same batch ID.

This would test a different behavior, chiefly optimize microbatch processing throughput or scale compute before merely decreasing trigger interval. A shorter trigger does not make a slow microbatch run faster and cannot resolve sustained capacity deficits. The specific evidence needed here is that restarting after a simulated failure does not duplicate external records for the same batch ID.

This would test a different behavior, chiefly use appropriate data layout or liquid clustering based on actual filter patterns. Good layout clusters commonly filtered values to improve data skipping without overpartitioning the table. The specific evidence needed here is that after prescribed retention and purge, prohibited record bytes are no longer accessible through supported historical reads.

This would test a different behavior, chiefly optimize the affected Delta table to compact small files into more efficient scan units. File compaction reduces file-opening overhead and improves pruning and read efficiency for many small files. The specific evidence needed here is that after prescribed retention and purge, prohibited record bytes are no longer accessible through supported historical reads.

This would test a different behavior, chiefly prefer supported Unity Catalog managed tables with automated maintenance where data-governance requirements permit. Managed tables allow platform automation of storage optimization and lifecycle operations that reduce operational overhead. The specific evidence needed here is that after prescribed retention and purge, prohibited record bytes are no longer accessible through supported historical reads.

Row-filter policies evaluate authorized rows at query time across all matching governed objects. That option implements apply an ABAC row-filter policy driven by governed region tags and an identity-aware UDF. It addresses a different problem; here the distinguishing need is to define governed tags with limited assignment permissions before relying on tag-scoped ABAC.

Tag-conditioned ABAC policies provide centralized reusable protection as new matching columns appear. That option implements create Unity Catalog ABAC masking policies scoped to governed tags at the appropriate catalog or schema. It addresses a different problem; here the distinguishing need is to define governed tags with limited assignment permissions before relying on tag-scoped ABAC.

Fine-grained ABAC restrictions have compute prerequisites and cannot be assumed to work on all older runtimes. That option implements use a supported serverless or compatible runtime/access configuration for ABAC policy enforcement. It addresses a different problem; here the distinguishing need is to define governed tags with limited assignment permissions before relying on tag-scoped ABAC.