View Full Databricks Certified Data Engineer Associate Exam Dumps and Practice Test Dumps.

Question 341

A data engineer needs to load files from cloud storage into a Delta table and wants subsequent executions to avoid reprocessing files that have already been successfully loaded. Which command is designed for this use case?

1. COPY INTO
2. DESCRIBE HISTORY
3. VACUUM
4. OPTIMIZE

Correct Answer: 1

Explanation

COPY INTO is designed to incrementally load files from cloud storage into a Delta table while tracking which files have already been processed. This makes it useful for recurring file-ingestion workloads where new files arrive over time and previously loaded files should not be unnecessarily processed again. DESCRIBE HISTORY inspects Delta transaction history, while VACUUM removes eligible obsolete files and OPTIMIZE improves data layout. Therefore, COPY INTO is appropriate for incremental file ingestion from cloud storage.

Question 342

A data engineer runs a recurring file-ingestion process against a directory containing both old files and newly arrived files. The engineer wants the process to load only files that have not previously been copied into the target table. Which behavior of COPY INTO supports this requirement?

1. It automatically deletes old source files
2. It tracks previously loaded files and skips them on subsequent runs
3. It converts every source file into a view
4. It requires all files to be manually renamed

Correct Answer: 2

Explanation

COPY INTO is designed for incremental ingestion and keeps track of files that have already been loaded into the target table. When the same source directory is processed again, previously loaded files can be skipped while newly available files are ingested. This behavior makes COPY INTO useful for recurring batch-style file ingestion without requiring engineers to manually maintain a list of processed files. It does not delete source files or convert them into views, and manual renaming is unnecessary.

Question 343

A data engineer wants to repartition a DataFrame to increase the number of partitions before a large downstream operation. Which Spark method is appropriate when a shuffle-based repartition is acceptable?

1. cache()
2. collect()
3. repartition()
4. printSchema()

Correct Answer: 3

Explanation

The repartition() method can change the number of partitions of a DataFrame and can redistribute records across partitions. Because repartitioning generally involves a shuffle, it should be used when the resulting partition distribution is beneficial for the workload. cache() stores reusable data, collect() transfers records to the driver, and printSchema() displays schema information. Therefore, repartition() is the appropriate method when an engineer intentionally needs to redistribute data and increase or otherwise control the number of partitions.

Question 344

A data engineer has a DataFrame with many partitions and wants to reduce the number of partitions while minimizing a full shuffle when possible. Which method is generally appropriate?

1. repartition()
2. collect()
3. broadcast()
4. coalesce()

Correct Answer: 4

Explanation

coalesce() can reduce the number of partitions while generally avoiding a full shuffle. This can be useful after filtering operations have substantially reduced the amount of data and left many partitions with little work. repartition() can redistribute data through a shuffle, while collect() moves data to the driver and broadcast() is primarily used for distributing a small dataset to executors for joins. Therefore, coalesce is generally appropriate when reducing partitions without requiring a full redistribution.

Question 345

A data engineer wants to make a small reference DataFrame available to executors when joining it with a much larger DataFrame. Which Spark technique can reduce the need to shuffle the large dataset?

1. Broadcast join
2. Full outer join
3. collect()
4. repartition both datasets randomly

Correct Answer: 1

Explanation

A broadcast join can distribute a small DataFrame to the executors so that the larger DataFrame does not necessarily need to be shuffled across the cluster for the join. This can significantly improve performance when one side of the join is sufficiently small to be broadcast safely. A full outer join describes join semantics rather than an optimization, collect() moves data to the driver, and random repartitioning can introduce additional shuffle. Therefore, a broadcast join is appropriate for a small reference dataset.

Question 346

A data engineer needs to remove all records from a Delta table that match a condition while keeping the table itself available for future inserts. Which SQL operation should be used?

1. DROP TABLE
2. DELETE
3. VACUUM
4. ALTER TABLE DROP

Correct Answer: 2

Explanation

The DELETE operation removes rows that satisfy a specified condition while leaving the table itself available. For example, DELETE FROM orders WHERE status = ‘cancelled’ removes matching records without removing the table definition. DROP TABLE removes the table object, VACUUM handles eligible obsolete files, and ALTER TABLE DROP is used for supported table-definition changes rather than conditional row deletion. Therefore, DELETE is the appropriate operation when selected records must be removed while preserving the table.

Question 347

A data engineer needs to change the value of an existing column for rows that meet a specific condition without replacing the entire table. Which Delta Lake operation is appropriate?

1. INSERT
2. DELETE
3. UPDATE
4. CREATE TABLE

Correct Answer: 3

Explanation

The UPDATE operation modifies existing rows that satisfy a specified condition. This makes it suitable when a data engineer needs to correct or change values in selected records without rebuilding the entire table. INSERT adds new records, DELETE removes records, and CREATE TABLE defines a table. For example, an engineer could update a customer’s status only where a specific customer identifier matches. Therefore, UPDATE is the appropriate operation for modifying existing Delta table records conditionally.

Question 348

A data engineer wants to add new records to an existing Delta table without modifying or removing records already present. Which write behavior is appropriate?

1. Overwrite
2. Merge with delete condition only
3. Update
4. Append

Correct Answer: 4

Explanation

Append mode adds new records to an existing table while preserving the records already stored there. It is appropriate when incoming data represents additional rows that do not need to replace or modify existing records. Overwrite replaces the existing data according to the operation’s scope, UPDATE changes existing rows, and MERGE is used for conditional combinations of inserts, updates, and potentially deletes. Therefore, append is the appropriate write behavior when new records should simply be added to the existing Delta table.

Question 349

A data engineer needs to change the name of an existing Delta table column without changing the actual values stored in that column. Which SQL operation is appropriate?

1. ALTER TABLE … RENAME COLUMN
2. DELETE
3. VACUUM
4. MERGE

Correct Answer: 1

Explanation

ALTER TABLE … RENAME COLUMN can change the name of a table column while preserving the data associated with that column. This is useful when a column name needs to be corrected or standardized without rebuilding the table. DELETE removes rows, VACUUM handles obsolete files, and MERGE performs conditional data modifications. Renaming a column is a schema-definition operation rather than a data-transformation operation. Therefore, ALTER TABLE with the RENAME COLUMN clause is appropriate for this requirement.

Question 350

A data engineering team wants to add a new column to an existing table definition without recreating the entire table. Which SQL operation should be considered?

1. DROP TABLE
2. ALTER TABLE … ADD COLUMNS
3. VACUUM
4. DELETE FROM

Correct Answer: 2

Explanation

ALTER TABLE … ADD COLUMNS can modify the table schema by adding new columns without requiring the entire table to be recreated. This is useful when the data model evolves and an additional field needs to be introduced. DROP TABLE removes the table, VACUUM manages obsolete files, and DELETE removes matching records. Adding a column is a schema modification rather than a row-level data operation. Therefore, ALTER TABLE with ADD COLUMNS is appropriate for this requirement.

Question 351

A data engineer wants to make a Delta table’s transaction history available for investigation without modifying the table data. Which operation should be used?

1. UPDATE
2. DESCRIBE HISTORY
3. DELETE
4. INSERT

Correct Answer: 2

Explanation

DESCRIBE HISTORY reads information about the historical transactions performed on a Delta table without changing the table’s data. It can help engineers investigate versions, operations, timestamps, and other transaction-related details. UPDATE and DELETE modify existing data, while INSERT adds new records. Therefore, DESCRIBE HISTORY is appropriate when the objective is to inspect the table’s transaction history without performing a data modification.

Question 352

A data engineer wants to reduce the amount of data scanned when queries frequently filter on a column with useful file-level statistics. Which capability can help the query engine avoid reading irrelevant files?

1. Git folders
2. Service principals
3. Data skipping
4. Job notifications

Correct Answer: 3

Explanation

Data skipping uses file-level statistics to identify files that cannot contain records matching a query’s filter conditions. When the relevant statistics are available, the engine can avoid scanning those files, reducing the amount of data processed. Git folders manage source code, service principals provide identities, and job notifications communicate workflow events. Therefore, data skipping is the capability directly associated with reducing unnecessary file reads based on file-level statistics.

Question 353

A data engineer needs to inspect the schema of a DataFrame during development to verify column names and data types before applying transformations. Which Spark method should be used?

1. printSchema()
2. collect()
3. count()
4. write()

Correct Answer: 1

Explanation

The printSchema() method displays the structure of a DataFrame, including column names, data types, and nested fields where applicable. It is useful during development and troubleshooting when an engineer needs to verify that the input data has the expected structure. collect() retrieves records to the driver, count() calculates the number of records, and write() is used to persist data. Therefore, printSchema() is the appropriate method for inspecting a DataFrame’s schema.

Question 354

A data engineer wants to calculate the number of records in a DataFrame to validate that an ingestion process loaded approximately the expected volume. Which Spark operation should be used?

1. select()
2. count()
3. filter()
4. withColumn()

Correct Answer: 2

Explanation

The count() action returns the number of records in a DataFrame and can be useful for validating ingestion volumes or checking the results of transformations. Because it is an action, Spark must execute the required computation to produce the result. select() chooses columns, filter() restricts rows based on conditions, and withColumn() creates or transforms a column. Therefore, count() is the appropriate operation when an engineer needs to determine the number of records in a DataFrame.

Question 355

A data engineer needs to create a new column based on an expression involving two existing columns. Which PySpark DataFrame method is designed for adding or replacing a column?

1. withColumn()
2. orderBy()
3. dropDuplicates()
4. groupBy()

Correct Answer: 1

Explanation

The withColumn() method can create a new DataFrame column or replace an existing column using a specified expression. For example, an engineer can calculate a total amount from quantity and unit price and store the result in a new column. orderBy() sorts records, dropDuplicates() removes duplicate rows, and groupBy() creates grouped data for aggregation. Therefore, withColumn() is the appropriate method for creating a derived column from existing DataFrame fields.

Question 356

A data engineer needs to remove an unnecessary column from a DataFrame before writing the result to a target table. Which method should be used?

1. groupBy()
2. withColumn()
3. drop()
4. orderBy()

Correct Answer: 3

Explanation

The drop() method can remove one or more columns from a DataFrame. This is useful when unnecessary fields should be excluded before downstream processing or writing the transformed data to a target table. groupBy() organizes rows for aggregation, withColumn() creates or replaces columns, and orderBy() sorts the DataFrame. Therefore, drop() is the appropriate method when a data engineer needs to remove an unwanted column from the DataFrame.

Question 357

A data engineer wants to remove duplicate customer records based on a stable customer identifier while retaining one record for each identifier. Which DataFrame operation is appropriate?

1. dropDuplicates([“customer_id”])
2. orderBy(“customer_id”)
3. groupBy(“customer_id”)
4. select(“customer_id”)

Correct Answer: 1

Explanation

dropDuplicates([“customer_id”]) removes duplicate rows based on the specified customer identifier. This is useful when multiple records may represent the same customer and the pipeline needs to retain one record for each identifier. orderBy() only changes ordering, groupBy() creates grouped data for aggregation, and select() chooses columns without removing duplicates. Therefore, dropDuplicates() with the stable customer ID is the appropriate operation for this deduplication requirement.

Question 358

A data engineer needs to combine two DataFrames that contain the same set of columns and represent records from different regions. Which operation can combine their rows into one DataFrame?

1. join()
2. groupBy()
3. union()
4. orderBy()

Correct Answer: 3

Explanation

The union() operation combines the rows of two DataFrames that have compatible schemas. It is useful when separate datasets represent the same type of records, such as sales from different regions, and need to be combined into a single dataset. join() combines columns based on matching conditions, groupBy() creates groups for aggregation, and orderBy() sorts records. Therefore, union() is appropriate when two compatible DataFrames need to be stacked vertically.

Question 359

A data engineer wants to combine customer records with matching account records while returning only customers that have a corresponding account. Which join type is appropriate?

1. Left join
2. Full outer join
3. Inner join
4. Cross join

Correct Answer: 3

Explanation

An inner join returns rows where the join condition matches in both datasets. In this scenario, only customers with corresponding account records should be included, making an inner join appropriate. A left join would retain customers even when no account exists, a full outer join would retain unmatched rows from both sides, and a cross join produces combinations without requiring matching keys. Therefore, an inner join is the suitable join type when only matching customer-account records are required.

Question 360

A data engineer needs to retain every customer from the customer table while adding account information when a matching account exists. Customers without accounts must remain in the result. Which join should be used?

1. Inner join
2. Cross join
3. Right join
4. Left join

Correct Answer: 4

Explanation

A left join preserves every row from the left DataFrame and adds matching information from the right DataFrame when available. In this case, the customer table should be the left side, ensuring that customers without an account still appear in the result with null values for missing account information. An inner join would remove unmatched customers, while a cross join creates combinations of rows. Therefore, a left join is appropriate for retaining all customers regardless of account matches.