View Full Databricks Certified Associate Developer for Apache Spark Exam Dumps  and Practice Test Dumps

 

Question 361.

Which Spark SQL function can calculate the average value of a numeric column across grouped rows?

1. avg()
   2. count()
   3. collect_list()
   4. first()

Correct Answer: 1. avg()

Explanation:

avg() computes the arithmetic mean of a numeric expression and is commonly used with groupBy() to calculate averages for logical groups. For example, groupBy(“department”).agg(avg(“salary”)) returns the average salary for each department. count() measures row or value frequency, collect_list() aggregates multiple values into an array, and first() returns one value according to aggregation semantics. avg() is therefore the appropriate function when a developer needs a standard mean calculation. Like other aggregations, grouped averages may require Spark to shuffle records so values belonging to the same key can be processed together.

Question 362.

Which Spark SQL aggregation returns the total value of a numeric column?

1. max()
   2. sum()
   3. avg()
   4. countDistinct()

Correct Answer: 2. sum()

Explanation:

sum() calculates the total of a numeric column or expression. It can be used across an entire DataFrame or within groups defined by groupBy(). For example, summing an amount column by customer can calculate each customer’s total purchases. max() returns the largest value, avg() calculates the mean, and countDistinct() counts unique values. sum() is one of the most common aggregation functions in Spark and is widely used in financial reporting, inventory analysis, usage metrics, and other workloads where numeric values must be accumulated across many distributed records.

Question 363.

Which Spark SQL aggregation should be used to return the greatest value in a column?

1. min()
   2. first()
   3. max()
   4. last()

Correct Answer: 3. max()

Explanation:

max() returns the greatest value found in the specified expression. It can be used on many orderable data types, including numeric, date, and timestamp columns. min() returns the smallest value, while first() and last() return values based on aggregation semantics rather than comparing every value to find the maximum. max() is useful for identifying the largest transaction, latest date when represented directly by a comparable timestamp, highest score, or other upper-bound value. When used with groupBy(), Spark computes a separate maximum for each group.

Question 364.

Which Spark SQL aggregation returns the smallest value from a column?

1. first()
   2. avg()
   3. sum()
   4. min()

Correct Answer: 4. min()

Explanation:

min() returns the smallest value in a specified column or expression. It can be applied across an entire DataFrame or within groups created through groupBy(). The function is useful for finding the lowest price, earliest date, minimum score, or other lower-bound value. first() simply returns one value based on aggregation behavior, avg() computes a mean, and sum() computes a total. Because aggregation often requires records with the same group key to be processed together, grouped min() calculations can involve shuffle activity when data is distributed across partitions.

Question 365.

Which aggregation function counts the number of unique values in a column?

1. countDistinct()
   2. count()
   3. size()
   4. length()

Correct Answer: 1. countDistinct()

Explanation:

countDistinct() calculates the number of unique values in one or more expressions. It is commonly used to determine metrics such as unique customers, devices, products, or sessions. count() counts rows or non-null values depending on how it is called and does not automatically eliminate duplicates. size() measures nested arrays or maps, while length() is typically used with strings or binary values. countDistinct() may require distributed aggregation and data movement, so it can be more expensive than a simple count() when applied to very large high-cardinality datasets.

Question 366.

Which DataFrame pattern is most appropriate for calculating several grouped statistics in one operation?

1. select() followed by collect()
   2. groupBy().agg()
   3. orderBy().show()
   4. répartition().cache()

Correct Answer: 2. groupBy().agg()

Explanation:

groupBy().agg() allows multiple aggregation expressions to be calculated for each group in one logical operation. For example, a developer can calculate count(), sum(), avg(), min(), and max() for every product category. select() is used for projections, orderBy() sorts rows, and repartition() changes data distribution. Although Spark may optimize several aggregation expressions together, grouped aggregation can still cause a shuffle because rows with the same grouping keys must be brought together. Using agg() is therefore both expressive and efficient for producing multi-metric summary tables from distributed datasets.

Question 367.

Which aggregation function collects grouped values into an array while retaining duplicate values?

1. collect_set()
   2. array()
   3. collect_list()
   4. array_distinct()

Correct Answer: 3. collect_list()

Explanation:

collect_list() gathers values from multiple rows into an array and retains duplicates. It is useful when every observed value associated with a group should remain available, such as a customer’s complete purchase history or a list of event types. collect_set() removes duplicates during aggregation, array() constructs an array from expressions within a single row, and array_distinct() removes duplicate elements from an already existing array. Since distributed aggregation does not inherently guarantee a meaningful element order, additional ordering logic may be required if downstream processing depends on a deterministic sequence.

Question 368.

Which aggregation function creates an array containing only unique grouped values?

1. collect_list()
   2. array_distinct()
   3. distinct()
   4. collect_set()

Correct Answer: 4. collect_set()

Explanation:

collect_set() aggregates values from many rows into an array while eliminating duplicate values. It is useful for generating unique lists of categories, identifiers, locations, or statuses associated with each group. collect_list() retains duplicates, array_distinct() removes duplicates from an array that already exists within one row, and distinct() operates on complete DataFrame rows. Because collect_set() focuses on uniqueness rather than ordering, developers should not normally assume the returned elements have a stable sequence unless explicit sorting is applied afterward.

Question 369.

Which Spark SQL aggregation returns the first value observed according to the function’s aggregation semantics?

1. first()
   2. min()
   3. lead()
   4. row_number()

Correct Answer: 1. first()

Explanation:

first() returns the first value encountered for an aggregation and can optionally be configured to ignore nulls. It is different from row_number(), which is a window function that assigns sequential numbers according to an explicit ordering. lead() accesses a following row within a window, while min() compares all values to find the smallest. Developers should be careful when using first() in distributed aggregations because the concept of “first” may not represent a meaningful chronological order unless ordering has been explicitly established through an appropriate window or other deterministic logic.

Question 370.

Which Spark SQL aggregation can return the last value according to aggregation semantics?

1. first()
   2. last()
   3. max()
   4. lag()

Correct Answer: 2. last()

Explanation:

last() returns a final value from an aggregation according to Spark’s aggregation behavior and can optionally ignore null values. It should not automatically be interpreted as the chronologically latest record unless the data has been explicitly ordered using an appropriate technique. max() can identify the greatest timestamp but returns only that timestamp rather than another associated field. lag() is a window function that retrieves a previous row. When a developer needs the latest complete record by timestamp, a window ordered by time combined with row_number() is often clearer and more deterministic than relying on last().

Question 371.

A developer needs to calculate a cumulative sum while preserving every original row. Which approach is most appropriate?

1. groupBy() with sum() only
   2. collect_list()
   3. sum() over an ordered Window
   4. crossJoin()

Correct Answer: 3. sum() over an ordered Window

Explanation:

A windowed sum allows Spark to calculate cumulative totals while preserving each original row. The developer can partition by a logical key, order rows by a timestamp or sequence column, and define a frame from the beginning of the partition through the current row. groupBy() with sum() would collapse multiple input rows into summary rows and therefore lose row-level detail. collect_list() creates arrays, and crossJoin() creates unnecessary row combinations. Windowed aggregations are ideal when each record should remain visible alongside running totals, moving averages, rankings, or other contextual calculations.

Question 372.

Which window function is most appropriate for selecting exactly one latest row per customer when records are ordered by timestamp descending?

1. rank()
   2. dense_rank()
   3. lag()
   4. row_number()

Correct Answer: 4. row_number()

Explanation:

row_number() assigns a unique sequential number to each row within a window partition. By partitioning by customer, ordering timestamps in descending order, and filtering for row_number() == 1, a developer can retain exactly one latest record per customer. rank() and dense_rank() can assign the same rank to tied timestamps, potentially leaving more than one row with rank 1. lag() retrieves a previous value but does not select the latest record. row_number() is therefore a common and clear solution for deterministic top-one-per-group or deduplication workflows.

Question 373.

Which window function assigns equal ranks to ties and leaves gaps in the rank sequence afterward?

1. rank()
   2. dense_rank()
   3. row_number()
   4. lead()

Correct Answer: 1. rank()

Explanation:

rank() assigns the same rank to tied rows and leaves a gap after the tie. For example, ranking values might produce 1, 2, 2, and 4. dense_rank() also gives ties the same rank but produces 1, 2, 2, and 3 without a gap. row_number() assigns a unique sequence to every row regardless of ties, while lead() retrieves a value from a future row in the window. rank() is suitable when competition-style ranking semantics are required and skipped positions after ties are meaningful.

Question 374.

Which window function assigns equal values the same rank without skipping the next rank?

1. row_number()
   2. dense_rank()
   3. rank()
   4. lag()

Correct Answer: 2. dense_rank()

Explanation:

dense_rank() gives tied records the same rank and then continues with the next consecutive rank. For example, values could be ranked 1, 2, 2, and 3. rank() would instead produce 1, 2, 2, and 4 because it leaves a gap after the tie. row_number() assigns every row a unique sequence, while lag() retrieves a prior value. dense_rank() is commonly used when categories or scores should share a rank but the ranking sequence should remain compact without skipped positions.

Question 375.

Which window function should be used to access a value from the previous row in an ordered partition?

1. lead()
   2. first()
   3. lag()
   4. row_number()

Correct Answer: 3. lag()

Explanation:

lag() retrieves a value from a preceding row according to the ordering defined in a Window specification. It is useful for comparing current values with previous transactions, events, balances, or timestamps. lead() retrieves a value from a following row, first() is an aggregation or window function with different semantics, and row_number() generates a sequential number. By using partitionBy() and orderBy() carefully, lag() allows developers to calculate changes, elapsed time, growth rates, or transitions without self-joining the DataFrame.

Question 376.

Which window function should be used to retrieve a value from a following row in an ordered partition?

1. lag()
   2. rank()
   3. row_number()
   4. lead()

Correct Answer: 4. lead()

Explanation:

lead() retrieves a value from a subsequent row within the same logical window partition, using the ordering defined by the Window specification. It is useful for finding the next event, next timestamp, future balance, or upcoming status. lag() retrieves the previous value, while rank() and row_number() produce ranking or sequence information. lead() avoids the need for a self-join when a developer simply needs to compare each record with the next record in an ordered sequence.

Question 377.

Which Window method defines logical groups so calculations restart independently for each customer or account?

1. partitionBy()
   2. repartition()
   3. groupBy()
   4. coalesce()

Correct Answer: 1. partitionBy()

Explanation:

Window.partitionBy() defines logical partitions for analytical calculations. For example, partitioning by customer_id ensures that row_number(), lag(), lead(), rank(), or cumulative calculations restart separately for each customer. This is conceptually different from DataFrame.repartition(), which changes physical Spark partitions across the cluster. groupBy() collapses records when aggregations are performed, while coalesce() normally reduces physical partitions. Window partitioning preserves row-level detail while defining the groups within which analytical functions operate.

Question 378.

Which Window method determines the sequence in which rows are evaluated inside each logical partition?

1. rowsBetween()
   2. orderBy()
   3. groupBy()
   4. repartition()

Correct Answer: 2. orderBy()

Explanation:

Window.orderBy() establishes the row sequence used by functions such as row_number(), rank(), lag(), lead(), and cumulative aggregations. For temporal data, timestamp columns are frequently used so events are evaluated chronologically. rowsBetween() defines the frame boundaries relative to that ordering, while groupBy() is used for aggregation rather than window sequencing. DataFrame.repartition() changes physical distribution and should not be confused with logical window ordering. Without a meaningful order, many positional window calculations would not produce useful or deterministic business results.

Question 379.

Which window frame includes all rows from the beginning of the logical partition through the current row?

1. rowsBetween(Window.currentRow, Window.unboundedFollowing)
   2. rowsBetween(-1, 1)
   3. rowsBetween(Window.unboundedPreceding, Window.currentRow)
   4. rowsBetween(Window.currentRow, Window.currentRow)

Correct Answer: 3. rowsBetween(Window.unboundedPreceding, Window.currentRow)

Explanation:

A frame from Window.unboundedPreceding through Window.currentRow includes the entire history from the beginning of the logical partition up to the current row. It is commonly used for running sums, cumulative counts, and cumulative averages. A frame beginning at the current row and extending forward supports future-looking calculations. A -1 to 1 frame includes only nearby rows, while currentRow to currentRow includes only the current row. Defining the correct frame is essential because the same aggregation function can produce very different analytical results depending on which neighboring records are included.

Question 380.

Which approach should a developer use when they need grouped totals but do not need to preserve every original input row?

1. Use a window function for every aggregation
   2. Use collect() and aggregate on the driver
   3. Use a cross join
   4. Use groupBy() with aggregation functions

Correct Answer: 4. Use groupBy() with aggregation functions

Explanation:

groupBy() with aggregation functions is the standard Spark approach when many input records should be collapsed into summary rows. For example, grouping by region and applying sum(“sales”) returns one total for each region. Window functions are preferable when every original row must remain visible alongside analytical values. Collecting data to the driver sacrifices distributed processing and can exceed driver memory, while cross joins create unnecessary combinations. groupBy() may involve a shuffle because rows with the same key must be processed together, but it provides the appropriate distributed semantics for producing grouped summaries.