View Full Databricks Certified Associate Developer for Apache Spark Exam Dumps  and Practice Test Dumps

 

Question 261.

A developer wants to combine two DataFrames by column name and allow missing columns to be filled with null values. Which operation is most appropriate?

1. union()
   2. unionByName(…, allowMissingColumns=True)
   3. crossJoin()
   4. left_outer join

Correct Answer: 2. unionByName(…, allowMissingColumns=True)

Explanation:

unionByName() matches columns according to their names instead of their positions. When allowMissingColumns=True is supported and enabled, Spark can align schemas even when one DataFrame contains columns that the other does not, filling the missing fields with null values. This is useful when combining evolving datasets from different processing runs or sources. union() generally depends on compatible positional schemas, crossJoin() creates a Cartesian product, and a left outer join combines rows according to a matching condition rather than vertically appending them.

Question 262.

Which Spark function is most appropriate when a developer wants to create a single struct column from several existing columns?

1. collect_list()
   2. array()
   3. struct()
   4. explode()

Correct Answer: 3. struct()

Explanation:

struct() combines multiple column expressions into a single nested struct column. This is useful when related values should be grouped into one logical object, such as combining street, city, and postal code into an address structure. array() creates an ordered collection of values rather than named fields, collect_list() aggregates values across rows, and explode() expands nested collections into separate rows. struct() is commonly used when preparing nested data for JSON output or when representing hierarchical records in a structured format.

Question 263.

Which Spark SQL function should be used to expand an array column so that each array element becomes a separate row?

1. explode()
   2. collect_set()
   3. array_union()
   4. flatten()

Correct Answer: 1. explode()

Explanation:

explode() takes an array or map column and produces multiple output rows, one for each element. This is useful when nested collections must be normalized before filtering, joining, or aggregation. collect_set() aggregates values into an array while removing duplicates, array_union() combines arrays, and flatten() removes one level of array nesting without generating separate rows. explode() is therefore the appropriate choice when the goal is to transform nested collection elements into individual records while retaining the remaining columns from the original row.

Question 264.

Which Spark SQL function returns both the position and value of each element when expanding an array?

1. explode_outer()
   2. flatten()
   3. array_position()
   4. posexplode()

Correct Answer: 4. posexplode()

Explanation:

posexplode() expands an array into multiple rows while returning both the element value and its position within the original array. This is useful when element ordering matters and the developer needs to preserve index information. explode() would return only the values, while explode_outer() adds row-preservation behavior for null or empty collections. array_position() finds the location of a specified value but does not expand every element. posexplode() is therefore the best choice when an array must be flattened into rows and its original ordering information should remain available.

Question 265.

Which Spark SQL function can combine nested arrays into a single-level array?

1. flatten()
   2. explode()
   3. collect_list()
   4. array_repeat()

Correct Answer: 1. flatten()

Explanation:

flatten() removes one level of nesting from an array of arrays. For example, a value such as [[1, 2], [3, 4]] can become [1, 2, 3, 4]. explode() creates separate rows from collection elements, collect_list() aggregates values across rows into an array, and array_repeat() creates an array containing repeated instances of one value. flatten() is particularly useful when nested collection structures have been created during aggregation or parsing and the developer wants to work with a single-level array instead.

Question 266.

Which Spark SQL function creates an array containing a value repeated a specified number of times?

1. repeat()
   2. array_repeat()
   3. collect_list()
   4. sequence()

Correct Answer: 2. array_repeat()

Explanation:

array_repeat() creates an array containing the specified value repeated a defined number of times. This is different from repeat(), which operates on strings. collect_list() aggregates values from multiple rows, while sequence() generates an ordered sequence of numeric or temporal values. array_repeat() can be useful for test data, filling nested structures, or constructing arrays of known size. Because the repetition occurs within each row’s expression, it does not require aggregation across multiple records.

Question 267.

Which Spark SQL function can generate an array containing a sequence of numbers or dates between specified start and stop values?

1. range()
   2. collect_list()
   3. sequence()
   4. monotonically_increasing_id()

Correct Answer: 3. sequence()

Explanation:

sequence() creates an array containing values from a specified start to stop, optionally using a defined step. It can work with numeric values and supported date or timestamp intervals. This is useful when generating calendar sequences, integer ranges within rows, or synthetic interval values. collect_list() aggregates existing row values into an array, monotonically_increasing_id() creates distributed unique identifiers, and range() is associated with generating DataFrames rather than constructing a per-row array expression. sequence() is therefore the correct higher-level SQL function for sequence arrays.

Question 268.

Which Spark SQL function can test whether every element of an array satisfies a Boolean condition?

1. transform()
   2. filter()
   3. exists()
   4. forall()

Correct Answer: 4. forall()

Explanation:

forall() is a higher-order function that evaluates a Boolean predicate against every element of an array and returns true only when the condition holds for all applicable elements. This is useful for validation tasks such as checking whether all scores are positive or all identifiers follow a required rule. exists() checks whether at least one element satisfies a condition, transform() changes elements, and filter() returns only matching elements. forall() provides a concise way to perform universal-condition checks directly on array data without exploding it into separate rows.

Question 269.

Which higher-order Spark SQL function returns true when at least one element of an array satisfies a specified condition?

1. exists()
   2. forall()
   3. aggregate()
   4. transform()

Correct Answer: 1. exists()

Explanation:

exists() evaluates a condition against array elements and returns true when at least one element satisfies the predicate. It is useful for membership-style checks that involve complex logic rather than a single literal value. forall() requires every element to satisfy the condition, aggregate() reduces array elements into one accumulated result, and transform() returns a modified array. exists() can help developers express nested-data logic efficiently without expanding arrays into multiple rows, which can make the transformation easier to understand and potentially reduce unnecessary intermediate data.

Question 270.

Which higher-order Spark SQL function changes each element of an array and returns a new array?

1. aggregate()
   2. transform()
   3. explode()
   4. collect_set()

Correct Answer: 2. transform()

Explanation:

transform() applies an expression or lambda-style function to every element of an array and returns the resulting values in a new array. For example, it can multiply every numeric array element by a constant or normalize each string element. aggregate() reduces an array into a single result, explode() creates rows from array elements, and collect_set() aggregates unique values from multiple input rows. transform() is especially useful for nested-data processing because it allows array elements to be modified without changing the number of DataFrame rows.

Question 271.

A developer wants to keep only positive values from an array column without creating additional DataFrame rows. Which higher-order function is appropriate?

1. explode()
   2. collect_list()
   3. filter()
   4. flatten()

Correct Answer: 3. filter()

Explanation:

The higher-order filter() function evaluates each array element against a Boolean condition and returns a new array containing only the elements that satisfy it. This is different from DataFrame.filter(), which removes complete rows. For example, filter can keep only values greater than zero inside an array while preserving the original DataFrame row. explode() would create additional rows, collect_list() aggregates values from multiple rows, and flatten() removes one level of nested arrays. Array filtering is useful when nested collections must remain nested but require element-level cleanup.

Question 272.

Which higher-order function is most appropriate for reducing all elements of an array into one value, such as a custom total?

1. transform()
   2. filter()
   3. explode()
   4. aggregate()

Correct Answer: 4. aggregate()

Explanation:

aggregate() processes an array using an accumulator and returns a single result. For example, a developer can define an initial value of zero and add each array element to create a custom sum. More complex logic can also be implemented, including structures or calculations that cannot be expressed with a simple built-in function. transform() returns an array, filter() returns selected elements, and explode() converts elements to separate rows. aggregate() is therefore appropriate when an array must be reduced while remaining within a single DataFrame row.

Question 273.

Which Spark SQL function can return the keys from a map column as an array?

1. map_keys()
   2. map_values()
   3. element_at()
   4. create_map()

Correct Answer: 1. map_keys()

Explanation:

map_keys() returns all keys from a map expression as an array. This can be useful when map keys need to be inspected, filtered, sorted, or expanded with explode(). map_values() returns the map’s values, element_at() retrieves an individual map value using a key, and create_map() constructs a map from expressions. Maps are frequently used for semi-structured or dynamic key-value data, and map_keys() allows developers to work directly with the set of available keys.

Question 274.

Which Spark SQL function returns the values stored in a map column as an array?

1. element_at()
   2. map_values()
   3. map_keys()
   4. explode()

Correct Answer: 2. map_values()

Explanation:

map_values() returns the values from a map expression as an array. It is useful when only the values need to be analyzed regardless of their associated keys. map_keys() provides the keys, element_at() retrieves a single entry, and explode() expands the map into multiple rows. After using map_values(), developers can apply array-oriented functions such as size(), filter(), transform(), or explode() depending on the required downstream logic. This makes map_values() a convenient bridge between map-based and array-based processing.

Question 275.

Which Spark SQL function can retrieve a value from a map using a key or an element from an array using a position?

1. array_position()
   2. map_keys()
   3. element_at()
   4. size()

Correct Answer: 3. element_at()

Explanation:

element_at() provides direct access to a value inside an array or map. For arrays, it retrieves an element according to the function’s positional semantics, while for maps it returns the value associated with a specified key. array_position() performs the reverse kind of lookup by identifying where a value appears in an array. map_keys() returns every map key, and size() returns the number of elements. element_at() is therefore useful when a developer knows exactly which nested entry is required and does not want to expand the complete collection.

Question 276.

Which Spark SQL function returns the number of elements in an array or entries in a map?

1. length()
   2. count()
   3. cardinality() only
   4. size()

Correct Answer: 4. size()

Explanation:

size() returns the number of elements in an array or the number of entries in a map. It is useful for validating nested structures, detecting empty collections, or filtering based on collection size. length() is generally used for strings or binary values, while count() is an aggregation across rows. Depending on Spark interfaces and versions, related collection-size functions may also exist, but size() is the standard DataFrame SQL function commonly used in Apache Spark code for arrays and maps.

Question 277.

Which Spark SQL function can merge two arrays and return only distinct elements?

1. array_union()
   2. concat()
   3. flatten()
   4. collect_set()

Correct Answer: 1. array_union()

Explanation:

array_union() returns an array containing the distinct elements appearing in either of the two input arrays. It applies set-like union semantics and eliminates duplicate elements in the result. concat() can append array contents but does not provide the same uniqueness behavior, flatten() removes nested-array levels, and collect_set() aggregates values across multiple rows rather than combining two array expressions within the same row. array_union() is useful when nested lists from different sources should be merged without retaining repeated values.

Question 278.

Which Spark SQL function returns only the distinct elements common to two arrays?

1. arrays_overlap()
   2. array_intersect()
   3. array_except()
   4. array_union()

Correct Answer: 2. array_intersect()

Explanation:

array_intersect() returns the distinct elements that exist in both input arrays. This is appropriate when developers need the shared memberships between two lists, such as common permissions, tags, products, or categories. arrays_overlap() only returns a Boolean indicating whether any shared element exists. array_except() returns values from the first array that do not appear in the second, while array_union() combines distinct elements from either array. array_intersect() is therefore the correct function for retrieving the actual common values.

Question 279.

Which Spark SQL function returns elements present in the first array but absent from the second?

1. array_union()
   2. array_intersect()
   3. array_except()
   4. arrays_overlap()

Correct Answer: 3. array_except()

Explanation:

array_except() implements a set-difference style operation for arrays by returning distinct values that occur in the first array but not the second. It is useful for identifying missing permissions, removed categories, unmatched IDs, or changed memberships. array_union() combines distinct values from both arrays, array_intersect() returns common values, and arrays_overlap() simply reports whether any overlap exists. array_except() provides a concise alternative to manually exploding arrays and performing anti-join logic when the comparison can remain at the nested-array level.

Question 280.

Which Spark SQL function returns a Boolean indicating whether two arrays share at least one common element?

1. array_intersect()
   2. array_contains()
   3. array_position()
   4. arrays_overlap()

Correct Answer: 4. arrays_overlap()

Explanation:

arrays_overlap() checks two arrays and returns a Boolean result indicating whether they contain at least one common non-null element. It is useful when developers only need an overlap test rather than the actual intersecting values. array_intersect() returns the common elements themselves, array_contains() tests whether one array contains a particular specified value, and array_position() returns the position of a target element. arrays_overlap() is therefore efficient and expressive for row-level comparisons involving two collection columns.