View Full Databricks Certified Associate Developer for Apache Spark Exam Dumps  and Practice Test Dumps

 

Question 121.

Which Spark SQL function checks whether two arrays have at least one element in common?

1. arrays_overlap()
   2. array_union()
   3. array_except()
   4. flatten()

Correct Answer: 1. arrays_overlap()

Explanation:

arrays_overlap() returns a Boolean result indicating whether two arrays share at least one common non-null element. It is useful when comparing tag lists, category memberships, permissions, or other array-based attributes. array_union() combines distinct elements from two arrays, array_except() returns elements present in the first array but not the second, and flatten() removes one level of nested arrays. arrays_overlap() is therefore appropriate when the goal is simply to test for shared membership.

Question 122.

Which Spark SQL function returns elements that exist in the first array but not in the second array?

1. array_intersect()
   2. array_except()
   3. array_union()
   4. arrays_overlap()

Correct Answer: 2. array_except()

Explanation:

array_except() returns an array containing elements from the first input array that do not appear in the second. It is useful for comparing lists and identifying missing or unique elements. array_intersect() returns common elements, array_union() combines unique elements from both arrays, and arrays_overlap() returns only a Boolean result indicating whether the arrays share anything. array_except() is therefore the correct function for set-style subtraction between arrays.

Question 123.

Which Spark SQL function returns only elements that appear in both input arrays?

1. array_union()
   2. array_except()
   3. array_intersect()
   4. flatten()

Correct Answer: 3. array_intersect()

Explanation:

array_intersect() returns the distinct elements that are present in both input arrays. It is useful for comparing categories, permissions, tags, or other list-based values. array_union() returns distinct elements from either array, array_except() returns values unique to the first array, and flatten() reduces nested arrays by one level. array_intersect() therefore performs the set intersection operation for Spark array columns.

Question 124.

Which Spark SQL function checks whether an array contains a specific value?

1. size()
   2. array_position()
   3. arrays_overlap()
   4. array_contains()

Correct Answer: 4. array_contains()

Explanation:

array_contains() returns a Boolean indicating whether the specified value exists within an array column. It is useful for filtering rows based on membership in array-valued fields. size() returns the number of elements, array_position() returns the position of an element, and arrays_overlap() compares two arrays for any shared elements. array_contains() is therefore the most direct function when testing whether one particular value appears in an array.

Question 125.

Which Spark SQL function returns the position of a specified element within an array?

1. array_position()
   2. element_at()
   3. size()
   4. sort_array()

Correct Answer: 1. array_position()

Explanation:

array_position() returns the position of the first occurrence of a specified value within an array. It is useful when the location of an element is important for downstream logic. element_at() retrieves an element by position or map key, size() returns the number of elements, and sort_array() orders array elements. array_position() is therefore the correct function for locating an element’s index-like position within an array.

Question 126.

Which function can return a specific element from an array or a value from a map?

1. map_keys()
   2. element_at()
   3. explode()
   4. array_position()

Correct Answer: 2. element_at()

Explanation:

element_at() retrieves a value from an array by position or from a map by key. This makes it useful for accessing nested collection values without exploding the entire structure. map_keys() returns all keys from a map, explode() expands arrays or maps into rows, and array_position() finds where an array value occurs. element_at() is the appropriate function when one specific nested value is required.

Question 127.

Which Spark SQL function returns all keys from a map column as an array?

1. map_values()
   2. element_at()
   3. map_keys()
   4. create_map()

Correct Answer: 3. map_keys()

Explanation:

map_keys() returns an array containing the keys from a map column. It is useful when map keys themselves need to be inspected, filtered, or exploded. map_values() returns the values, element_at() retrieves a value associated with a particular key, and create_map() constructs a map. map_keys() is therefore the direct choice for extracting the complete set of keys from a map expression.

Question 128.

Which Spark SQL function returns all values from a map column as an array?

1. map_keys()
   2. element_at()
   3. create_map()
   4. map_values()

Correct Answer: 4. map_values()

Explanation:

map_values() returns an array containing all values from a map expression. It is useful when the values need to be analyzed independently of their associated keys. map_keys() returns keys, element_at() retrieves one value by key, and create_map() builds a map. map_values() provides a convenient way to transform map contents into an array for further Spark processing.

Question 129.

Which Spark SQL function can sort the elements of an array?

1. sort_array()
   2. array_union()
   3. array_except()
   4. collect_list()

Correct Answer: 1. sort_array()

Explanation:

sort_array() sorts elements within an array, typically in ascending order by default, with options for ordering behavior. It is useful when deterministic array ordering is required before comparison, output, or further transformation. array_union() combines arrays, array_except() performs set subtraction, and collect_list() aggregates row values into an array. sort_array() is therefore appropriate when ordering array contents is the primary goal.

Question 130.

Which higher-order Spark SQL function applies an expression to every element of an array?

1. filter()
   2. transform()
   3. aggregate()
   4. explode()

Correct Answer: 2. transform()

Explanation:

transform() is a higher-order function that applies a provided expression or lambda-style function to each element of an array and returns a new array. It is useful for modifying nested data without exploding it into separate rows. filter() selects array elements based on a condition, aggregate() reduces array elements to a single result, and explode() creates multiple rows. transform() is therefore appropriate when every array element needs to be individually changed.

Question 131.

Which higher-order Spark SQL function keeps only array elements that satisfy a specified condition?

1. transform()
   2. aggregate()
   3. filter()
   4. array_contains()

Correct Answer: 3. filter()

Explanation:

The higher-order filter() function evaluates each element of an array against a condition and returns a new array containing only elements that satisfy that condition. It differs from DataFrame.filter(), which filters rows. transform() modifies each array element, aggregate() reduces array contents, and array_contains() checks for one specific value. Understanding the context is important because Spark uses the term filter for both row-level and array-level operations.

Question 132.

Which higher-order function can reduce all elements of an array into a single accumulated result?

1. transform()
   2. explode()
   3. filter()
   4. aggregate()

Correct Answer: 4. aggregate()

Explanation:

The higher-order aggregate() function processes array elements using an accumulator and returns a single result. It can be used for custom sums, concatenations, or more complex reductions without exploding the array. transform() returns a modified array, filter() returns a subset of array elements, and explode() expands elements into rows. aggregate() is appropriate when the contents of an array must be reduced to one derived value.

Question 133.

Which DataFrame operation can remove duplicate rows based only on selected columns?

1. dropDuplicates([“col1”, “col2”])
   2. distinct()
   3. drop()
   4. filter()

Correct Answer: 1. dropDuplicates([“col1”, “col2”])

Explanation:

dropDuplicates() can accept a subset of columns used to determine whether rows are duplicates. This allows Spark to preserve one row for each unique combination of those selected columns even if other column values differ. distinct() considers the entire row, drop() removes columns, and filter() selects rows based on a condition. dropDuplicates() is therefore useful for key-based deduplication.

Question 134.

Which join type returns rows from the left DataFrame that have a match in the right DataFrame but returns only left-side columns?

1. Inner join
   2. Left semi join
   3. Left anti join
   4. Left outer join

Correct Answer: 2. Left semi join

Explanation:

A left semi join returns rows from the left DataFrame for which a matching right-side row exists, but it includes only columns from the left side. This makes it useful for existence checks. An inner join can return columns from both sides, a left anti join returns unmatched left rows, and a left outer join preserves all left rows. Left semi joins are efficient when the right dataset is needed only to determine whether a match exists.

Question 135.

Which DataFrame method can be used to randomly sample rows from a dataset?

1. limit()
   2. take()
   3. sample()
   4. collect()

Correct Answer: 3. sample()

Explanation:

sample() returns a sampled subset of a DataFrame using parameters such as fraction, replacement behavior, and an optional seed. It is useful for testing, exploration, and approximate analysis on large datasets. limit() returns a bounded set of rows but does not provide random sampling semantics, take() returns rows to the driver, and collect() returns the entire result. sample() is therefore the appropriate operation for probabilistic row selection.

Question 136.

Which parameter can make repeated sampling operations reproducible?

1. fraction
   2. replacement
   3. partitions
   4. seed

Correct Answer: 4. seed

Explanation:

A seed initializes the pseudo-random number generator used during sampling. Providing the same seed and sampling configuration helps produce reproducible results, which is valuable for testing and debugging. fraction controls the approximate proportion of rows sampled, while replacement determines whether a row may be selected more than once. Partition count is not the parameter used to directly control random reproducibility.

Question 137.

Which DataFrame method randomly divides a dataset into multiple DataFrames using specified weights?

1. randomSplit()
   2. sample()
   3. repartition()
   4. union()

Correct Answer: 1. randomSplit()

Explanation:

randomSplit() divides a DataFrame into multiple DataFrames according to relative weights and can accept a seed for reproducibility. It is commonly used to create training, validation, and test datasets. sample() returns one sampled dataset, repartition() changes physical partitioning, and union() combines datasets. randomSplit() is therefore the appropriate method when a dataset needs to be randomly separated into multiple subsets.

Question 138.

Which DataFrame method is most appropriate for checking whether a DataFrame is cached?

1. storageLevel
   2. is_cached
   3. cacheStatus
   4. persisted

Correct Answer: 2. is_cached

Explanation:

is_cached indicates whether a DataFrame has been marked for caching. It can be useful when debugging persistence behavior or checking whether an expensive intermediate result has already been cached. storageLevel provides more detailed information about how the data is persisted, while cacheStatus and persisted are not the standard DataFrame properties for this purpose. is_cached therefore provides a simple Boolean check for caching state.

Question 139.

Which DataFrame property provides information about the current persistence configuration?

1. schema
   2. columns
   3. storageLevel
   4. dtypes

Correct Answer: 3. storageLevel

Explanation:

storageLevel describes how a DataFrame is persisted, including whether data is stored in memory, on disk, serialized, or replicated depending on configuration. schema describes column structure, columns returns names, and dtypes returns column-type pairs. storageLevel is therefore the relevant property when examining the persistence configuration of a cached or persisted DataFrame.

Question 140.

Which method is commonly used to truncate a long lineage by materializing a DataFrame to checkpoint storage?

1. persist()
   2. cache()
   3. unpersist()
   4. checkpoint()

Correct Answer: 4. checkpoint()

Explanation:

checkpoint() materializes a DataFrame to checkpoint storage and truncates its lineage. This can be useful when execution plans become very long or when iterative workloads build increasingly complex dependency chains. cache() and persist() retain computed data but do not necessarily cut lineage in the same way, while unpersist() removes stored data. checkpoint() is therefore useful for controlling lineage complexity when appropriate checkpoint storage has been configured.