View Full Databricks Certified Data Engineer Professional Exam Dumps and Practice Test Dumps

 

Question 341. Which Spark SQL function splits a string into an array using a specified delimiter?

1) split()

2) string_split()

3) split_string()

4) tokenize()

Answer: 1) split()

Explanation:

The split() function divides a string into an array based on a specified regular-expression delimiter. It is useful when source data contains multiple values inside one textual field, such as comma-separated categories or pipe-delimited identifiers. Once the string has been converted into an array, functions such as explode() can process individual elements. Data engineers should ensure that the delimiter is correctly defined because split() interprets its pattern as a regular expression. Proper parsing at ingestion can make downstream transformations easier and more reliable.

Question 342. Which Spark SQL function removes duplicate elements from an array?

1) array_unique()

2) array_distinct()

3) distinct_array()

4) unique_elements()

Answer: 2) array_distinct()

Explanation:

The array_distinct() function removes duplicate values from an array while retaining the unique elements. It is useful when source records contain repeated categories, identifiers, or attributes inside array columns. This operation works at the individual-row level rather than removing duplicate rows from an entire DataFrame. That distinction is important because dropDuplicates() addresses row-level duplication across records, while array_distinct() operates on the contents of an array. Data engineers can use it before further array processing or aggregation to simplify nested data.

Question 343. Which Spark SQL function returns the number of elements contained in an array?

1) array_length()

2) size()

3) count_array()

4) elements()

Answer: 2) size()

Explanation:

The size() function returns the number of elements in an array or the number of entries in a map. It is useful when data engineers need to validate nested data, filter records according to collection size, or create derived metrics. For example, a pipeline can identify records containing more than a certain number of products in an array. Because size() operates directly on collection types, it avoids the need to explode arrays solely to count their elements, which can help keep transformations simpler and more efficient.

Question 344. Which Spark SQL function checks whether an array contains a specified value?

1) contains()

2) array_contains()

3) contains_array()

4) has_element()

Answer: 2) array_contains()

Explanation:

The array_contains() function checks whether an array contains a specified element. It is useful when filtering records based on membership within nested collections. For example, a customer record containing an array of subscribed services can be filtered for customers whose array includes a particular service. This approach avoids exploding the array when only a membership check is required. Data engineers should also account for null values and the data type of the searched value to ensure the expression behaves as intended.

Question 345. Which Spark SQL function returns the keys contained in a map?

1) map_keys()

2) keys()

3) get_map_keys()

4) map_entries_keys()

Answer: 1) map_keys()

Explanation:

The map_keys() function extracts the keys from a map and returns them as an array. This is useful when nested data stores attributes as key-value pairs and the pipeline needs to inspect or transform the available keys. For example, a data engineer can use map_keys() to identify which attributes are present before applying additional logic. The resulting array can then be processed with other array functions. This provides a convenient way to work with semi-structured map data without manually parsing serialized text.

Question 346. Which Spark SQL function returns the values contained in a map?

1) map_values()

2) values()

3) get_map_values()

4) map_entries_values()

Answer: 1) map_values()

Explanation:

The map_values() function extracts the values from a map and returns them as an array. It is useful when a data pipeline needs to process the contents of a key-value collection independently from its keys. For example, a transformation can retrieve all attribute values and then apply array functions such as size() or explode(). Separating keys and values can simplify processing of semi-structured records. Data engineers should preserve the relationship between keys and values when that relationship is required for downstream business logic.

Question 347. Which Spark SQL function converts a map into an array of key-value entries?

1) map_entries()

2) entries_map()

3) map_to_array()

4) explode_map()

Answer: 1) map_entries()

Explanation:

The map_entries() function converts a map into an array of key-value structures. This representation is useful when a data engineer needs to process map elements using array-oriented functions. For example, the resulting array can be exploded so that each key-value pair becomes a separate row. This is particularly useful when transforming nested semi-structured data into a more relational form. Using a built-in function also keeps the transformation inside Spark’s execution plan instead of requiring custom parsing logic.

Question 348. Which Spark SQL function constructs a map from alternating key and value expressions?

1) map()

2) create_map()

3) build_map()

4) make_map()

**Answer: 1) map()

Explanation:

The map() function constructs a map from alternating key and value expressions. It is useful when a transformation needs to create key-value structures directly from existing columns or literals. For example, a pipeline can combine an attribute name with its corresponding value to produce a nested map. The keys and values must follow compatible data-type requirements. Data engineers can use this function when producing structured output for downstream processing, especially when working with semi-structured data formats that naturally represent attributes as key-value pairs.

Question 349. Which Spark SQL function creates a struct from multiple expressions?

1) struct()

2) make_struct()

3) row_struct()

4) create_record()

**Answer: 1) struct()

Explanation:

The struct() function creates a structured value containing multiple fields. It is useful for grouping related columns into a nested object, creating complex schemas, or preparing data for operations involving nested structures. For example, a data engineer can combine customer ID, name, and status into one structured column. Structs are especially useful when working with formats such as JSON or nested Parquet data. Using struct() allows Spark to preserve field names and data types rather than representing the combined information as an unstructured string.

Question 350. Which Spark SQL function creates an array from multiple expressions?

1) array()

2) make_array()

3) create_array()

4) array_create()

**Answer: 1) array()

Explanation:

The array() function creates an array from multiple expressions. It is useful when a data engineer needs to combine several values into a collection within each row. For example, multiple related product or status columns can be represented as one array before applying functions such as array_distinct(), size(), or explode(). The expressions should have compatible data types so that Spark can establish an appropriate array schema. This function is particularly useful when transforming relational columns into nested or semi-structured representations.

Question 351. Which Spark SQL function retrieves an element from an array or map using a specified index or key?

1) element_at()

2) get_element()

3) lookup_element()

4) fetch_value()

**Answer: 1) element_at()

Explanation:

The element_at() function retrieves a value from an array or map. For arrays, it uses an index, while for maps it uses a key. This makes it useful for extracting specific values from nested data without expanding the entire collection. Data engineers should pay attention to array indexing behavior and ensure that the requested position is valid. For map data, the requested key must match the map’s key type. Using element_at() can simplify nested-data transformations when only one particular element is required.

Question 352. Which Spark SQL function converts an array or struct into a JSON string representation?

1) to_json()

2) json_encode()

3) serialize_json()

4) make_json()

**Answer: 1) to_json()

Explanation:

The to_json() function converts supported complex data types such as structs, arrays, and maps into JSON strings. It is useful when preparing structured Spark data for APIs, message systems, external applications, or storage formats that require JSON text. The function preserves the nested structure rather than simply concatenating values into a flat string. Data engineers should consider whether JSON serialization is actually required because converting structured data to text can make subsequent analytical processing more difficult.

Question 353. Which Spark SQL function parses a JSON string into a structured value using a specified schema?

1) from_json()

2) parse_json_string()

3) json_decode()

4) read_json()

**Answer: 1) from_json()

Explanation:

The from_json() function parses a JSON string according to a supplied schema and returns a structured Spark value. It is particularly useful for processing semi-structured event data stored in string columns. Once parsed, individual fields can be accessed using standard column expressions rather than repeatedly applying string operations. Providing an explicit schema also makes the expected structure clear and can improve reliability. Data engineers should define how malformed JSON should be handled so invalid records do not silently produce unexpected downstream results.

Question 354. Which Spark SQL function converts a timestamp into Unix epoch seconds?

1) unix_timestamp()

2) epoch_seconds()

3) timestamp_epoch()

4) to_epoch()

**Answer: 1) unix_timestamp()

Explanation:

The unix_timestamp() function can convert timestamp information into Unix epoch seconds. This representation is commonly used when integrating with systems that store time as an integer rather than a native timestamp. Data engineers must verify whether the source or destination expects seconds, milliseconds, or another unit because an incorrect assumption can produce dates far from the intended value. Epoch representations can be convenient for interoperability, but native timestamp types are generally easier to work with for analytical date and time operations.

Question 355. Which Spark SQL function truncates a date or timestamp to a specified unit such as month or year?

1) date_trunc()

2) truncate_date()

3) date_round()

4) timestamp_floor()

**Answer: 1) date_trunc()

Explanation:

The date_trunc() function truncates a timestamp to a specified time unit, such as year, month, day, hour, or minute. It is useful for creating consistent reporting boundaries and grouping events into standardized periods. For example, truncating timestamps to the month allows records to be associated with a common month-start timestamp. Data engineers should distinguish truncation from rounding because truncation moves the value to the beginning of the requested period. This function is particularly useful in time-based analytical transformations and aggregations.

Question 356. Which Spark SQL function returns the year component of a date or timestamp?

1) year()

2) get_year()

3) date_year()

4) extract_year()

**Answer: 1) year()

Explanation:

The year() function extracts the year component from a date or timestamp. It is useful when creating reporting dimensions, filtering records by calendar year, or deriving year-based analytical attributes. Because it operates directly on date and timestamp types, it avoids manual string parsing. Data engineers can combine year() with other date functions to build calendar attributes or partition-related logic. When fiscal years differ from calendar years, a dedicated calendar dimension or explicit business rule may be more appropriate than relying solely on the extracted year.

Question 357. Which Spark SQL function returns the month component of a date or timestamp?

1) month()

2) get_month()

3) date_month()

4) extract_month()

**Answer: 1) month()

Explanation:

The month() function extracts the numeric month component from a date or timestamp. It can be used for monthly reporting, filtering, grouping, and creating derived calendar attributes. The returned value represents the calendar month rather than a textual month name. Data engineers can combine it with year() when creating a year-month reporting key to avoid ambiguity across different years. For fiscal calendars or custom reporting periods, additional calendar logic may be necessary because the calendar month does not always correspond to the organization’s fiscal month.

Question 358. Which Spark SQL function returns the day of the month from a date?

1) dayofmonth()

2) month_day()

3) date_day()

4) day_number()

**Answer: 1) dayofmonth()

Explanation:

The dayofmonth() function extracts the day number within the month from a date or timestamp. For example, it can be used to identify records occurring on the first day, middle days, or final days of a month. This is different from dayofweek(), which identifies the weekday associated with the date. Data engineers can use dayofmonth() when implementing calendar-based filters or creating date dimensions. Keeping the original date column alongside the derived value is often useful for maintaining flexibility in downstream transformations.

Question 359. Which Spark SQL function returns the quarter of the year for a date?

1) quarter()

2) year_quarter()

3) quarter_of_year()

4) date_quarter()

**Answer: 1) quarter()

Explanation:

The quarter() function returns the calendar quarter associated with a date or timestamp. The result identifies whether the date belongs to the first, second, third, or fourth quarter of the calendar year. It is useful for quarterly reporting, aggregations, and deriving analytical dimensions. Data engineers should remember that calendar quarters may not match an organization’s fiscal quarters. When fiscal reporting follows a different schedule, a date dimension containing fiscal-period attributes is generally more suitable than relying directly on quarter().

Question 360. Which Spark SQL function returns the day of the year for a date?

1) dayofyear()

2) year_day()

3) date_dayofyear()

4) ordinal_day()

**Answer: 1) dayofyear()

Explanation:

The dayofyear() function returns the numeric day within the calendar year for a given date. It can be useful for seasonal analysis, year-progress calculations, and transformations that require an ordinal calendar position. The result ranges according to the number of days in the relevant year, so leap years must be considered when interpreting the value. Data engineers can combine this function with year-based attributes when comparing seasonal patterns across multiple years or building calendar-related analytical features.