[SPARK-59018][SQL] Add variant_array_length expression

[WangGuangxin]

What changes were proposed in this pull request?

This PR adds a new variant_array_length expression that returns the number of elements in a variant array.

If the input is a SQL NULL, the function returns NULL. If the input variant is not an array, including variant null, object, or scalar values, the function returns NULL. If the input is an empty array, the
function returns 0.

Usage examples:

> SELECT variant_array_length(parse_json('[1,2,3]'));
 3

> SELECT variant_array_length(parse_json('[]'));
 0

> SELECT variant_array_length(parse_json('{"a":1}'));
 NULL

> SELECT variant_array_length(parse_json('null'));
 NULL

> SELECT variant_array_length(NULL);
 NULL

Why are the changes needed?

Spark already supports expanding variant arrays through variant_explode, but using a generator is unnecessarily cumbersome when users only need the array length. Users currently need to explode the value and
count rows, or cast the extracted value into a typed array when the element type is known.

variant_array_length provides a direct scalar operation for a common semi-structured data use case: inspecting array cardinality before filtering, validating, or transforming variant data. It is also cheaper and
easier to optimize than expanding the array when only the number of elements is needed, because the variant binary format stores array size in the array header.

This function also improves parity with Spark's JSON function surface, which already provides json_array_length for JSON strings. variant_array_length provides the corresponding operation for parsed variant
values.

Does this PR introduce any user-facing change?

Yes. This PR adds a new SQL function, variant_array_length.

Example:

SELECT variant_array_length(parse_json('[1,2,3]'));

returns:

3

How was this patch tested?

Unit tests were added for variant_array_length, including:

• variant array with multiple elements
• empty variant array
• SQL NULL input
• variant null input
• non-array variant inputs such as objects and scalars
• nested usage together with variant_get

Was this patch authored or co-authored using generative AI tooling?

Generated-by: TraeCode (GPT-5)

[WangGuangxin]

@bojana-db @uros-b Can you please help review this?

[uros-b]

@bojana-db @harshmotw-db Could you please help review this PR? Thank you @WangGuangxin!