[SPARK-59020][SQL] Support DSv2 Parquet shredded Variant predicate pushdown

[WangGuangxin]

What changes were proposed in this pull request?

This PR enables Parquet predicate pushdown for shredded Variant fields in the DSv2 scan path after #58050

The existing shredded Variant predicate pushdown support can translate filters on variant_get
expressions into filters on shredded Variant struct fields, so Parquet can use them for row-group
skipping. However, DSv2 regular filter pushdown happens before Variant extraction pushdown, so these
rewritten filters are not available during the normal pushDownFilters phase.

This PR adds a narrow internal pushdown path for this case:

• Adds an internal SupportsPushDownVariantPredicateFilters hook for scan builders.
• Rewrites remaining filters after Variant extraction pushdown and translates the rewritten filters
  with nested predicate pushdown enabled.
• Lets ParquetScanBuilder accept these Variant predicate filters only when both
  spark.sql.parquet.filterPushdown and
  spark.sql.variant.shreddedPredicatePushdown.enabled are enabled.
• Carries accepted filters through ParquetScan into ParquetPartitionReaderFactory.
• Combines regular Parquet filters with accepted Variant predicate filters only when the read schema
  contains shredded Variant metadata, and then reuses the existing ParquetFilters shredded Variant
  logic to build Parquet row-group predicates.
• Keeps the original Spark filters above the scan, so this is only a scan pruning optimization and
  does not change query results.

Why are the changes needed?

Without this change, DSv2 Parquet scans can push Variant extraction into the scan, but predicates
that become pushdown-able only after that rewrite are not passed to Parquet. As a result, queries
filtering on shredded Variant fields may still read row groups that Parquet could otherwise skip.

This is especially useful for Variant columns stored with shredding, where common paths are materialized
as typed Parquet fields. Pushing these predicates to Parquet lets Spark avoid unnecessary IO while
preserving the existing post-scan filter semantics.

Does this PR introduce any user-facing change?

No.

This only improves Parquet row-group skipping for eligible DSv2 scans when existing pushdown-related
configuration is enabled. Query results and visible SQL semantics are unchanged.

How was this patch tested?

Updated VariantShreddingFilterPushdownSuite to cover DSv2 shredded Variant predicate pushdown,
including row-group skipping assertions for vectorized reads and correctness coverage across DSv1/DSv2
and vectorized/non-vectorized readers.

[WangGuangxin]

@dongjoon-hyun @viirya @peter-toth @uros-b Can you help review this when you have time?

[uros-b]

Seems like a correct and well-contained implementation. The new private[v2] trait keeps the hook internal, the original filters remain above the scan to guarantee result correctness, and the ParquetFilters variant-resolution logic from SPARK-55817 is reused without modification. The test update extends the row-group-skip assertion to cover DSv2 with the vectorized reader, directly validating the new behavior while preserving full correctness coverage across all four reader combinations.

Adding @viirya to take a better look here ^^