Fix three-valued NULL semantics for tuple IN (SELECT ...)

[vishnujayvel]

Closes the remaining IN (SELECT ...) NULL stragglers left open by #3651.

Related lineage: dolthub/dolt#11024 (tuple/NULL equality); our earlier draft #3640 was superseded by #3651 and is not reused here.

Context

#3651 (thanks @nicktobey) fixed NullUnsafe tuple comparisons for equality and list-form IN, and wired NullUnsafeCompareTuples into the hash-hit path of InSubquery.Eval. The PR body notes:

This PR fixes most of the incorrect comparisons. Some stragglers remain involving WHERE x IN (SELECT ...) subqueries.

Projection-form InSubquery still returned definite FALSE in several three-valued cases.

Problem

For row constructors, SQL membership is three-valued:

Case	Before	MySQL / correct
(1, 5) IN (SELECT 1, NULL …)	FALSE	NULL
multi-row with one partial-NULL match	FALSE	NULL
(1, NULL) IN (SELECT 1, 2 …)	FALSE	NULL
(NULL, NULL) IN (SELECT 1, 2 …)	FALSE	NULL
(1, 5) NOT IN (SELECT 1, NULL …)	TRUE	NULL
(1, NULL) IN (SELECT 2, NULL …)	FALSE	FALSE (control)

Two gaps in sql/plan/insubquery.go:

1. Hash miss only probed whole-row NULL via nilKey. Partial-NULL right rows (and lefts with NULL components) never hash-hit, so membership fell through to definite FALSE.
2. Convert gate — SELECT 1, NULL types the right as tuple(tinyint, null). Converting a non-NULL left component into a null-typed column fails (ErrValueNotNil), and the old code mapped any Convert error to FALSE.

WHERE forms are largely OK post-#3651 (unnest to join); residual bugs were in expression-form InSubquery.Eval.

Fix

Minimal continuation of #3651 idioms:

• On hash miss for TupleType, scan subquery rows with NullUnsafeCompareTuples (TRUE wins; else any hasNil && cmp == 0 → NULL; else FALSE).
• On Convert-to-right failure for tuples, use the same scan with the unconverted left instead of returning FALSE.
• NOT IN is Not(InSubquery) — NULL falls out naturally.

No planner redesign; no dead guards; comments match house density.

Performance note (tuple hash-miss path)

On the tuple hash-miss / convert-failure path, evalTupleInSubquery calls Subquery.EvalMultiple once per outer Eval (sql/plan/insubquery.go). When canCacheResults() is true (correlated.Empty() && !volatile, sql/plan/subquery.go), the result cache that HashMultiple already populated is shared, so EvalMultiple returns the cached slice — uncorrelated dual/constant subqueries (all the cases fixed here) pay a linear scan of cached rows, not a re-execution.

When canCacheResults() is false (correlated or volatile subqueries), results are not cacheable, so on the tuple miss path the subquery now executes twice per outer row — once for HashMultiple's hash build and once for the scan — versus once before this change. Scalar / non-tuple paths are unchanged. If that cost matters for correlated tuple IN, a follow-up could precompute a has-partial-NULL-rows flag (or retain the row slice) at hash build so the miss path avoids re-executing; left as the maintainer's call.

Tests

Promoted the two BrokenQueries counterexamples into QueryTests (main var QueryTests, adjacent to existing scalar NULL IN (SELECT …) cases):

• (1, 5) IN (SELECT 1, NULL FROM dual) → NULL
• (1, 5) IN (SELECT * FROM (SELECT 1, NULL … UNION ALL SELECT 2, 3 …) t) → NULL

and added five new cases:

• (1, NULL) IN (SELECT 1, 2 FROM dual) → NULL
• (NULL, NULL) IN (SELECT 1, 2 FROM dual) → NULL
• (1, 5) NOT IN (SELECT 1, NULL FROM dual) → NULL
• (1, NULL) IN (SELECT 2, NULL FROM dual) → FALSE (definite-mismatch control)
• (1, 5) IN (SELECT 1, 5 FROM dual) → TRUE (exact-match baseline)

(7 entries total; KeylessQueries left byte-identical to pin.)

Verification

Pin: 1fed79c2dd1c7efd12c5e329818b3ab911057542 (upstream/main, contains #3651)

CGO_ENABLED=0 go test -tags gms_pure_go ./sql/plan/ ./sql/expression/ ./sql/rowexec/ -count=1
# exit 0 (plan: no test files; expression + rowexec ok)

CGO_ENABLED=0 go test -tags gms_pure_go ./enginetest/ -count=1 -timeout 15m -v \
  -run 'TestQueriesSimple/SELECT_\(1,_5\)_IN_\(SELECT_1,_NULL_FROM_dual\)|TestQueriesSimple/SELECT_\(1,_5\)_IN_\(SELECT_\*_FROM_\(SELECT_1,_NULL|TestQueriesSimple/SELECT_\(1,_NULL\)_IN_\(SELECT_1,_2_FROM_dual\)|TestQueriesSimple/SELECT_\(NULL,_NULL\)_IN_\(SELECT_1,_2_FROM_dual\)|TestQueriesSimple/SELECT_\(1,_5\)_NOT_IN_\(SELECT_1,_NULL_FROM_dual\)|TestQueriesSimple/SELECT_\(1,_NULL\)_IN_\(SELECT_2,_NULL_FROM_dual\)|TestQueriesSimple/SELECT_\(1,_5\)_IN_\(SELECT_1,_5_FROM_dual\)'
# exit 0; 7 === RUN TestQueriesSimple/... lines (all PASS):
#   SELECT_(1,_5)_IN_(SELECT_1,_NULL_FROM_dual)
#   SELECT_(1,_5)_IN_(SELECT_*_FROM_(SELECT_1,_NULL_FROM_dual_UNION_ALL_SELECT_2,_3_FROM_dual)_t)
#   SELECT_(1,_NULL)_IN_(SELECT_1,_2_FROM_dual)
#   SELECT_(NULL,_NULL)_IN_(SELECT_1,_2_FROM_dual)
#   SELECT_(1,_5)_NOT_IN_(SELECT_1,_NULL_FROM_dual)
#   SELECT_(1,_NULL)_IN_(SELECT_2,_NULL_FROM_dual)
#   SELECT_(1,_5)_IN_(SELECT_1,_5_FROM_dual)

gofmt -l sql/plan/insubquery.go enginetest/queries/queries.go  # empty

Full ./enginetest/ run end-to-end at base and at this branch (-count=1): identical results — the only failure on both sides is the pre-existing TestRegex (42 subtests, environment-dependent regex expectations; the sorted failing-subtest name sets are identical between base and branch), with zero new failures and zero disappeared. No intentional unrelated changes.

Files

• sql/plan/insubquery.go — Eval + evalTupleInSubquery
• enginetest/queries/queries.go — promote/add QueryTests