Skip to content

Tableau: unstable pagination, fatal RATE_EXCEEDED, and silent truncation on large Tableau Cloud instances #19689

Description

@rospe

Describe the bug

Several defects in the Tableau connector surface on large Tableau Cloud instances (thousands of datasources, frequent catalog changes):

  1. Unstable cursor pagination — paginated Metadata API queries have no orderBy, so when the catalog changes mid-ingestion, items shift between pages and entities are skipped or duplicated.
  2. RATE_EXCEEDED is fatal — when Tableau Cloud throttles queries (cost budget exhausted), the RATE_EXCEEDED GraphQL error raises a fatal RuntimeError and crashes the pipeline, instead of being retried like timeouts.
  3. NullValueInNonNullableField crashes ingestion — when the 20k node limit truncates a response, nested objects (e.g. workbook.owner) come back null. This error lacks severity: WARNING, so the existing non-fatal handling doesn't catch it and ingestion crashes, even though the partial data is usable.
  4. Field upstream page size silently truncated — page_size * 10 (e.g. 2000 for page_size: 200) exceeds Tableau's fieldsConnection limit of 1000, which silently truncates results with no error, producing incomplete field-level lineage.

To Reproduce

Run the Tableau connector against a large Tableau Cloud instance with page_size: 200, ongoing catalog changes, and datasources large enough to hit the 20k node limit. Observe skipped/duplicated entities, pipeline crashes on throttling or null fields, and incomplete field upstreams.

Expected behavior

Ingestion should paginate stably, retry on RATE_EXCEEDED, tolerate NullValueInNonNullableField partial data, and not silently truncate field upstream results.

Version

DataHub ingestion (Tableau source), current master.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBug report

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions