Skip to content

Metadata/data validation should produce a warning for an unused datetime_format entry #2930

Description

@npatki

Problem Description

For datetime columns, the metadata specification allows me to input a datetime_format string that specifies exactly how my datetimes are to be formatted. For example, '21 Jul 2026' vs '21-07-2026'.

If the data is represented as a dtype object (containing string values), then SDV can explicitly validate that the provided format matches the data.

However if for some reason the data is represented as a datetime64, then there is no format really associated with those values. So in this case, SDV does not explicitly validate whether the data matches any explicitly provided format string. But as a user, I don't know that this validation is skipped.

Expected behavior

The metadata.validate_data functions should raise a warning if:

  • The metadata has a specified datetime_format string for a particular column AND
  • The data is represented as a dtype object (or any other dtype where the format cannot be checked)

The warning should specifically indicate that the datetime format cannot be verified and will be skipped.

Warning: The datetime format for column 'checkin_date' (table 'guests') could not be verified because the data is represented as dtype 'datetime64[ns]'.
Please omit the datetime format string from the metadata or cast the data to strings with the right format.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions