Skip to content

timestamp in orderfilled.parquet does not match Polygon block timestamp #3

Description

@Yjejuy

Hi, thanks for open-sourcing this dataset.

I found what looks like a timestamp issue in the orderfilled.parquet files.

I compared the exported timestamp against Polygon block timestamps using both:

  • Polygon RPC eth_getBlockByNumber
  • Etherscan v2 eth_getBlockByNumber

For sampled fills, the two chain sources agree with each other, but the exported parquet timestamp differs.

From a larger 100-sample check on my side:

  • RPC timestamp == Etherscan timestamp: 100/100
  • Exported parquet timestamp == RPC/Etherscan: 0/100

So the exported timestamp does not appear to be the chain block.timestamp.

Here is a minimal reproduction script using a few reference fills:

import os
import time
import requests

RPC_URL = "https://polygon-bor-rpc.publicnode.com"
API_KEY = os.environ["POLYGONSCAN_API_KEY"]

SAMPLES = [
    {
        "fill_id": "0x876afb3e013514ba3091eafeff4c64c8bdaf9bc66d4e528d59c9a84b3a26b6a3_348",
        "tx_hash": "0x876afb3e013514ba3091eafeff4c64c8bdaf9bc66d4e528d59c9a84b3a26b6a3",
        "exported_timestamp": 1674519220,
    },
    {
        "fill_id": "0xf59d3f04414c2d291b4cb632f33645988fef636381ca56af802a2ad170b458b3_186",
        "tx_hash": "0xf59d3f04414c2d291b4cb632f33645988fef636381ca56af802a2ad170b458b3",
        "exported_timestamp": 1670544960,
    },
    {
        "fill_id": "0xe9a45356419702dc6a3298d4d4aa6338fe9ba7d50e8d77078f5f0c38aee98635_156",
        "tx_hash": "0xe9a45356419702dc6a3298d4d4aa6338fe9ba7d50e8d77078f5f0c38aee98635",
        "exported_timestamp": 1697846476,
    },
]

def rpc_call(method, params):
    payload = {"jsonrpc": "2.0", "id": 1, "method": method, "params": params}
    r = requests.post(RPC_URL, json=payload, timeout=30)
    r.raise_for_status()
    data = r.json()
    if "error" in data:
        raise RuntimeError(data["error"])
    return data["result"]

def get_block_timestamp_from_rpc(tx_hash):
    receipt = rpc_call("eth_getTransactionReceipt", [tx_hash])
    block_number = int(receipt["blockNumber"], 16)
    block = rpc_call("eth_getBlockByNumber", [hex(block_number), False])
    return block_number, int(block["timestamp"], 16)

def get_block_timestamp_from_etherscan(block_number):
    url = "https://api.etherscan.io/v2/api"
    params = {
        "chainid": "137",
        "module": "proxy",
        "action": "eth_getBlockByNumber",
        "tag": hex(block_number),
        "boolean": "false",
        "apikey": API_KEY,
    }
    r = requests.get(url, params=params, timeout=30)
    r.raise_for_status()
    data = r.json()
    if "result" not in data or data["result"] is None:
        raise RuntimeError(data)
    return int(data["result"]["timestamp"], 16)

for sample in SAMPLES:
    block_number, rpc_ts = get_block_timestamp_from_rpc(sample["tx_hash"])
    etherscan_ts = get_block_timestamp_from_etherscan(block_number)
    time.sleep(0.4)  # avoid free-tier rate limit

    print(
        sample["fill_id"],
        {
            "exported_timestamp": sample["exported_timestamp"],
            "rpc_block_ts": rpc_ts,
            "etherscan_block_ts": etherscan_ts,
            "exported_eq_rpc": sample["exported_timestamp"] == rpc_ts,
            "rpc_eq_etherscan": rpc_ts == etherscan_ts,
        },
    )

Example output from this script:

0x876afb3e013514ba3091eafeff4c64c8bdaf9bc66d4e528d59c9a84b3a26b6a3_348 {
  'exported_timestamp': 1674519220,
  'rpc_block_ts': 1674519246,
  'etherscan_block_ts': 1674519246,
  'exported_eq_rpc': False,
  'rpc_eq_etherscan': True
}

0xf59d3f04414c2d291b4cb632f33645988fef636381ca56af802a2ad170b458b3_186 {
  'exported_timestamp': 1670544960,
  'rpc_block_ts': 1670544922,
  'etherscan_block_ts': 1670544922,
  'exported_eq_rpc': False,
  'rpc_eq_etherscan': True
}

0xe9a45356419702dc6a3298d4d4aa6338fe9ba7d50e8d77078f5f0c38aee98635_156 {
  'exported_timestamp': 1697846476,
  'rpc_block_ts': 1697846488,
  'etherscan_block_ts': 1697846488,
  'exported_eq_rpc': False,
  'rpc_eq_etherscan': True
}

Could you confirm what the exported timestamp field is intended to represent?

  • If it is supposed to be Polygon block.timestamp, it seems incorrect.
  • If it represents another timestamp semantic, it would be helpful to document that explicitly.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions