[testing] use nbval instead of nbmake for notebook testing

[richrines1]

No description provided.

[pgoiporia]

Any reason we abandoned this? Seems like nbval has 2x the stars that nbmake does. I'm happy to pick this up.

[richrines1]

Any reason we abandoned this? Seems like nbval has 2x the stars that nbmake does. I'm happy to pick this up.

iirc when i stopped nbval was failing due to a missing/invalid execution_count in a couple notebook cells. This should be super easy to correct though - it should just require rerunning notebooks or manually editing the notebook json (but instead i think i went down a rabbit hole of auto-standardizing execution counts/notebook metadata lol, which i still think would be nice but definitely isn't necessary for this pr)

[natibek]

Both are pretty similar. Both allow for parallel testing with pytest-xdist, and specifying kernels to run the tests in. nbval does not test the cells that have not been executed, but nbmake does. nbval uses comments in the code cells to tell it to ignore the cell (or test it if using --nbval-lax) or by writing tags in the metadata of the cell. The same can be done with nbmake, but you need to write tags in the metadata of the cell.

nbval runs the code from the notebook by directly connecting to the kernel with jupyter_client, then sending the code cells to be executed. nbmake uses NotebookClient to run notebooks without having to worry about connecting to the kernel and handling outputs. nbval does show test failures with nbdime, which seems to be a bit nicer. Both have timeouts as well.

Neither has random seeding for tests, which would be useful in cases where some cells start throwing errors like the ACES notebook did (that took a while to debug).

The biggest difference seems to be that you can use nbval to also compare the outputs of the test run and the stored outputs in the notebook as well. What's pretty cool about nbmake (even though we don't have a use case for it) is that you can mock cells with it.