Enable fetching scores at runtime again

[JensAc]

Hey again 🙂 ,

In v3.2.0 fetching scores at runtime seems to be broken due to the change in flushing integer_motion features. It appears to me that motionx features are not computed in extract() anymore but in flush once all pictures are read. However, here it is still stated that vmaf_score_at_index(..) could be called to fetch a score once pictures are read.

I suppose the current behavior was not intended and this PR provides a "hotfix". I know, it might not be the most elegant solution to poll for the sad feature to be available in a loop, but at least this works in local verification tests on my machine.

If there is anything I could/should do to verify the behavior or on the implementation side, please let me know. I will do my best to make this ready for the patch release you announced here.

[kylophone]

One of the main motivators for the motion refactor was it improved the multithreading performance significantly. See the graph data in the original PR here. I didn't test it yet, but I think your changes probably re-introduce a cross-frame dependency that limits the threading throughput.

I agree that this is a problem we need to fix, however I think we need to make sure we are not hurting the multithreading performance. I have some ideas on how to do this, but I am curious to know what you think?

[JensAc]

Yes, I agree. These changes introduce a cross-frame dependency and block threads for some time in case the sad feature is not available (see here). This is why I called it a "hotfix". Performance wise, however, this is not very different from computing the motionx features when flushing, as they are calculated sequentially either, aren't they?

When I measure the performance impact of these changes, the runtime difference between v3.2.0 and this branch for a 1080p sequence of 2100 frames is minor. I measure on a AMD Ryzen 9 9950X 16-Core Processor with the command

 time ./libvmaf/build/tools/vmaf -r ... -d .. -w 1920 -h 1080 -p 420 -b 8 -m version=vmaf_v0.6.1neg --json -o vmaf.json --threads 32 

and get

this branch:  62.14s user 8.66s system 891% cpu 7.938 total --> 264.6 fps
v3.2.0:       59.70s user 8.48s system 867% cpu 7.861 total --> 267.1 fps

Don't get me wrong, if you have ideas to solve this is a better way, let's do that. I am happy to help, but would need some more insight into your ideas. I just read your comment about an upcoming patch release and wanted to make sure that this is on the radar.