Update TensorFlow ?

[fwyzard]

Hi @smuzaffar @akritkbehera

@kopperp has been looking at the recent versions of Eigen: they seem to have much better support for GPUs, and may not need our patches any more (see for example #10712).

However TensorFlow does not seem to like it (#10712 (comment)).

Any objections to try and update TF to the latest version currently 2.21 ?

[cmsbuild]

cms-bot internal usage

[cmsbuild]

A new Issue was created by @fwyzard.

@Dr15Jones, @ftenchini, @makortel, @mandrenguyen, @sextonkennedy, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[fwyzard]

assign core

[fwyzard]

assign ml

[cmsbuild]

New categories assigned: core,ml

@Dr15Jones,@hjkwon260,@makortel,@smuzaffar,@valsdav,@y19y19 you have been requested to review this Pull request/Issue and eventually sign? Thanks

[smuzaffar]

@fwyzard , @akritkbehera is already looking in to TF 2.21 since couple of days. He can check if the latest eigen 5x works with TF 2.21. We will be more than happy if we can use eigen from upstream without massive CMS patch

[akritkbehera]

I have successfully built TensorFlow for both ARM and x86_64.

In summary, updating TensorFlow to 2.21.0 requires updating the following dependency chain:

Abseil-CPP → Protobuf → gRPC

Updating Protobuf also requires regenerating its generated objects in CMSSW. Updating only Abseil-CPP and Protobuf is not sufficient, as gRPC depends on the newer Protobuf version. However, updating gRPC causes TensorFlow 2.17.0 to fail to build, making it necessary to upgrade TensorFlow and its dependency stack together. So all the changes need to go together. So if we decide to update Eigen we need to get all these changes in together since they depend on regenerating protobuf objects in cmssw.

[fwyzard]

@akritkbehera great, thank you very much for all the work!

Can we bring this up at a Core Software meeting, either tomorrow or next week ?