Enhancement: Performance optimized functions for the FSDA toolbox

[sindhujapr]

This issue is regarding performance improvement opportunities identified using the "matlab-optimize-performance" agentic AI skill that is available in the MATLAB Agentic Toolkit (https://github.com/matlab/matlab-agentic-toolkit). The code changes to various function in the FSDA toolbox show up to 4.3x improvement in performance. This also includes tests that validate the optimizations for correctness. Assuming there are no flags on this, I will open a PR corresponding to this issue.

[MarcoRianiUNIPR]

@sindhujapr first of all thank you very much.
Yes please open a PR corresponding to this issue. We are eager to test the new code.

[sindhujapr]

Thank you, @MarcoRianiUNIPR ! I have opened a PR here: #127. Please let me know how the testing goes. I look forward to our continued collaboration!

[DomenicoPerrottaJRC]

Dear @sindhujapr,

Thank you very much for your pull request #127, which addresses issue #126. We are genuinely pleased to see the FSDA toolbox being used as a testbed for the MATLAB Agentic Toolkit's matlab-optimize-performance and matlab-write-performance-tests skills.

We have thoroughly reviewed the summary of the proposed changes. The contribution is valuable on two distinct levels. First, the optimizations themselves target some of the most computationally intensive workhorses of the toolbox. Even modest per-call speedups can lead to significant gains, as these routines are executed thousands of times in typical robust analyses and simulation studies.

We understand cases where it is crucial to adopt the mex (as in the case of quickselectFS). We are working on this issue but are not ready to adopt this solution in FSDA because an article about this solution has already been submitted (a relatively old pre-print is here, if you are interested: https://arxiv.org/abs/2302.05705).

We would like to provide explicit feedback on the current behavior of the matlab-optimize-performance and matlab-write-performance-tests skills. While reviewing the diffs, we noticed that our function headers and some inline comments were removed. This is a serious issue for FSDA because our headers follow a strict format used to automatically generate the HTML help documentation shipped with the toolbox. Any alteration breaks the documentation build. Inline comments are also important as they document the statistical reasoning for future maintainers. Therefore, we encourage the Agentic Toolkit team to consider a "preserve comments and headers" mode, or make this the default. Other toolbox authors who keep documentation inside source files may face the same issue. If helpful, we can open an issue on the matlab-agentic-toolkit repository with concrete examples from this PR.

A few further points we would like to discuss during the review:

1. Could you please share with us the benchmark results you obtained, including the problem sizes and measured speedups? This information will be valuable for documenting the gains in the release notes and reproducing them on our HPC environment in Parma.

2. After updating the other files, the creation of the MEX file TBrhowei appears unnecessary. Specifically, the difference in computational time between using and not using the mex file is about half a second out of the total computational time of approximately 40 seconds. Therefore, we have decided not to include it. Please verify whether running your agentic AI in the updated version of FSDA still requires combining the two routines and creating the mex file.

3. We would greatly appreciate it if you could integrate the performance suite into our continuous integration workflow on GitHub. We would also welcome any suggestions you may have on how to run these tests.

Thank you once again for this excellent contribution. Collaborations of this nature between MathWorks and the FSDA team are highly valued by us and the user community of our toolbox.

Best regards,

The FSDA development team
University of Parma and EC Joint Research Centre

[sindhujapr]

Thank you to you and the rest of the FSDA development team for reviewing the changes thoroughly and providing great feedback. Let me get back to you after consulting with my team to see what is the best way to move forward on:

1. implementing the "headers" feedback on the matlab-optimize-performance skill.
2. sharing of results and problem size, etc.
3. integrating the performance suite into your CI workflow

We are honored and looking forward to this continued collaboration!

[sindhujapr]

Hi @DomenicoPerrottaJRC

Thank you again for validating the optimizations and providing us with valuable feedback.

1. I have emailed all the artifacts including the benchmarking report and data to your team at "FSDA@unipr.it". Please let me know if that is what you were looking for.
2. I will carve out some time this week to "verify whether running your agentic AI in the updated version of FSDA still requires combining the two routines and creating the mex file" and get back to you.
3. The PR did include a "tests" folder with the performance tests. These tests were generated using the matlab-write-performance tests skill. These tests are also available in the emailed artifacts. It would be good to validate these tests too and automate running them before submitting any change to this repo.
4. Regarding the issue of "preserve comments and headers", please create an issue as you suggested on the MATLAB Agentic Toolkit for the "matlab-optimize-performance" skill.

We look forward to our continued engagement.