Report a failed CPU or GPU stress test instead of always passing - #7
Open
munzzyy wants to merge 1 commit into
Open
Report a failed CPU or GPU stress test instead of always passing#7munzzyy wants to merge 1 commit into
munzzyy wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The CPU and GPU stress tests report a pass in the board report even when the stress process dies on startup and no load is ever applied.
Two things combine to cause it.
generate_combined_report.pychecks the marker file like this:cpu_test.shwritesCPU_STRESS=RUNNINGat line 21 andPID=at line 38, both unconditionally, so the first branch always matches and theSTATUS=FAILEDbranch can't be reached. Nothing writesSTATUS=FAILEDin the first place, so even reordering alone wouldn't change anything.gpu_test.shand the GPU half of the report have the same shape.So I did both halves: check
STATUS=FAILEDfirst, and actually write it when the stress process isn't alive a second after launch. For the GPU that means checking all four PIDs and only marking failed if none survived.Reproduced with a
stress-ngstub that exits 1 immediately, running the realcpu_test.sh:Same marker file through both versions of the check:
The one-second sleep is there because the PID exists for a moment either way, so
kill -0right after$!would pass even for a process that's already on its way out. It only delays the two stress scripts by a second each.Worth saying: I could only test this on a normal x86 Linux box with stubbed binaries, not on an RK3576 board, so a look from someone with hardware would be good — particularly on whether one second is enough headroom for glmark2 to get going on that GPU.
One thing worth heading off: the
kill -0check would be wrong if a stress run could legitimately finish inside that second. It can't here —stress-ng --cpu 0 --metrics-briefis started with no--timeout, and all four glmark2 processes use--run-forever. Both are meant to run untilstart-tests.shkills them, so a process that's gone after a second has died, not finished.