Repository navigation
Conversation
|
I drafted this PR for now since I am not currently finished implementing all the logouts, but will undraft it once I am finished. I just had a few clarifying questions about some of the tests. |
| mocha.before(async () => { | ||
| // login | ||
| let res = await chai.request(app).post('/api/loginLogout/login') | ||
| .send({ username: testUser.username, password: testUser.password }); |
There was a problem hiding this comment.
If this is supposed to test both ADMIN and CSV roles, then it looks like it doesn't access the CSV role. The test looks like it simply runs ADMIN both times instead of depending on the role being looped. If this behavior is unintended, then I can make the appropriate changes.
There was a problem hiding this comment.
What I see seems very similar to the next test for the other roles:
mocha.describe('Admin role & CSV role:', () => {
for (const role in User.role) {
if (User.role[role] !== User.role.OBVIUS && User.role[role] !== User.role.EXPORT) {I see what you mean and you seem correct. I think it should login with the desired role and use that token. Thanks for offering to make that change.
Unlike the next test which should test everyone else, this one should only test these roles. Thus, I think it would be much better to check if the role is admin or csv rather than not the others. Then it will still work if more roles are added. If you agree, could you also do that?
There was a problem hiding this comment.
I adjusted the test to login with the same role, but I noticed that it would only pass if I changed the before hook to a beforeEach hook in src/server/test/web/meters.js on line 162. Otherwise, I kept getting the following errors for both admin and csv:
21 passing (35s)
2 failing
1) meters API
Admin role & CSV role:
should return all meters for ADMIN:
AssertionError: expected { id: 1, name: null, url: null, …(31) } to have property 'name' of 'Meter 1', but got null
at expectMetersToBeEquivalent (src/server/test/web/meters.js:54:26)
at Context.<anonymous> (src/server/test/web/meters.js:213:6)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2) meters API
Admin role & CSV role:
should return all meters for CSV:
AssertionError: expected { id: 1, name: null, url: null, …(31) } to have property 'name' of 'Meter 1', but got null
at expectMetersToBeEquivalent (src/server/test/web/meters.js:54:26)
at Context.<anonymous> (src/server/test/web/meters.js:213:6)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
I tested and found that the same error message is given for export and obvius roles for the same test, regardless of whether before or beforeEach is used (This might be the expected outcome, but shouldn't be for admin or csv). I also found that there's no difference between using an after or afterEach hook on line 176, but I kept it as afterEach to be consistent. I'm unsure of why I would strictly need a beforeEach for this test, but I plan on leaving it like this unless you have any input.
There was a problem hiding this comment.
I was thinking about something similar while reviewing the code in other files. It seems that tests generally do a before/after if a single test and beforeEach/afterEach if multiple tests. For multiple tests it is generally needed since the different tests use different users. For a single test it should not matter. Now if someone adds more tests to the currently single test with before/after then it might pick up a token that is not intended. Thus, what do you think about consistently using beforeEach/afterEach as long as it does not cause issues? This would invalidate the token so if someone tries it would fail the token validation on the route. I'm only proposing for the tests you touched with login/out.
To be clear, this is an example of a test with the Each usage but only has a single test so, as you state, it should not matter.
There was a problem hiding this comment.
Thus, what do you think about consistently using beforeEach/afterEach as long as it does not cause issues?
I pushed a change to all the relevant before/after hooks to consistently use beforeEach/afterEach hooks, including those that only have a single test. One file that I don't think it's necessary for is src/server/test/web/usersTest.js since all the tests use the same admin token, but I included the change since it seemed to have no problems. I don't mind reverting the change though.
There was a problem hiding this comment.
The change seems to work fine. However, I was unsure why it made such a difference so I tried to analyze it more. Unfortunately, I was unable to resolve it but have figured out there is something subtle (at least to me) going on here. Not being very knowledgeable here may mean I'm missing something. With debug statements I determined that the before/afterEach runs for both user roles that pass the if but it does it before both set of tests. Thus, the before is running for all tests for all user roles. You also see the loop starts and finishes all roles before it does any actual tests. It uses the correct token for each test but gets tokens for all roles that pass the if. I did some research and that is how mocha works. I thought not using a for loop but forEach by looping over the role Object types would fix it but it did not (at least so far). I have my code that I used for testing and some web references if someone is willing to look at this. This pattern is used in several places in OED testing so knowing exactly what is happening would be nice even though the tests pass.
There was a problem hiding this comment.
I had a difficult time fully grasping the behavior/issue described, so I tried running experiments with debug statements for a different role loop in src/server/test/web/groups.js. If my understanding is correct, the problem is that instead of logging in/out for only the current role in the for loop iteration, it performs the login/logout for every role available, which creates unnecessary work.
I also tested changing it back to mocha.before, and saw that it only logs in/out for all roles at the beginning and the end of all tests instead of logging in/out once for each loop iteration, which seems to be confirmation that it's not tied to the for loop at all, but to the mocha.it tests, which do run for each loop iteration.
There was a problem hiding this comment.
Another potentially relevant issue: When I ran the before hook instead of beforeEach, it consistently gave 401 Authentication errors instead of the expected 403 Forbidden errors, despite using the correct tokens. I also commented out the logout request, and the before version still produces 401, so logging out doesn't affect the error code. The debug output shows that all three role login hooks execute before any tests when using before. I'm not entirely sure this is directly related to the issue above, but I thought it was worth noting.
|
|
|
@huss I realized that the last 2 checks on my pull request never ran at all. Do you know how to fix this issue? My guess would be that it has to do with manually stopping a previous check. |
|
I'll run the tests once I review the PR. If checks and tests pass on your machine then they should on GitHub. Hopefully I can get to this later in the week. |
| log.logToDb = true; | ||
| }); | ||
|
|
||
| mocha.after(async () => { |
There was a problem hiding this comment.
For this test, I wanted to apply the potential change I noted in the issue description:
For tests with multiple users, instead of logging out all the users at the very end, it might be better to do one user at a time, log in, do the test & log out for each one.
Even though it doesn't have multiple users, I thought this would be a good place to login/logout for every test since it runs for a long time. However, when I tried changing to beforeEach/afterEach, the checks had continually failed.
I'm ok with leaving it like this, but I think this should be addressed if the root cause of the failures can be found.
There was a problem hiding this comment.
I'm actually considering removing the changes to this file entirely. After implementing this, I've noticed that even though it passed the most recent test, the node checks have a chance of failing different tests for a reason I'm unaware of. If it gets merged, I'm afraid that other people will start seeing similar issues.
There was a problem hiding this comment.
Curious. Do you want to wait until I review the PR to see if I have any ideas? One test should not, in a perfect world, impact another test. It makes me wonder if something else is wrong that really should be tracked down. If it cannot be tracked down during this work then I suspect a new issue should be opened to look at this further. Thoughts?
There was a problem hiding this comment.
I'm okay with opening a new issue if it doesn't get resolved after a code review. If that's the case, then I think I should comment out the changes with a TODO describing the problem.
There was a problem hiding this comment.
I changed the tests in this file to use beforeEach/afterEach. I ran the single file of tests and the complete suite of tests multiple times but have yet to see it fail. Thus, I wanted to ask if you are still seeing this. If so, how often does it happen and is there anything special in your setup?
260921 update: This may be related to another comment on before/afterEach. I'm not adding any info but leaving it as part of that.
There was a problem hiding this comment.
Thanks for responding. I also don't see it on my local machine, but I was referring to the GitHub testing suite which seems to either successfully pass or give an error on different files, such as src/server/test/web/csvPipelineTest.js or src/server/test/routes/logsRouteTests.js even though it ran mostly the same code.
Recently it seems to be working on GitHub, but it could potentially be an issue if other people pull in these changes and it returns a phantom failure.
I'll change the code to beforeEach/afterEach for now and see if the problem persists.
There was a problem hiding this comment.
I made another comment about a potential issues I could not resolve. If this is not figured out soon then I propose to create an issue that documents the code you add (probably via a commit id) and discussed what is happening. Then it can be addressed later since there is a good chance it is not a direct result of this work. How does that sound?
There was a problem hiding this comment.
Sounds good to me. I think I have a better understanding of what's happening now, but I agree that this could be a separate issue. I'm fine with creating an issue referencing the commit and including the debug output and any relevant information about the Mocha hook behavior.
|
@Nespina24 I appreciate you keeping this up-to-date. Per my message on Discord, it will take me some time to get to review this. |
Just noting that all tests run fine on my machine and are passing on GitHub. |
huss
left a comment
There was a problem hiding this comment.
@Nespina24 First, I want to apologize for how long this review took (see OED Discord server msg in developer channel).
Thank you for another contribution. Review and testing found it works well. I appreciate you fixing some other, minor issues while doing this work. I made a few comments to consider but nothing that seems substantial to me. Please let me know if anything is not clear or you have questions/thoughts.
| mocha.before(async () => { | ||
| // login | ||
| let res = await chai.request(app).post('/api/loginLogout/login') | ||
| .send({ username: testUser.username, password: testUser.password }); |
There was a problem hiding this comment.
I was thinking about something similar while reviewing the code in other files. It seems that tests generally do a before/after if a single test and beforeEach/afterEach if multiple tests. For multiple tests it is generally needed since the different tests use different users. For a single test it should not matter. Now if someone adds more tests to the currently single test with before/after then it might pick up a token that is not intended. Thus, what do you think about consistently using beforeEach/afterEach as long as it does not cause issues? This would invalidate the token so if someone tries it would fail the token validation on the route. I'm only proposing for the tests you touched with login/out.
To be clear, this is an example of a test with the Each usage but only has a single test so, as you state, it should not matter.
| log.logToDb = true; | ||
| }); | ||
|
|
||
| mocha.after(async () => { |
There was a problem hiding this comment.
I changed the tests in this file to use beforeEach/afterEach. I ran the single file of tests and the complete suite of tests multiple times but have yet to see it fail. Thus, I wanted to ask if you are still seeing this. If so, how often does it happen and is there anything special in your setup?
260921 update: This may be related to another comment on before/afterEach. I'm not adding any info but leaving it as part of that.
huss
left a comment
There was a problem hiding this comment.
@Nespina24 Thank you for updating the code. I have mostly resolved the previous comments and the code is running well. There are three remaining (may be related) that may become future issues. If you have thoughts on them I would love to hear them. I also made a few new comments but I think they should be straightforward. I think this is almost ready to merge. The larger issue(s) will probably either be new issue(s) or resolved here in a reasonable amount of time. Please let me know if anything is not clear or if you have thoughts/ideas.
| mocha.beforeEach(async () => { | ||
| // insert test user | ||
| const conn = testDB.getConnection(); | ||
| const password = 'password'; |
There was a problem hiding this comment.
You didn't do this and it matters less now than before since you logout each time. However, I am pushing for each login to use a password that is specific to the user. For example, 'password${role}' and this is similar to the login name/email.
Also, I'm realizing that a lot of tests do login/out. I was thinking about a utility function to do this and then having all tests use it. What do you think? It would be an issue of the future.
There was a problem hiding this comment.
I made sure to get all the 'password' strings in each test that loops over each user.
I see you already made an issue for the utility function, and I agree that it should be implemented. I don't mind taking the issue if it's still available after I finish this PR.
| unauthorizedUser.password = password; | ||
|
|
||
| // login | ||
| let res = await chai.request(app).post('/api/loginLogout/login') |
| mocha.beforeEach(async () => { | ||
| // insert test user | ||
| const conn = testDB.getConnection(); | ||
| const password = 'password'; |
| }); | ||
| mocha.it('returns all meters', async () => { | ||
| mocha.beforeEach(async () => { | ||
| // insert test user |
| mocha.before(async () => { | ||
| // login | ||
| let res = await chai.request(app).post('/api/loginLogout/login') | ||
| .send({ username: testUser.username, password: testUser.password }); |
There was a problem hiding this comment.
The change seems to work fine. However, I was unsure why it made such a difference so I tried to analyze it more. Unfortunately, I was unable to resolve it but have figured out there is something subtle (at least to me) going on here. Not being very knowledgeable here may mean I'm missing something. With debug statements I determined that the before/afterEach runs for both user roles that pass the if but it does it before both set of tests. Thus, the before is running for all tests for all user roles. You also see the loop starts and finishes all roles before it does any actual tests. It uses the correct token for each test but gets tokens for all roles that pass the if. I did some research and that is how mocha works. I thought not using a for loop but forEach by looping over the role Object types would fix it but it did not (at least so far). I have my code that I used for testing and some web references if someone is willing to look at this. This pattern is used in several places in OED testing so knowing exactly what is happening would be nice even though the tests pass.
| log.logToDb = true; | ||
| }); | ||
|
|
||
| mocha.after(async () => { |
There was a problem hiding this comment.
I made another comment about a potential issues I could not resolve. If this is not figured out soon then I propose to create an issue that documents the code you add (probably via a commit id) and discussed what is happening. Then it can be addressed later since there is a good chance it is not a direct result of this work. How does that sound?
|
@Nespina24 I see you marked some comments but not the hard and potentially future ones. If you want me to review since you are done with updates for now then please let me know. |
Description
This PR cleans up tests suites by adding in a logout route to test users who login. When a test logs in, it should also clean up by logging out, so the authentication lifecycle follows a consistent pattern (login → perform the test → logout). This PR addresses the issue in the following files:
Fixes #1683
Type of change
Checklist
Limitations
No limitations.