-
-
Notifications
You must be signed in to change notification settings - Fork 4.9k
test: replace WebdriverIO with Cypress #19465
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
✅ Deploy Preview for docs-eslint canceled.
|
|
Thanks for the new pull request @Pixel998. I can see that the browser test is still running quite slowly in CI. The current run took over 8 minutes. I was trying to use cypress-io/github-action to speed it up, but it didn't help. So I'm not sure. Either Cypress is just this lame in GitHub or we don't know how to set it up properly. |
|
Maybe we could try https://modern-web.dev/docs/test-runner/overview/ instead. |
|
@fasttime So it appears that making Cypress not render the runner ui fixes the performance issue. |
|
I think this is quite acceptable now. It's not as fast as WebdriverIO but still faster then Mocha unit tests and it never timed out in my test runs. Probably it isn't even necessary to use Firefox as a browser. |
|
Note that the output of the Test step is showing two strange warnings:
The first warning is filed as a bug in Cypress (cypress-io/cypress#21805 and cypress-io/cypress#22273), so maybe we could leave it as it is until they fix it. If we want to get rid of the warning, we could create a directory The second warning seems to only occur when test_on_browser:
name: Browser Test
runs-on: ubuntu-latest
+ env:
+ TERM: xterm-256colorClearly there are also other options. |
You're right. Chrome is now as fast locally, so I've switched back to it.
I think we can avoid the workaround for now. Given that the warning isn't causing any major issues, waiting for the Cypress fix seems like the best approach. Unless you have a strong preference for implementing the workaround, I'd suggest we leave it as is.
Done. |
|
I think the output of Cypress is also easier to read. For comparison, this is what a test failure in GitHub Actions currently looks like with WebdriverIO — notice that there's no stack trace: And this is the same failure with Cypress and the |
fasttime
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks! I would like another review from @nzakas.
mdjermanovic
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks!
nzakas
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. I tested this locally and it has roughly the same speed as the WebDriver approach. Well done!
Just need to wait for the patch release window to close.


Prerequisites checklist
What is the purpose of this pull request? (put an "X" next to an item)
[ ] Documentation update
[x] Bug fix (template)
[ ] New rule (template)
[ ] Changes an existing rule (template)
[ ] Add autofix to a rule
[ ] Add a CLI option
[ ] Add something to the core
[ ] Other, please explain:
Fix #19234
What changes did you make? (Give an overview)
Replace WebdriverIO with Cypress
Is there anything you'd like reviewers to focus on?