Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

One failing test causes other async tests to fail in the Karma DEBUG RUNNER #2811

Closed
daanstolp opened this issue Aug 18, 2017 · 2 comments · Fixed by karronoli/redpen#10 · May be fixed by Omrisnyk/npm-lockfiles#122, Omrisnyk/npm-lockfiles#132 or satoshinakamoto007/bitcore#1135

Comments

@daanstolp
Copy link
Contributor

daanstolp commented Aug 18, 2017

Consider the following tests (in Jasmine), where the first is synchronous and the second is asynchronous:

describe("The system", function() {
    
    it("should fail", function() {
        expect(true).toBe(false);
    });

    it("should succeed", function(done) {
        setTimeout(function() {
            expect(1).toBe(1);
            done();
        }, 10);
    });
});

Expected behaviour

  • When I run the tests in the default runner, I expect the first test to fail and the second test to pass.
  • When I run the tests in the Karma DEBUG RUNNER, I expect the first test to fail and the second test to pass.

Actual behaviour

  • When I run the tests in the default runner, the first test fails and the second test passes, as expected.
  • When I run the tests in the Karma DEBUG RUNNER, both tests fail. The error message for the second test is the same failure message that caused the first test to fail:
Uncaught Expected true to be false.
    at UserContext.<anonymous> (http://localhost:9876/base/test.spec.js:4:22) thrown

Environment Details

  • Karma version (output of karma --version): 1.7.0
  • karma.config.js file is default out of the box, configured to load '*.js' files.

Steps to reproduce the behaviour

  1. Create a new application using npm init
  2. npm install karma karma-jasmine jasmine-core karma-chrome-launcher --save-dev
  3. Create a new file called test.spec.js and add the test code that is listed above.
  4. Run karma init and accept all defaults. Enter *.js and *.spec.js as files location.
  5. Add "test": "karma start" to the package.json scripts section.
  6. run npm run test
  7. Click the Debug button to open the Karma DEBUG RUNNER.
  8. In the console of Google Chrome, you can now see the error messages.
dignifiedquire pushed a commit that referenced this issue Aug 25, 2017
Use console.error to log errors to the console instead of throwing
an error. This makes sure that errors are scoped properly for
asynchronous tests, instead of showing up as uncaught errors on
all asynchronous tests.

Fixes #2811
johnjbarton pushed a commit to johnjbarton/karma that referenced this issue Sep 6, 2017
Use console.error to log errors to the console instead of throwing
an error. This makes sure that errors are scoped properly for
asynchronous tests, instead of showing up as uncaught errors on
all asynchronous tests.

Fixes karma-runner#2811
@michael-letcher
Copy link

michael-letcher commented Jul 25, 2018

@daanstolp I think this has resurfaced. Using Angular CLI 6.0.6 (karam ^2.0.4).
@dignifiedquire

If a component throws an error (in our case we have a validator that throws an error if it's used incorrectly), the next test will fail. Running the tests in isolation will return a success.

In our case, we have a custom validator that matches two fields, but if the selected field does not exist the validator with throw an error. This causes the next it() test to fail.

@ovitrif
Copy link

ovitrif commented Jan 24, 2019

@michael-letcher I'm encountering this with such code:

spyOn(service, 'uploadFile').and.returnValue(throwError(Error()));

If I test in isolation, it works, if I run all tests, the second one fails.
Also, if I use DoneFn and I call it, the test including the spyOn fails

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment