Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add autoWatchDelay option #2331

Closed
danielcompton opened this issue Aug 25, 2016 · 7 comments · Fixed by karronoli/redpen#10 · May be fixed by satoshinakamoto007/bitcore#1135
Closed

Add autoWatchDelay option #2331

danielcompton opened this issue Aug 25, 2016 · 7 comments · Fixed by karronoli/redpen#10 · May be fixed by satoshinakamoto007/bitcore#1135

Comments

@danielcompton
Copy link
Contributor

Problem

When incrementally recompiling ClojureScript, many files are changed as the compilation process runs. Currently, it is difficult to use Karma to autorun tests after detecting a ClojureScript recompile, because as soon as the first file is changed, Karma wants to rerun the tests.

Current behaviour

There is a config option autoWatchBatchDelay available which at first seems like it would be ideal for this use case. It starts a timer after it detects the first file system change, and waits n milliseconds before running tests. It is possible to use this option, but it is suboptimal.

The timer starts from the first change detected, so when configuring this parameter, I need to specify what the maximum time my ClojureScript compile could take, so it doesn't start rerunning while recompiling. I can shoot for a low number that my app will recompile in (say 1 second), but then will hit problems if an incremental recompile takes longer (which happens from time to time when editing certain files).

Proposed behaviour

I propose that there is a new option autoWatchDelay (or perhaps changing the behaviour of the current option) to start the delay timer from the last filesystem change detected, waiting n more milliseconds before starting the tests. In practice this might look like resetting the timer, each time a new file change was detected.

IMHO this behaviour is probably closer to what most people are wanting when they use autoWatchBatchDelay.

@nschipperbrainsmith
Copy link

nschipperbrainsmith commented Sep 22, 2016

Think the thing most of us would like to have is a system that allows us to debounce the incoming changes till no further changes have occurred for an x amount of ms.

A perfect example of this is illustrated within RXJS:
http://reactivex.io/documentation/operators/debounce.html

Having an option like that would be very useful.

Stand alone version of debounce:
https://davidwalsh.name/javascript-debounce-function

Which would mean the following function probably would have to be modified:

https://github.com/karma-runner/karma/blob/master/lib/watcher.js#L93

Once i change the bind function to this i have the desired behaviour:

    function bind(func, wait, immediate) {
        var timeout;
        return function(path) {
            var context = this, args = arguments;
            var later = function() {
                timeout = null;
                if (!immediate) func.call(fileList, helper.normalizeWinPath(path));// func.apply(context, args);
            };
            var callNow = immediate && !timeout;
            clearTimeout(timeout);
            timeout = setTimeout(later, wait);
            if (callNow) func.call(fileList, helper.normalizeWinPath(path));
        };
    };

    chokidarWatcher.on('add', bind(fileList.addFile, 1000))
        .on('change', bind(fileList.changeFile, 1000))
        .on('unlink', bind(fileList.removeFile, 1000))

@danielcompton
Copy link
Contributor Author

@schippie this is exactly what I'm after.

@dignifiedquire
Copy link
Member

Sounds good to me. I agree it does make sense to change the option to this behaviour.

@ghost
Copy link

ghost commented Oct 25, 2016

I ran into this exact issue - chokidar has a nice option called awaitWriteFinish which solves this problem quite well.

@danielcompton
Copy link
Contributor Author

awaitWriteFinish looks useful and may be helpful, but wouldn't solve this problem on it's own probably. I still need to handle the case where multiple files are written in one recompile cycle.

@ghost
Copy link

ghost commented Oct 25, 2016

You're right - opened issue #2420 to discuss a different but related issue.

danielcompton added a commit to danielcompton/karma that referenced this issue Feb 8, 2017
danielcompton added a commit to danielcompton/karma that referenced this issue Feb 9, 2017
danielcompton added a commit to danielcompton/karma that referenced this issue May 7, 2017
danielcompton added a commit to danielcompton/karma that referenced this issue May 7, 2017
danielcompton added a commit to danielcompton/karma that referenced this issue May 8, 2017
- renamed batchInterval to autoWatchBatchDelay to aid in greppability.
- Modified debouncing tests to wait on promises.
- Added documentation explaining how list.removeFile is different from
list.addFile and list.changeFile.

Closes karma-runner#2331
danielcompton added a commit to danielcompton/karma that referenced this issue May 8, 2017
- renamed batchInterval to autoWatchBatchDelay to aid in greppability.
- Modified debouncing tests to wait on promises.
- Added documentation explaining how list.removeFile is different from
list.addFile and list.changeFile.

Closes karma-runner#2331
danielcompton added a commit to danielcompton/karma that referenced this issue May 8, 2017
- renamed batchInterval to autoWatchBatchDelay to aid in greppability.
- Modified debouncing tests to wait on promises.
- Added documentation explaining how list.removeFile is different from
list.addFile and list.changeFile.

Closes karma-runner#2331
danielcompton added a commit to danielcompton/karma that referenced this issue May 8, 2017
- renamed batchInterval to autoWatchBatchDelay to aid in greppability.
- Modified debouncing tests to wait on promises.
- Added documentation explaining how list.removeFile is different from
list.addFile and list.changeFile.

Closes karma-runner#2331
danielcompton added a commit to danielcompton/karma that referenced this issue May 8, 2017
- renamed batchInterval to autoWatchBatchDelay to aid in greppability.
- Modified debouncing tests to wait on promises.
- Added documentation explaining how list.removeFile is different from
list.addFile and list.changeFile.

Closes karma-runner#2331
@danielcompton
Copy link
Contributor Author

danielcompton commented Dec 14, 2018

Just a note for future searchers who find this, the second half of making this work for testing ClojureScript files is to disable the caching of any files written by the compiler.

The Google Closure Compiler (GCC) writes files to disk in chunks. By default, Karma will cache the files on disk when serving web requests. However there is a race condition that exists where:

  1. GCC writes part of a file. At this point, the file is probably not even syntactically correct.
  2. Karma file watcher detects the change
  3. Karma caches the partially written file
  4. GCC finishes writing the file. The file is now valid JavaScript.
  5. autoWatchBatchDelay time passes
  6. Karma runs the tests
  7. The browser requests the file.
  8. Karma serves the partial file to the browser
  9. You get a confusing test error like "message": "Uncaught SyntaxError: missing ) after argument list..."

The fix is to update your file watchers to set nocache: true: e.g.

files: [
            ...
            {pattern: root + '/**/*.+(cljs|cljc|clj|js|js.map)', included: false, nocache: true}
        ],

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment