Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to exclude node_modules but one #2031

Closed
borm opened this issue Feb 11, 2016 · 48 comments
Closed

How to exclude node_modules but one #2031

borm opened this issue Feb 11, 2016 · 48 comments
Labels

Comments

@borm
Copy link

borm commented Feb 11, 2016

Subj,
as example
I create some module in another folder ( /projects/MY_MODULE )
MY_MODULE not compiled, source code
then run npm link
after go to my project and run npm link MY_MODULE
Compile my project and have error two copies React

i`am try

{
        test: /\.js$/,
        exclude: [/node_modules/],
        include: [/node_modules\/MY_MODULE/]
      },
@sokra
Copy link
Member

sokra commented Feb 12, 2016

{
        test: /\.js$/,
        include: [/node_modules\/MY_MODULE/]
      },

@borm
Copy link
Author

borm commented Feb 12, 2016

@sokra

./src/client.js
[0] Module parse failed: /Volumes/www/src/client.js Line 4: Unexpected token
[0] You may need an appropriate loader to handle this file type.
[0] | * THIS IS THE ENTRY POINT FOR THE CLIENT, JUST LIKE server.js IS THE ENTRY POINT FOR THE SERVER.
[0] | */
[0] | import 'babel/polyfill';
[0] | import React from 'react';
[0] | import ReactDOM from 'react-dom';
[0]  @ multi main
[0] webpack built 12ed16c561926b4fa5a1 in 204ms

and not compiled :(

my webpack config

require('babel/polyfill');

// Webpack config for development
var fs = require('fs');
var path = require('path');
var webpack = require('webpack');
var WebpackIsomorphicTools = require('webpack-isomorphic-tools');
var assetsPath = path.resolve(__dirname, '../static/dist');
var host = (process.env.HOST || 'localhost');
var port = parseInt(process.env.PORT) + 1 || 3001;

// https://github.com/halt-hammerzeit/webpack-isomorphic-tools
var WebpackIsomorphicToolsPlugin = require('webpack-isomorphic-tools/plugin');
var webpackIsomorphicToolsPlugin = new WebpackIsomorphicToolsPlugin(require('./webpack-isomorphic-tools'));

var babelrc = fs.readFileSync('./.babelrc');
var babelrcObject = {};

try {
  babelrcObject = JSON.parse(babelrc);
} catch (err) {
  console.error('==>     ERROR: Error parsing your .babelrc.');
  console.error(err);
}

var babelrcObjectDevelopment = babelrcObject.env && babelrcObject.env.development || {};
var babelLoaderQuery = Object.assign({}, babelrcObject, babelrcObjectDevelopment);
delete babelLoaderQuery.env;

babelLoaderQuery.plugins = babelLoaderQuery.plugins || [];
if (babelLoaderQuery.plugins.indexOf('react-transform') < 0) {
  babelLoaderQuery.plugins.push('react-transform');
}

babelLoaderQuery.extra = babelLoaderQuery.extra || {};
if (!babelLoaderQuery.extra['react-transform']) {
  babelLoaderQuery.extra['react-transform'] = {};
}
if (!babelLoaderQuery.extra['react-transform'].transforms) {
  babelLoaderQuery.extra['react-transform'].transforms = [];
}
babelLoaderQuery.extra['react-transform'].transforms.push({
  transform: 'react-transform-hmr',
  imports: ['react'],
  locals: ['module']
});

module.exports = {
  devtool: 'inline-source-map',
  context: path.resolve(__dirname, '..'),
  entry: {
    'main': [
      'webpack-hot-middleware/client?path=http://' + host + ':' + port + '/__webpack_hmr',
      './src/client.js'
    ]
  },
  output: {
    path: assetsPath,
    filename: '[name]-[hash].js',
    chunkFilename: '[name]-[chunkhash].js',
    publicPath: 'http://' + host + ':' + port + '/dist/'
  },
  module: {
    loaders: [
      {
        test: /\.js$/,
        exclude: [/node_modules/],
        include: [/node_modules\/ui/],
        loaders: ['babel?' + JSON.stringify(babelLoaderQuery)]
      }
    ]
  },
  progress: true,
  resolve: {
    modulesDirectories: [
      'src',
      'node_modules'
    ],
    extensions: ['.js']
  },
  plugins: [
    // hot reload
    new webpack.HotModuleReplacementPlugin()
  ]
};

@sokra
Copy link
Member

sokra commented Feb 12, 2016

Include you src directory and the other directory. Don't use exclude.

  include: [
    path.resolve(__dirname, "src"),
    path.resolve(__dirname, "node_modules/ui")
  ]

@borm
Copy link
Author

borm commented Feb 12, 2016

@sokra
if i don't use exclude: [/node_modules/], i will get an error parsing jquery and other libraries over 200Kb size, and compiling takes a lot of time

@ghigt
Copy link

ghigt commented May 12, 2016

I have the same issue, I can't include all node_modules but just the one written in es6 to babelify it.

@ghigt
Copy link

ghigt commented May 12, 2016

I don't know if it could be the fix but in lib/LoadersList.js:

LoadersList.prototype.matchObject = function matchObject(str, obj) {
    if(obj.test && !this.matchPart(str, obj.test)) return false;
    if(obj.include && !this.matchPart(str, obj.include)) return false;
    if(obj.exclude && this.matchPart(str, obj.exclude)) return false;
    return true;
};

Could it be better to have something like:

- if(obj.include && !this.matchPart(str, obj.include)) return false;
+ if(obj.include && this.matchPart(str, obj.include)) return true;

@ghigt
Copy link

ghigt commented May 13, 2016

@borm: a solution:

{
  test: /\.js$/,
  exclude: /node_modules\/(?!(MY-MODULE|ANOTHER-ONE)\/).*/,
},

@borm
Copy link
Author

borm commented May 16, 2016

@ghigt, oh thanks, but i just use webpack-node-externals
import nodeExternals from 'webpack-node-externals' ... externals: [nodeExternals({ whitelist: ['MY-MODULE','ANOTHER-ONE'] })], ...

eastonqiu added a commit to eastonqiu/RNStarter that referenced this issue Jul 30, 2016
reason: (1) babel-polyfill confict with other dependence lib that all compiled by webpack config
        (2) fix the exclude config for the react-nav-bar and other dependence lib
reference:
  https://github.com/react-native-community/react-native-navbar#usage-with-webpack
  webpack/webpack#2031
@borm borm closed this as completed Aug 12, 2016
@megatolya
Copy link

megatolya commented Sep 6, 2016

dont know why but @sokra solution raised new exception
exclude inside exclude is my solution 😅:

exclude: /node_modules(?!\/my-es6-raw-module)/

UPD IMO exclude as a function (comments below) is better option

jaredhirsch added a commit to jaredhirsch/testpilot that referenced this issue Feb 9, 2017
- Remove the restriction on ES6 module processing from babel config
  (hopefully this is the right option to change)

- Rather than exclude all of node_modules, just include the one module
  we need to process, and implicitly exclude the rest

  - `include` syntax based on webpack/webpack#2031 (comment)
@nowells
Copy link

nowells commented Mar 2, 2017

I didn't see this option listed here, so I thought I might as well drop in my findings. I found it useful to leverage the ability to specify an include or exclude as a function (I prefer the explicit include over exclude personally).

{
  test: /\.js$/,
  exclude: function(modulePath) {
    return /node_modules/.test(modulePath) &&
        !/node_modules\/MY_MODULE/.test(modulePath);
  }
}

@KhramtsovMaksim
Copy link

example for webpack 2 documentation

exclude: {
    test   : resolve(__dirname, "node_modules"),
    exclude: resolve(__dirname, "node_modules/unfetch") // or your module - also can be an array (read doc)
}

@lxjwlt
Copy link

lxjwlt commented Apr 18, 2017

In Windows, modulePath would be C:\path\to\project-name\node_modules\MY_MODULE,solution may be :

exclude: /node_modules(?!(\/|\\)MY_MODULE)/

@mcelotti
Copy link

Linux uses "/" while Windows uses "\" in modulePath so I ended up using the exclude: function (modulePath) to handle both.

@flybayer
Copy link

I finally got a node_modules package to compile with babel-loader after hours of struggling.

The problem was that the package had it's own .babelrc published which was overriding my babel config (which is in my package.json).

node_modules/
  vue-match-media/
    .babelrc   <--- babel-loader was using this which didn't have my required config
    dist/
      index.js  <--- file I needed compiled
package.json  <--- contains the babel config I need everything to use
build/
  webpack.base.config.js <-- my weback config
src/

My solution is to set babelrc: false in the loader config and specify the babel config in the loader. However, I read this config from my package.json, so it's not duplicated.

var readPackageJson = require('read-pkg')
var babelOptions = readPackageJson.sync().babel
babelOptions.babelrc = false

// ... rest of config
      {
        test: /\.js$/,
        loader: 'babel-loader',
        include: [path.join(__dirname, '..', 'node_modules/vue-match-media')],
        options: babelOptions,
      },

@jh3141
Copy link

jh3141 commented Jul 25, 2017

Building on @nowells suggestion above and incorporating the comment from @lxjwlt about Windows paths being different, I decided to make a function to build the necessary regexps automatically with the correct path separator:

function excludeNodeModulesExcept (modules)
{
    var pathSep = path.sep;
    if (pathSep == '\\') // must be quoted for use in a regexp:
        pathSep = '\\\\';
    var moduleRegExps = modules.map (function (modName) { return new RegExp("node_modules" + pathSep + modName)})

    return function (modulePath) {
        if (/node_modules/.test(modulePath)) {
            for (var i = 0; i < moduleRegExps.length; i ++)
                if (moduleRegExps[i].test(modulePath)) return false;
            return true;
        }
        return false;
    };
}

Usage is to put the above function in your preamble, and then call it to generate the "exclude" value, e.g.:

module.exports = {
  module: {
      rules: [
          {
              test: /\.js$/,
              loader: 'babel-loader',
              options: { presets: ['es2017', 'es2016', 'es2015' ] },
              exclude: excludeNodeModulesExcept(["lru_map"])
          },

You can add however many modules you need to exclude from exclusion to the list, although note that the test is O(n) in the number of modules, so if you have a lot of exclusions to process it may be worth finding a better way.

@liqiang0335
Copy link

liqiang0335 commented Apr 30, 2019

https://babeljs.io/docs/en/config-files#6x-vs-7x-babelrc-loading
If you use babel 7.x, according to this documentation, you should change .babelrc to babel.config.js.

thanks,It help me

@LeaveTheCapital
Copy link

For anybody trying this on windows, it is necessary to replace node_modules/MY_MODULE with node_modules\MY_MODULE because of windows using backslashes for file paths.. anyone who has ever diagnosed a bug to being a conflict between the direction of slashes on Windows vs Unix you will feel my pain!

@lake2
Copy link

lake2 commented Jul 9, 2019

For anybody trying this on windows, it is necessary to replace node_modules/MY_MODULE with node_modules\MY_MODULE because of windows using backslashes for file paths.. anyone who has ever diagnosed a bug to being a conflict between the direction of slashes on Windows vs Unix you will feel my pain!

Thanks !! Save my life!!

@WaterHong
Copy link

@borm: a solution:

{
  test: /\.js$/,
  exclude: /node_modules\/(?!(MY-MODULE|ANOTHER-ONE)\/).*/,
},

it's work!

@dmitrienkop
Copy link

@jh3141 the most elegant solution, thanks!

@gabstin
Copy link

gabstin commented Dec 5, 2019

 rules: [
      {
        test: /\.js$/,
        loader: 'babel-loader',
        exclude: {
          test: path.resolve(__dirname, "node_modules"),
          exclude: [
            path.resolve(__dirname, "node_modules/react-dnd"),
          ]
        },
      },

@brownieboy
Copy link

brownieboy commented Jan 11, 2020

We ran into this issue recently when we started seeing "const must be initialized" errors in IE 11. It turned out that some our dependencies, notably some of the D3 libraries, were no longer being transpiled to ES6. (That's a deliberate decision on the part of D3's maintainer, FYI.)

Here's a rule that I added to our Webpack config file to transpile just the libraries affected:

{
       include: mPath => {
         // Some npm modules no longer transpiled to ES5, which
         // causes errors such as "const must be initialized" IE 11 and crash
         // the build.  So we need to transpile just those modules here.

        const inclusionReg = /node_modules\/(.*(d3-array))|(.*(d3-delaunay))|(.*(d3-sankey))|(.*(d3-scale))/;
         // On Windows, mPath use backslashes for folder separators.  We need
         // to convert these to forward slashes because our
         // test regex, inclusionReg, contains one.
         const modulePath = mPath.replace(/\\/g, "/");

         if (inclusionReg.test(modulePath)) {

           // Don't need to see entire path in console
           const pathArray = modulePath.split(DIRNAMENIX);
           console.log("TCL: transpiling module: ", pathArray[1] || modulePath);
           return true;
         }
         return false;
       },
       test: /\.js$/,
       use: [
         {
           loader: "babel-loader",
           options: {
               presets: ["@babel/preset-env"]
             }
         },
       ],
     },

I find an include easier to get my head around than an exclude. Like @nowells, I also prefer to implement it as a function, at least during the dev phase. That way I can use a console.log() to track exactly which libraries are being picked up by the rule.

Note my inclusion Regex:

const inclusionReg = /node_modules\/(.*(d3-array))|(.*(d3-delaunay))|(.*(d3-sankey))|(.*(d3-scale))/;

This picks up the listed libraries no matter far down they're nested in node_modules; they may be in there as dependencies of dependencies, e.g.:

  • node_modules/d3-scale
  • node_modules/@nivo/colors/node_modules/d3-scale/
    and so on.

Finding which dependencies were causing our const errors in the first place took a bit of work. Here's a Regex that I paste into VSCode's search box when searching through our /build folder:

const(?:\s[^\s]+)\s{1,}(of|in)\s{1,}\w

You'll need to turn on Regex search in VSCode for this to work. It's the .* icon to the right of the search box. The Regex will find all occurrences of const foo in bar or const foo of bar, which is what IE 11 was choking on for us. (IE 11 actually supports const except for these two usages.)

@gottayan
Copy link

gottayan commented Jan 2, 2021

@borm: a solution:

{
  test: /\.js$/,
  exclude: /node_modules\/(?!(MY-MODULE|ANOTHER-ONE)\/).*/,
},

it's work!

why does it not work for this?
{
test: /.js$/,
use: ['babel-loader'],
exclude: /node_modules/(?!(cnchar|cnchar-trad)/).*/,
}

@jamesj2
Copy link

jamesj2 commented Jan 2, 2021

Try adding a backslash before the second to last forward slash.

exclude: /node_modules/(?!(cnchar|cnchar-trad)\/).*/,

@gottayan
Copy link

gottayan commented Jan 2, 2021 via email

@gottayan
Copy link

gottayan commented Jan 4, 2021

{ test: /.js$/, use: ['babel-loader'], exclude: /node_modules/(?!(cnchar|cnchar-trad)/)./ } yeat.I had changed for this,but it did not work too. could you give me a demo in the github? 发自我的iPhone

------------------ Original ------------------ From: James Johnson <[email protected]> Date: Sun,Jan 3,2021 2:43 AM To: webpack/webpack <[email protected]> Cc: gottayan <[email protected]>, Comment <[email protected]> Subject: Re: [webpack/webpack] How to exclude node_modules but one (#2031) Try adding a backslash before the second to last forward slash. exclude: /node_modules/(?!(cnchar|cnchar-trad)/).
/, — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

the regular expression is wrong.It can't match the package path in the node_modules.
the right one should be this.
/node_modules(?!/(.*cnchar|.cnchar-trad)/.)/.test('node_modules/_cnchar@2.2.9@cnchar/cnchar.min.js')

@imantahriri
Copy link

rules: [
  {
    test: /\.(js|jsx)$/,
    exclude: /[\\/]node_modules[\\/](?!(nanoid)[\\/])/,
    loader: 'babel-loader',
  }
]

This used to work like a charm on Webpack 4, but since migrating to Webpack 5 I get this error in the console:

Uncaught ReferenceError: module is not defined
    at eval (index.js:7)
    at eval (index.js:8)
    at Module../node_modules/nanoid/url-alphabet/index.js (bundle.js:3489)

I have been trying to fix it for a couple of days but I am running out of ideas now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests