Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to match certain webpack bundles #33

Open
mochaaP opened this issue Dec 21, 2023 · 13 comments · May be fixed by #50
Open

Unable to match certain webpack bundles #33

mochaaP opened this issue Dec 21, 2023 · 13 comments · May be fixed by #50
Labels
bug Something isn't working webpack

Comments

@mochaaP
Copy link

mochaaP commented Dec 21, 2023

Describe the bug

Could not recognize a certain webpack bundle.

Details

Webpack runtime requirements in the sample:

  • runtimeId = "__webpack_require__.j";
  • onChunksLoaded = "__webpack_require__.O";
  • nodeModuleDecorator = "__webpack_require__.nmd";
  • ensureChunkHandlers = "__webpack_require__.f";
  • hasOwnProperty = "__webpack_require__.o";
  • ensureChunk = "__webpack_require__.e";
  • exports = "__webpack_exports__";
  • getChunkScriptFilename = "__webpack_require__.u";
  • definePropertyGetters = "__webpack_require__.d";
  • compatGetDefaultExport = "__webpack_require__.n";
  • startup = "__webpack_require__.x";
  • moduleFactories = "__webpack_require__.m";

All wrapped in an IIFE.

Expected Behaviour

__webpack_require__ and __webpack_modules__ should be inferred from usage, and unpacked correctly

Code

https://fars.ee/FrBW.js

Logs

No response

@mochaaP mochaaP added the bug Something isn't working label Dec 21, 2023
@j4k0xb
Copy link
Owner

j4k0xb commented Dec 21, 2023

Fixed, it checks for a few more patterns like __webpack_require__.j = now and doesn't require an entry id anymore (__webpack_require__.s)
You can try it in the playground and the npm package will probably be updated in the next few days

@j4k0xb j4k0xb closed this as completed Dec 21, 2023
@mochaaP
Copy link
Author

mochaaP commented Dec 21, 2023

Another one spotted:

exports.id = 885;
exports.ids = [885];
exports.modules = {.../* webpack 5 modules */};

I wonder, is it possible for users to provide extra chunks in AST to support code splitting bundles?

@mochaaP
Copy link
Author

mochaaP commented Dec 21, 2023

@j4k0xb This is still unfinished, since entryId is not retrieved

@j4k0xb
Copy link
Owner

j4k0xb commented Dec 21, 2023

which one is it?
I haven't really looked at how multiple chunks work yet

@mochaaP
Copy link
Author

mochaaP commented Dec 21, 2023

sample.tar.gz

See vendor.bundle.js and .bundle.js

@j4k0xb j4k0xb reopened this Dec 21, 2023
@0xdevalias
Copy link

0xdevalias commented Dec 21, 2023

is it possible for users to provide extra chunks in AST to support code splitting bundles?

This is also something I would love to see, though it probably should be it's own issue.

I haven't really looked at how multiple chunks work yet

@j4k0xb Do you mean how they work in webpack/etc itself? Or just in implementing support for them in webcrack?

Not sure if this will be helpful, but this gist has some notes/references I captured while understanding more about it myself:

From a quick GitHub code search, these might be interesting too:

@mochaaP
Copy link
Author

mochaaP commented Dec 22, 2023

which one is it? I haven't really looked at how multiple chunks work yet

The matching logic is here:

(() => { 
  var installedChunks = {
    '377': 1
  };
  __webpack_require__.O.require = chunkId => installedChunks[chunkId];
  __webpack_require__.f.require = (chunkId, promises) => {
    if (!installedChunks[chunkId]) {
      (chunk => {
        var moreModules = chunk.modules;
        var chunkIds = chunk.ids;
        var runtime = chunk.runtime;
        for (var moduleId in moreModules) {
          if (__webpack_require__.o(moreModules, moduleId)) {
            __webpack_require__.m[moduleId] = moreModules[moduleId];
          }  
        }
        if (runtime) {
          runtime(__webpack_require__);
        }
        for (var i = 0; i < chunkIds.length; i++) {
          installedChunks[chunkIds[i]] = 1;
        }
        __webpack_require__.O();
      })(require(`./${__webpack_require__.u(chunkId)}`));
    }
  }; 
})();

@mochaaP
Copy link
Author

mochaaP commented Dec 22, 2023

We may also parse nested webpack chunks: https://github.com/webpack/webpack/tree/main/lib/CompatibilityPlugin.js

@mochaaP
Copy link
Author

mochaaP commented Dec 22, 2023

Should also match webpack ESM modules in SequenceExpression.

Sample: https://fars.ee/GIKB.js

@mochaaP
Copy link
Author

mochaaP commented Dec 22, 2023

unpacked_samples.tar.gz

Some unpacked samples for further inspection. Notice how some webpack runtime globals still got persisted (search for regex /require\./)

@j4k0xb j4k0xb added the webpack label Dec 22, 2023
@j4k0xb
Copy link
Owner

j4k0xb commented Dec 23, 2023

Should also match webpack ESM modules in SequenceExpression.

It already converts all kinds of sequences before in unminify
The modules are also converted to esm (more or less) if they're inside of a bundle and it removes require.r, require.d.
What's still left is require.j and module = require.nmd(module) and Object.defineProperty(exports, "__esModule", { value: true });

@j4k0xb
Copy link
Owner

j4k0xb commented Dec 24, 2023

Found another issue in that bundle (fars.ee/FrBW.js):

71017: module => {
  module.exports = require("path");
},

should not be transformed to:

module.exports = require( /*webcrack:missing*/"./path.js");

The code needs to be refactored to find references to the __webpack_require__ argument instead of blindly assuming that require(id) calls are the same. This should also result in better performance as it avoids traversing the whole AST.

/**
* Replaces `require(id)` calls with `require("./relative/path.js")` calls.
*/
private replaceRequirePaths() {
const requireId = m.capture(m.or(m.numericLiteral(), m.stringLiteral()));
const requireMatcher = m.or(
m.callExpression(m.identifier('require'), [requireId]),
);
const importId = m.capture(m.stringLiteral());
const importMatcher = m.importDeclaration(m.anything(), importId);
this.modules.forEach((module) => {
traverse(module.ast, {
'CallExpression|ImportDeclaration': (path) => {

@j4k0xb
Copy link
Owner

j4k0xb commented Jan 13, 2024

#50 / deploy-preview-50--webcrack.netlify.app contains a rewrite of the whole matching logic (very experimental)

  • It will find anything with this format instead of looking for 'e', 'd', 'j', 'm', 'r' etc properties which should be more resilient
var __webpack_modules__ = { ... };
// ...
function __webpack_require__(moduleId) {
  // ...
  __webpack_modules__[moduleId](module, module.exports, __webpack_require__);
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working webpack
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants