Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modularize library #6

Open
uruuru opened this issue Sep 12, 2017 · 13 comments
Open

Modularize library #6

uruuru opened this issue Sep 12, 2017 · 13 comments

Comments

@uruuru
Copy link
Member

uruuru commented Sep 12, 2017

It would be nice to have have several modules instead of one large file: core, alg.layered, alg.stress, etc. However, I don't see any way to realize this with the current GWT compilation solution.

Currently the library is about 1.3 MB in size (minified); about 600 kB for core and layered, and 100 kB for the remaining algorithms.

@tonatos
Copy link

tonatos commented Mar 2, 2018

same issue!))

@uruuru
Copy link
Member Author

uruuru commented Jan 17, 2020

An update:

There's the j2cl project which seems promising to me. After all the GWT transpiler itself is more or less discontinued (future GWT will be somewhat based on j2cl) and doesn't properly support newer Java versions.

Thus, elkjs has to transition to a different transpiler anyway. However, getting everything to work (if possible) with j2cl seems to be quite some effort.

@Nic30
Copy link

Nic30 commented Jul 16, 2020

I am using elkjs + d3.js and the minified bundle is 2.1MB in size. If I use elk in widget for jupyter notebook I am gettitng around 8MB of minified javascript on nearly empty page.
elkjs also significantly slows down webpack build.
Another problem is webworker-threads it's getting deprecated and is unreliable.

@uruuru
Copy link
Member Author

uruuru commented Jul 16, 2020

Unfortunately I don't see a modularized version of the algorithms itself any time soon. In addition to the necessity that the transpiler supports it, I see several further difficulties.

I'll look into the webworker-threads point though - I wasn't aware of the deprecation yet. It's an optional dependency, though.

In which way does it slow down the webpack build? Without being a webpack expert, can't you exclude elkjs from webpack itself (in particular, if you use a webworker)? I believe we do it like that in https://github.com/kieler/elkgraph-web

@Nic30
Copy link

Nic30 commented Jul 16, 2020

can't you exclude elkjs from webpack itself

In normal build yes. However I do not know how to do it in bundle for jupyter widget (and I do not think it is possible)

https://github.com/Nic30/jupyter_widget_hwt/blob/master/js/webpack.config.js#L39

I guess that it takes 90s just because it is too big. https://travis-ci.org/github/Nic30/jupyter_widget_hwt/builds/708874010
It is not a big problem unless it is in docker on Binder which is my case. https://mybinder.org/v2/gh/Nic30/jupyter_widget_hwt.git/master?filepath=examples%2Fexample_simple.ipynb

@BoykoAlex
Copy link

Any chance instead of modularizing we could have elkjs-layered for example? Same as klayjs with one layered algorithm only but based on ELK... The size of this becomes 550Kb and rather straight-forward to accomplish it seems... What do you think?

@uruuru
Copy link
Member Author

uruuru commented Aug 25, 2020

I guess I could do that.

So far I was still hoping to find a proper solution instead of simply publishing duplicate builds. It still requires some effort though to properly automate the process.

EDIT: I just double-checked the resulting file size. Did you achieve the 550kB with a build yourself or are you referring to my original comment, which seems inaccurate? For me removing all algorithms despite layered yields 1.3MB (instead of 1.4). This wouldn't justify the effort in my eyes (for layered at least).

@BoykoAlex
Copy link

BoykoAlex commented Aug 25, 2020

Hmm... i thought i got 550Kb but i was mistaken it is 1.3Mb :-(
I did build it myself by excluding other algorithms in the build.gradle...

Makes me wonder how klayjs is 500Kb. Is it the EMF, Xtext, Guava dependencies that make elk.bundled.js so heavy?

@uruuru
Copy link
Member Author

uruuru commented Aug 25, 2020

Regarding klayjs:

  • guava should already have been used there as well
  • xtext shouldn't add any significant size in this case (it's only required during compile time to generate code from the melk files)

EMF is the largest factor, I presume. Also, ELK layered itself continuously increased in size.

I thought about removing EMF from ELK itself for several reasons but that's probably at least a month's worth of effort.

@uruuru
Copy link
Member Author

uruuru commented Nov 14, 2020

As to the original intent of this issue:

By now, several bytecode to js/webassembly transpilers have emerged: bytecoder, TeaVM, JWebAssembly, to name what I'm aware of so far.

I could imagine them being a viable alternative for the future, especially, since they allow compilation to web assembly as well.

@redexp
Copy link

redexp commented Oct 30, 2022

I wrote few scripts, which removes functions declarations from elk-worker.js which are not used in my project. I'v got 2.1 MB of source file (comparing to 4.5 MB of original file) and 800 KB of minified file (I think it could be better if spend more time on minifier options).

To make it possible I went through next steps:

  1. Run first script which adds to each function body line like $used[123] = true; (123 is id of this function) and creates new file like elk-worker-dev.js
  2. Run this file in real project
  3. Get keys from object $used and pass them to second script which will remove all functions which not in list and creates new file like elk-worker-clear.js
  4. Run this file in real project and see if there any errors. If it is then go to step 2 and try to use your project more meticulously

I can share those two scripts if you interested

@joryphillips
Copy link

@redexp Can you share your scripts? I would like to give that a try. Thank you!

@redexp
Copy link

redexp commented Feb 16, 2023

@redexp Can you share your scripts? I would like to give that a try. Thank you!

https://gist.github.com/redexp/7d4e7b7731c85e4ce8f29c26cb234fad

first npm install -D recast esprima-next

then run build-dev.js it will create elk-worker-dev.js with extra code which will every second check is array of used functions from elk is changed, if it is then it will log it to console. This array you should write to used.json and run build-clear.js. Tested on elkjs 0.8.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants