Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scancode does not start on windows exec #1129

Closed
jeffmcaffer opened this issue Jul 15, 2018 · 7 comments
Closed

scancode does not start on windows exec #1129

jeffmcaffer opened this issue Jul 15, 2018 · 7 comments

Comments

@jeffmcaffer
Copy link

jeffmcaffer commented Jul 15, 2018

I exec'ing "c:\install\scancode-toolkit-2.9.2\scancode" from node fails looking for "c:\install\scancode-toolkit-2.9.2\bin\scancode" (note the "bin" segment. Seems that the bin folder is actually a symlink to Scripts. Using normal windows tools (i.e., drag and drop) to unzip the Release zip, you get a windows Junction for bin. That is fine in the UI and from the Command shell but when exec'ing from Node, it seems like the junction is not being followed (or is not available). I don't know enough about the internal workings of node and exec to know why this would be different.

Since the "Scripts" folder seems to include all the executable stuff, can it just called bin rather than using a symlink?

as a workaround right now I just deleted the bin link, and copied Scripts to bin. Not sure if there has to be something called Scripts. (i.e., could I have just renamed?)

@pombredanne FWIW, this is for ClearlyDefined.

@pombredanne
Copy link
Member

@jeffmcaffer I do not know if node honours Windows junctions or not: it could be also that node may be less tested on Windows than POSIX

Now the reason why ScanCode generates a junction on first use (e.g. on the initial configure) is that the Python machinery that creates the bin and Scripts (e.g. setuptools) actually does generate a bin on POSIX and a Scripts dir on Windows. As a convenience, having a junction helps have the same paths referenced in help and documentation on every OS: e.g. bin.

These (the Scripts and the bin/junctions) are created either:

  • on the first run of a new installation
  • when calling the root configure.bat script (and this can be a combo of configure --clean then configure )

Can you point me to where in ClearlyDefined this happens?
Unless there is a bug in node I cannot fathom what happens exactly here.

Was this on the first run? or on other runs?
Was the 2.9.2 of scan a fresh extract in a clean directory? or was it overwritten on top of a previous installation?

@jeffmcaffer
Copy link
Author

Thanks @pombredanne. this is all runs of a freshly installed 2.9.2 (was also happening on 2.9.1). This happens in ClearlyDefined if you setup the crawler. If you unzip scancode in c:\installs (default) and run, then queue a request by POSTing something like

curl -d '{"type":"npm", "url":"cd:/npm/npmjs/-/redie/0.3.0"}' -H "Content-Type: application/json" -H "X-token: secret" -X POST http://localhost:5000/requests

That will trigger the crawler to get the npm and its source iteratively and run scancode (and other tools).

See https://github.com/clearlydefined/crawler/blob/master/providers/process/scancode.js#L71 for details of the exec call in node.

Stepping back from the how/why, Is it possible to have the init code just make a copy or rename of Scripts to bin when setting up on Windows? I get that a symlink is easier but it has these problems.

@pombredanne
Copy link
Member

Stepping back from the how/why, Is it possible to have the init code just make a copy or rename of Scripts to bin when setting up on Windows? I get that a symlink is easier but it has these problems.

The Python tools expect things on Windows to be under Scripts so using a copy instead of a link is likely to get things broken in many other use cases.

Now one way could be to still have a junction as a convenience but not use it in the root "scancode" script and reference \Scripts on windows and not the bin\junction.

@pombredanne
Copy link
Member

@jeffmcaffer one thing that I do not understand is why would any of this matters to node.js?
The use of the bin->Scripts junction is only done by the scancode.bat script. And node exec has pass over control to scancode.bat when this failure occurs IMHO.

if it is failing then this means that the junction was not created in the first place possibly. Do you have a more complete failure/stack trace log of sorts?

@jeffmcaffer
Copy link
Author

I don't know either but it just fails trying to run .../bin/scancode.exe. It is not a stack trace per se, just the exec gets an error. Are you able to reproduce? I'll see if anyone in the hackathon today can repo.

pombredanne added a commit that referenced this issue Jul 20, 2018
This hass likely little or no impact

Signed-off-by: Philippe Ombredanne <[email protected]>
pombredanne added a commit that referenced this issue Jul 20, 2018
This hass likely little or no impact

Signed-off-by: Philippe Ombredanne <[email protected]>
pombredanne added a commit that referenced this issue Jul 23, 2018
Use Scripts dir rather than bin junction #1129
@pombredanne
Copy link
Member

@jeffmcaffer the latest develop branch and tag 2.9.7 should have this resolved but I do not have an easy to to test this on hand. Can you check if this works now?

@pombredanne
Copy link
Member

This has been fixed so I am closing short of other feedback. Please reopen if this is still an issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants