Skip to content
This repository has been archived by the owner on Mar 8, 2020. It is now read-only.

Fix classes for backward compatibility #172

Merged
merged 1 commit into from
Jul 25, 2019
Merged

Fix classes for backward compatibility #172

merged 1 commit into from
Jul 25, 2019

Conversation

zurk
Copy link
Contributor

@zurk zurk commented Jul 23, 2019

Hey, LA team 👋

ML team found some time to check out new client (v3) you provide for us. Here a PR to make everything work as it was. src-d/ml-core#31 is the PR in ml-core that is waiting for a new release of bblfsh client.

It is more or less obvious what I changed and why, but ask me while reviewing if it is unclear. Every change is necessary or some test fails in ml-core`.


This change is Reviewable

@zurk
Copy link
Contributor Author

zurk commented Jul 23, 2019

I cannot assign reviewers, so
cc @creachadair

@creachadair creachadair requested review from creachadair and dennwc and removed request for creachadair July 23, 2019 15:03
Copy link
Contributor

@creachadair creachadair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks—this mostly looks fine, modulo one question I have. Also I'd like to get a second opinion from @dennwc.

Reviewed 3 of 3 files at r1.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @dennwc and @zurk)


bblfsh/node.py, line 61 at r1 (raw file):

        self._children = self._sync_children()

    def _sync_children(self) -> List["Node"]:

The fact that this didn't trigger errors before suggests we must not be checking type annotations in our tests.


bblfsh/node.py, line 247 at r1 (raw file):

            d["@pos"] = {
                "@type": "uast:Positions",
                "start": Node.DEFAULT_POSITION,

The problem with this change is that if the caller edits any one of the nodes that uses the default position, every node already extant with an alias of that dict will be effectively moved.

If you do want to consolidate the default, I suggest using a private function to generate a new instance of the default position each time. In any case, it probably doesn't need to be part of the public interface.

Copy link
Member

@dennwc dennwc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Cannot comment about type annotations though. Can be changed later on our side.

Reviewed 2 of 3 files at r1, 1 of 1 files at r2.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @zurk)

Copy link
Member

@dennwc dennwc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 2 files at r3.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @zurk)

Copy link
Member

@dennwc dennwc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @zurk)

a discussion (no related file):
Hmm, tests fail now in the CI.


@zurk
Copy link
Contributor Author

zurk commented Jul 24, 2019

yes, it was yesterday too. I just started to fix them, but I need your help because there is C extension involved and I am not familiar with it.

https://travis-ci.com/bblfsh/python-client/jobs/218999263#L332

iterator test fails. But if you change a position of the root node to 1 (root.start_position.offset = 1)
It works!

Any ideas what is wrong are welcome.

@zurk
Copy link
Contributor Author

zurk commented Jul 24, 2019

also If you take current master and change default positions from -1 to 0 (as it should be):

if "@pos" not in d:
d["@pos"] = {
"@type": "uast:Positions",
"start": {
"@type": "uast:Position",
"offset": -1,
"line": -1,
"col": -1,
},
"end": {
"@type": "uast:Position",
"offset": -1,
"line": -1,
"col": -1,

test also fails.

@zurk
Copy link
Contributor Author

zurk commented Jul 24, 2019

ok, I think this bug is unrelated to my changes and located somewhere in C extension code.
So, I fix the test to make it pass as it was.
If you want I can create an issue about the iterator I found.

Copy link
Member

@dennwc dennwc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r4.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @zurk)


bblfsh/node.py, line 260 at r4 (raw file):

        return {
            "@type": "uast:Position",
            "offset": 0,

On the second thought, it might not be a good idea to change offset from -1 to 0. It will make the position appear valid and break sorting and other things. line and col set to 0 are fine though - those are 1-based, thus 0 is considered an invalid value.

@zurk
Copy link
Contributor Author

zurk commented Jul 24, 2019

that is true, but when we have a protobuf and the node did not have positions all three values were zeros. I agree that it is not the best way to put things, but some of our code rely on (0,0,0) position.

UPD: I tried to change zeros to minus ones and indeed tests fail on our side in ml-core project.

Copy link
Member

@dennwc dennwc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @zurk)


bblfsh/node.py, line 260 at r4 (raw file):

Previously, dennwc (Denys Smirnov) wrote…

On the second thought, it might not be a good idea to change offset from -1 to 0. It will make the position appear valid and break sorting and other things. line and col set to 0 are fine though - those are 1-based, thus 0 is considered an invalid value.

OK, as long as those positions remain in the compatibility layer we may change them to 0 and update the test.

But instead of updating the position to 1, can you please change the expected order instead?

@zurk
Copy link
Contributor Author

zurk commented Jul 24, 2019

Ah! So you want to say that this testIteratorPositionOrder puts Nodes with invalid positions to the end? In this case, how iterator knows which position is invalid?

UPD: I changed the order.

Signed-off-by: Konstantin Slavnov <[email protected]>
Copy link
Member

@dennwc dennwc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! :lgtm: from my side, waiting for @creachadair.

Reviewed 1 of 1 files at r5.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @zurk)


bblfsh/node.py, line 260 at r4 (raw file):

Previously, dennwc (Denys Smirnov) wrote…

OK, as long as those positions remain in the compatibility layer we may change them to 0 and update the test.

But instead of updating the position to 1, can you please change the expected order instead?

Yes, exactly. It calls to libuast and SDK, where we check positions for a valid range. The 0 line and column is considered invalid, but it seems like it accepts -1 for offset, which is incorrect. Will open an issue for it.

@zurk
Copy link
Contributor Author

zurk commented Jul 24, 2019

Ok, waiting for a merge (if everything is fine) and a new release so I can use all these changes in ml-core. 🤞

@zurk
Copy link
Contributor Author

zurk commented Jul 24, 2019

@creachadair friendly ping :)

Copy link
Contributor

@creachadair creachadair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @zurk)

Copy link
Contributor Author

@zurk zurk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @dennwc)

a discussion (no related file):

Previously, dennwc (Denys Smirnov) wrote…

Hmm, tests fail now in the CI.

Done.



bblfsh/node.py, line 247 at r1 (raw file):

Previously, creachadair (M. J. Fromberger) wrote…

The problem with this change is that if the caller edits any one of the nodes that uses the default position, every node already extant with an alias of that dict will be effectively moved.

If you do want to consolidate the default, I suggest using a private function to generate a new instance of the default position each time. In any case, it probably doesn't need to be part of the public interface.

yes, totally makes sense.

@zurk
Copy link
Contributor Author

zurk commented Jul 25, 2019

Thanks! @dennwc Are we good to merge?
I am not familiar with a tool you use for a review but it is still pending

Copy link
Member

@dennwc dennwc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! all files reviewed, all discussions resolved

@dennwc dennwc merged commit 52b1936 into bblfsh:master Jul 25, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants