Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

please provide an API to get the entire expected and actual results #1713

Open
andyczerwonka opened this issue May 23, 2018 · 8 comments
Open

Comments

@andyczerwonka
Copy link

In order for tool providers (JetBrains Rider in this case) to provide diff tools, could you please provide an API to get the entire expected and actual results of a test?

@bradwilson
Copy link
Member

Where and when is your code running that you're looking for this API?

@andyczerwonka
Copy link
Author

As per the link in the request, the JetBrains Rider team would like to add a unit test diff tool similar to what they have in IntelliJ, but can't provide it without the API. See https://www.youtube.com/watch?v=R8l6D3qGSdg for an example of the tool working using JUnit.

@bradwilson
Copy link
Member

That doesn't really answer my questions as to what kind of an API you'd expect, and to be called at what stage during the process. Are you post-processing our XML file? Are you writing a test runner? etc.

@andyczerwonka
Copy link
Author

andyczerwonka commented May 23, 2018

Yes, it is extending the current runner in Rider. @gorohoroh, could you provide @bradwilson with some details on this please?

@gorohoroh
Copy link

I'd rather ask @AndreyAkinshin to provide any details and API expectations beyond what's covered in the comment.

@AndreyAkinshin
Copy link

@bradwilson, @andyczerwonka, @gorohoroh
Sorry for late reply. We have own runner which directly use managed API. I guess, the best option will be extending of the IFailureInformation interface, it should provide additional metadata which can be used in Rider. Specifically, we need 3 additional fields: the full actual value, the full expected value, and the "assert kind" (e.g., Equals, Contains, and so on).

@vldF
Copy link

vldF commented Dec 1, 2024

Any updates?

I think this feature is really needed by the compiler and tooling community (I'm in the community btw). There is really common pattern when some input data like a code is being processed and the result artefact (like code in the target language, intermediate representation, etc) is being compared with the expected one. In this case you need to understand where is that hucking extra space or comma

In the JVM world we already have JUnit which assertions have these fields. It is really useful, see video from andyczerwonka in the JB YouTrack

@bradwilson
Copy link
Member

@vldF This is not currently on our backlog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants