Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ambiguous message presented when URLs are duplicated or empty #436

Closed
artswie opened this issue Nov 21, 2018 · 2 comments
Closed

Ambiguous message presented when URLs are duplicated or empty #436

artswie opened this issue Nov 21, 2018 · 2 comments
Labels
bug help wanted Hey, contributor! We need your help. sum19

Comments

@artswie
Copy link

artswie commented Nov 21, 2018

If in single suite will exist empty or duplicated URLs (this might happen fairly easy on huge suites) you will get ambiguous messages while running "malformed" xml.

The message presented on jenkins on older version of aet (2.1.3):
[ERROR] Failed to process test suite: 'Not Found'

The message presented on jenkins on lastest version of aet (3.1.0):
[ERROR] Failed to schedule test suite: '[Status: 400] Bad Request

Please provide ERROR message which will be more informative OR prevent it in some other way.

Please run below code and unncomenty one of urls to see a failure.

<?xml version="1.0" encoding="UTF-8"?>
<suite name="testing" company="cognifide" project="cog" domain="https://www.cognifide.com">
  <test name="Accessibility" useProxy="rest">
    <collect>
	  <resolution width="1920"/>
      <open/>  
	  <accessibility standard="WCAG2AAA" />  
    </collect>
    <compare>
      <accessibility report-level="ERROR" />
    </compare>
	<urls>		
		<url href="/contact-us"/>
		<!--<url href=""/>-->
		<!--<url href="/contact-us"/>-->
	</urls>
	</test>
</suite>

Cheers!

@malaskowski malaskowski added bug help wanted Hey, contributor! We need your help. labels Nov 21, 2018
@radoslawkarwacki
Copy link
Contributor

radoslawkarwacki commented Dec 21, 2018

I looked into it and made a little improvement (will create pull request shortly).

The logs for this set of URLs:

<urls>      
    <url href=""/>
    <url href="/test0"/>
    <url href="/test0"/>
    <url href="/test1" name="name-1"/>
    <url href="/test1" name="name-1"/>
    <url href="/test2" name="name-2"/>
    <url href="/test2" name="name-3"/>
    <url href="/test3" description="description-1"/>
    <url href="/test3" description="description-1"/>
    <url href="/test4" description="description-2"/>
    <url href="/test4" description="description-3"/>
</urls>

Will be:

[Status: 400] Something is wrong with your suite definition! Detected errors: 
Empty url
Duplicated url, with URL: /test0, with name: /test0
Duplicated url, with URL: /test1, with name: name-1
Duplicated url, with URL: /test3, with name: /test3, with description: description-1
Duplicated url, with URL: /test4, with name: /test4, with description: description-3

Ideally, this should also contain line numbers. I didn't dig deep into it, but I'm not sure if the Simple XML Framework library can provide them. That should be considered as another improvement.

Also, please note that currently, entries with the same href and different names (href="/test2" here) are not treated as a duplicate. I'm not a frequent AET user, so I'm not sure if the name serves some other important purpose, but if not, then the logic should probably be changed to treat every entry with identical href as a duplicate.
The documentation states that "If provided it [the name] should be unique for each test in the test suite.". However, I don't know what would be the purpose of having the same URL entered in the suite with different names.

@annabudziszewska
Copy link
Collaborator

Tested and accepted
version: 3.3.1.SNAPSHOT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug help wanted Hey, contributor! We need your help. sum19
Projects
None yet
Development

No branches or pull requests

6 participants