These are some issues we see frequently. Maybe they’ll save you the effort of typing up an email to [email protected]
There are many reasons this could happen. To find yours, start by reading Troubleshooting missing coverage reports below.
Keep reading to see if you can find your error message below. If not, jump to Troubleshooting errors for next steps.
The first thing to check is whether you’ve given the Coveralls bot access to your repo. To do that, make sure you follow the instructions here.
If you’re sure you’ve given the Coveralls bot access to your repo, then the most common reason for a missing PR Comment is that Coveralls couldn’t generate the comment in the first place.
This can happen when Coveralls can’t find the base build for your PR build. The base build is the Coveralls build for the commit on which your PR is based and, likely, to which it will be merged. This is usually a push build on your default branch. So if that’s missing—for instance, if you’re not building push builds in CI—then you’ll want to make sure you start building those.
The easiest way to tell if your base build is missing is that the coverage table at the top of your PR build will read, “FIRST BUILD ON <BRANCH> at <COVERAGE %>”
The most common cause of this issue is lack of access permissions.
Is it possible the owner of your repo left your organization? Your repo’s “owner” is the user who added the repo to coveralls.io. Coveralls leverages the API access token of your repo’s owner to perform certain API-related actions, including sending status updates.
How do I change my repo owner?
Email us at [email protected] and we’ll be happy to switch your repo owner. The new owner will just need to be a user that: (a) has access to your repo, and (b) has logged into coveralls.io at least once (so they’re in our database).
Currently, there’s no way to change the owner of your repo via the Coveralls Web UI.
To avoid this problem in the future, we recommend adding repos to Coveralls with a service account—a generic account for your organization to which more than one user has access.
Being the “publishable” coverage for your repo, your badge is meant to represent coverage for the production (or production-ready) branch of your project. To fix this problem, simply submit coverage for at least one push build on your repo’s default branch.
This is usually due to overzealous caching at Github, or other repo hosts. It doesn’t happen as often as it used to but, if it’s happening for you, previous solutions include adding a query string parameter to your badge URL and removing the image from Github’s cache.
This UI-based error is not exactly what it seems. It is an error—but a temporary one that will go away in time with no further action.
What it indicates is that the build you’re trying to view has not yet completed.
To complete a build, Coveralls performs a series of background jobs, which include the job that calculates the build’s final, aggregate coverage, as well as some jobs that render interactive UI components on your build page. These jobs sit in background queues, waiting to run, which can back up when traffic spikes.
The “DataTables” error is an error thrown by one of those UI components—typically, the TREE view of your SOURCE FILES table—when it can’t get all the data it needs to render itself.
Rest assured that if your build is showing this error it will eventually complete and your build page will render without further errors of this kind.
But it’s happening a lot!
When this error occurs frequently, and across mutliple builds, it may indicate a general performance slowdown at coveralls.io.
If you find yourself waiting longer than a few minutes for your build pages to render, check our status page for updates. If you don’t see anything there, but your builds continually take longer than 5-10 minutes to complete, email us at [email protected] and we’ll look into it for you.
Similar to the case above, you might be experiencing a temporary slowdown at coveralls.io.
Check our status page for updates. If there are none, give it some time and see if build times don’t improve over the next 30-minutes or so.
Coveralls Cloud is designed to adjust its own infrastructure to address traffic. These adjustments can take a little time, but when they respond, performance should improve quickly and build times should decrease.
To understand what’s happening behind-the-scenes with your builds—what accounts for the time they take to complete—see the explanation above for, I’m getting a “DataTables” error.
To learn what you might be able to do to improve your build times, see How can I speed up my build times? in our FAQ.
As you might have guessed, this error means Coveralls couldn’t find a repo matching the identifier you passed in your POST to the Coveralls API.
If the integration you’re using happens to be the Coveralls Github Action see the next issue for why this might be happening.
If you’re using any other integration besides the Coveralls Github Action, you’ll want to check to make sure you’re passing your repo’s current, correct Coveralls Repo Token, which you can find on the Start Page for your Coveralls repo, or in its Settings. You may discover that a teammate has cycled the token.
Alternately, sometimes new users confuse their Coveralls Repo Token for their Personal Access Token (PAT) found on each user’s Coveralls account page.
If you’re getting this error and you’re using the Coveralls Github Action, the most likely cause is that you’re not passing your repo’s GITHUB_TOKEN as its identifier. Likely, you’re passing its Coveralls Repo Token, which is understandable.
The Coveralls Github Action is one of the only Coveralls integrations that breaks the pattern of using the Coveralls Repo Token to identify your repo to the Coveralls API. Since this integration uses an underlying Github App, it makes use of built-in Github environment variables, called secrets. Specifically, it leverages ${{ secrets.GITHUB_TOKEN }} as your repo identifier.
See this usage example from the Coveralls Github Action README for how to pass this value in your Github Actions config yaml.
Perhaps the least helpful error, a 500 error indicates an unexplained failure with some server in a chain of server requests, including the servers hosting the Coveralls API.
If it seems likely the error is being returned from the Coveralls API, the most likely culprit is malformed JSON in your coverage report—in other words, in the final JSON file packaged up and posted to the Coveralls API by your Coveralls integration.
The error itself provides few clues about its cause, but if you’ve re-run your build in verbose mode and don’t see anything obviously wrong with the JSON you’re posting to Coveralls, email us at [email protected] with your verbose build log and we’ll do our best to help identify the root cause.
Most often this error is caused by a temporary condition that’s soon remedied. The first remedy is to try again.
Next, check our status page. If everything looks fine there, but this error is not going away for you, read on to see what else may be causing the problem.
This error is sometimes persistent for extremely large projects.
Our CDN has a single file limit of 100MB, so if you think you might be posting coverage reports larger than that to the /jobs endpoint of the Coveralls API, it’s worth verifying that, and, potentially, breaking your single post up into multiple, smaller jobs.
While rare, we’ve also seen this error appear during web requests—but usually only for projects that are extreme outliers in terms of their number of files, or number of parallel jobs.
If you’re consistently seeing this error for one of your projects, email us at [email protected] and we’ll try to help.
Look at the top of your build page.
Under TOTAL FILES, is the number shown there also zero (0)?
If so, then the issue is that your integration can’t find your coverage report.
You can verify this by re-running your build in verbose mode and examining the JSON your integration is sending to the Coveralls API. If this is your problem, you’ll be passing an empty array ([]) for the source_files parameter.
The most likely cause for this—when you know your build is generating a coverage report and saving it to the right CI directory—is that there’s a difference between the file paths listed in your coverage report (LCOV) and paths to the same files relative to the root directory of your repo.
If you think this is the case for your project, and you’re using the Coveralls Github Action, try using the base-path parameter to pass the correct base path to your files. (Read more about that here.)
If you’re not using the Coveralls Github Action, you’ll need to find a different way to get your coverage report to display the correct paths to your source files. Options:
Did you already configure your CI for parallel builds?
As part of that, did you inform Coveralls that your build is parallel?
How do your builds vary from expectation?
Are you missing jobs? Or does coverage just seem off?
If coverage seems off, see Coveralls doesn’t seem to merge my coverage correctly below.
If you’re missing jobs, keep reading.
If your builds only show one job, the first one, the most likely problem is that Coveralls doesn’t know your build is parallel. In other words, in the JSON you’re passing to the Coveralls API, the parallel parameter is false, null or missing. You’ll want to make sure you’re passing, parallel=true.
If your builds only show one job, the last one, the most likely problem is that all your jobs are getting the same name. When Coveralls receives two jobs that have the same service number, index (or flag-name) from CI, it replaces the first job with the second (and so on).
Is it clear that one or more coverage reports are missing?
Or does your aggregate coverage % just seem off?
If there’s a big distinction, as though one or more reports is missing, see Coveralls fails to merge my (parallel) reports.
If coverage just seems off—say by one or two percentage points—see I’m seeing some discrepancies in coverage for why that might be.
Is the discrepancy occuring between two builds? Or within a single build?
If it’s between two builds:
Are the builds two push builds? Or is one of the builds a pull_request based on the other?
In both of these cases, sussing out the source of a discrepancy can take a little digging. Our first suggestion is to look at both builds side-by-side.
If both builds are push builds:
Once you’ve got the builds side-by-side, pay special attention to the RUN DETAILS table in each build and how the information there differs between the two builds.
Next, use the LIST view in each build’s SOURCE FILES table to compare changes to individual source files. (Use the search field to find specific files if your project has a lot of files.) See the CHANGED, SOURCE CHANGED and COVERAGE CHANGED tabs in that table and note the small numbers in red and green between columns indicating changes in RELEVANT, COVERED and MISSED lines. Sometimes, the discrepancies will lie there.
If one of the builds is a pull_request:
First, make sure the other build is the base build of the pull_request build. You can do this by making sure the pull_request build page contains a “PR Base” link to that build in its upper left hand corner.
In the pull_request build, look through the DIFFS section, which contains all files with coverage changes between the two builds—specifically, any files with UNCOVERED EXISTING LINES and any files with NEW UNCOVERED LINES. Make sure those details make sense to you.
If the discrepancies are within a single build:
Is the issue that the coverage of certain files and folders in the SOURCE FILES table don’t seem to add up to the aggregate coverage for the build? Or that they don’t seem to average out correctly?
If so, the first thing to check is whether your project is tracking branch coverage.
If your project tracks branch coverage, the source of the discrepancy may be that, while your overall (aggregate) coverage is a blend of line coverage and branch coverage, the numbers in your SOURCE FILES table only relate to line coverage.
You can find out if your project is tracking branch coverage by viewing RUN DETAILS. If it does, there will be a section there with numbers for branch coverage.
Tip: You can ignore the branch coverage in your incoming coverage reports by disabling the following setting: BRANCH COVERAGE, INCLUDE IN AGGREGATE COVERAGE %.
This usually means you aren’t calling the parallel build webhook—aka. closing the build—as the last step in your CI config.
Either that, or Coveralls can’t close your build from the values you’re passing.
Along with your call to http://coveralls.io/api/v1/webhook you should be passing the following two parameters: payload[build_num]=<your build number> and payload[status]=done.
Check your CI build log to see if Coveralls threw an error like, “build not found.” If so, try re-running your build in verbose mode and make sure the value you’re passing for payload[build_num] matches the build number for all your previous jobs.
If your build failed silently—Coveralls didn’t throw an error but the build still didn’t close—you may not have passed the other parameter, which is required: payload[status]=done.
Note: Many Coveralls integrations have been designed to work seamlessly with common CI systems, such that, for many, all that’s required to send your parallel build webhook is a statement like this:
notifications:
webhooks: https://coveralls.io/webhook
That’s because the integration is sourcing the required parameters from the CI’s environment variables.
If you’re using a less popular CI, or a custom CI, you may need to be explicit—perhaps by using a command line equivalent, like:
curl -k $COVERALLS_ENDPOINT/webhook?repo_token=$COVERALLS_REPO_TOKEN -d "payload[build_num]=$BUILD_NUMBER&payload[status]=done"
Visit the ADD REPOS page and click the GITHUB SETTINGS button at the bottom of the page. Check the organization’s 3rd party access restrictions and approve Coveralls. More info here: https://help.github.com/articles/approving-third-party-applications-for-your-organization/
Have them visit the ADD REPOS page and click the REFRESH PRIVATE REPOS button at the bottom of the page. Or enter this URL in their browser: https://coveralls.io/refresh?private=true
If your membership of an organization is not publicized (your username isn’t listed on the organization’s Github page), Coveralls can’t see it either, so you’ll need to upgrade your OAuth scope. Visit the ADD REPOS page and click the REFRESH PRIVATE REPOS button, or enter this URL into your browser: https://coveralls.io/refresh?private=true
You need to have Admin permissions at Github in order to access and regenerate (“cycle”) a repo’s token at coveralls.io.
If you do have Admin permissions and our data is out of sync, visit the ADD REPOS page and click the SYNC REPOS button. Or enter this URL into your browser: https://coveralls.io/refresh
Head over to Coveralls’ public issues board and see if you can find it there.
The next place to look is the public issues board for whatever Coveralls Integration you’re using. Most of those projects are open-source and available at Github.
If that doesn’t help—but you want to continue exploring on your own—read through Troubleshooting, below.
Or send an email with the problem you’re experiencing to [email protected].
But before you do!…
Enable verbose mode!:
Read Put it in verbose!, below, to get the most mileage from your support requests.
These are some notes on how to troubleshoot issues with Coveralls. They’re progressive so it’s best to start at the top.
“Putting it in verbose” means enabling verbose mode for the Coveralls Integration you’re using.
Most integrations will have a verbose mode, which adds debug information to your build log (aka. stdout) that makes it much easier for us, and the integration’s authors, to help resolve your issue.
Jump to the full topic to find out how to enable verbose mode for your integration.
First, let’s make sure you’re looking in the right place:
{"message":"Job #28.1","url":"https://coveralls.io/jobs/632218569"}https://coveralls.io/<your repo host>/<your organization>/<your repo name>Check your CI build log:
If you still can’t find the coverage report you’re looking for, check your CI build log for an error.
Most CI services provide a Web-based UI with a section displaying the log for each of your builds. You’ll want to examine that log and look for any errors.
Read Troubleshooting your build log below for what to do when you find some.
Congratulations. An error message is the first step toward resolving your issue.
The first and most important question is: Where did the error occur?
Did you see it:
This is the first thing you’ll want to share with us—or with the author(s) of your integration—when seeking help for your issue.
If your error occured in the Coveralls Web UI, jump back to Common Issues, above. Chances are we address it there.
If not, share the error with us at [email protected]. (But follow these steps for best mileage.)
If, instead, your error occurred in your CI build log, or on the command line, read the next topic, Troubleshooting your build log.
As stated elsewhere:
Most CI services provide a Web-based UI with a section displaying the log for each of your builds. You’ll want to examine that log and look for any errors related to the coverage job you just posted.
First step:
The first thing you’ll want to do is determine, if you can, whether the error in your build log is coming from Coveralls, or from the integration you’re using to reach Coveralls. (Or, of course, from your own code, but we’ll just cover the first two options here.)
How to understand build log errors:
Good stuff:
When a job succeeds at Coveralls, the Coveralls API returns a 200 success style message in the form of a JSON object with some details about the new job. Like:
{"message":"Job #28.1","url":"https://coveralls.io/jobs/632218569"}
In this circumstance, Coveralls is in the process of constructing a build based on this job—and any other jobs that might accompany it if your build is a parallel build. That means you can follow the URL in the response to check on that job—and it’s parent, the build—in the Coveralls Web UI.
Bad stuff:
By contrast, when a job encounters an issue at Coveralls, instead of a 200 success response, the Coveralls API will return one of several API errors, typically with a status code and message. Like:
{"error": true, "status": 422, message: "JSON file not found or failed to parse.", "url": ""}
That error means the Coveralls API had trouble parsing, or possibly even finding, the JSON file your integration sent as a form file upload with your POST.
So, while this error originates from the Coveralls API, given the nature of the error message we understand the root cause may still reside in your integration’s code—or in your original coverage report. But, at least now we have a place to start.
API errors, once removed:
Some integrations are good enough to wrap responses from the Coveralls API in their own error messages, which are often more detailed and, therefore, more helpful.
So while the errors themselves may not look like the API errors above, the context should let you know the error originates from Coveralls API. For example:
Report submission to Coveralls API failed with HTTP status 422: Unprocessable Entity (Couldn't find a repository matching this job.)
Obviously, that error indicates Coveralls couldn’t find a repo matching the one referenced in the coverage job just posted. Typically, this points to an issue with the repo’s Coveralls repo token, which functions like an ID to the Coveralls API.
So this gives us a clue where to start. But, as it happens, sometimes there’s another reason the IDs don’t match, even when you’re using the right Coveralls repo token. (See the issue “Couldn’t find a repository matching this job” when using Github Actions above.)
Coveralls as culprit:
Once you know an error originates from Coveralls, you can search for it in common issues above, or search Coveralls’ public issues board for similar errors, report a new issue there, or email us at [email protected] with your error message.
But before you do—
Enable verbose mode!
Read Put it in verbose!, below, to get the most mileage from your support requests.
Culprit unknown:
On the other hand, when an error originates from your integration, or from any point in time before your job reached the Coveralls API, you might see almost any other type of error in your build log.
Often it’ll be in the form of a stack trace, like this:
/usr/lib/node_modules/coveralls/bin/coveralls.js:19
throw err;
^
Bad response: 500 {"message":"Build processing error.","error":true,"url":""}
(Use `node --trace-uncaught ...` to show where the exception was thrown)
section_end:1601944275:step_script
[0Ksection_start:1601944275:upload_artifacts_on_failure
[0K[0K[36;1mUploading artifacts for failed job[0;m
[0;m[32;1mUploading artifacts...[0;m
coverage/cobertura-coverage.xml: found 1 matching files and directories[0;m
Uploading artifacts as "cobertura" to coordinator... ok[0;m id[0;m=774176534 responseStatus[0;m=201 Created token[0;m=Xtz38XWJ
section_end:1601944277:upload_artifacts_on_failure
[0K[31;1mERROR: Job failed: exit code 1
The top of this stack references a line of code from an integration.
How do we know that? Because—most often—Coveralls throws API-style errors like the ones discussed above, including the catchall, 500 Internal Server Error, which we’ll discuss lower down this page.
Climbing the stack:
The top of such a stack trace might display an anticipated error—an error with a title or error class, with or without an accompanying message. Or it might display an ad hoc error, like the one above. In those cases, the best we may have to go on is the line of code referenced in the error. Which is fine, as any and all of the above may be helpful when searching for next steps.
Integration as culprit:
When an error originates from your integration, you’ll want to visit the project’s README and documention. Or, as most Coveralls Integrations are open-source, you could visit the project’s repo and check out the source code, or search its public issues. If that doesn’t help, submit a new issue in the project’s issues board.
If you’ve read through the common issues and troubleshooting steps here to no avail, you can always email us at [email protected] and we’ll do our best to help resolve your issue.
But when you do that, include this information for best mileage:
To find out how to enable verbose mode for your integration, read Put it in verbose below.
This advice is so helpful we listed it twice. So if you came here for troubleshooting tips but only have time for one, make it this one.
Your integration’s verbose mode is the first place to start when troubleshooting a failed build.
As stated elsewhere, most integrations have a verbose mode, which adds debug information to your build log (aka. stdout) that makes it much easier for us, and the integration’s authors, to help resolve your issues.
Verbose mode should include, among other details, the exact JSON object you’re sending to the Coveralls API.
For example, here’s some verbose output from the coveralls-ruby integration:
[debug] "2020-07-12T23:47:34.123Z" 'SF:./lib/class_one.rb\nDA:1,1\nDA:3,1\nDA:4,1\nDA:7,1\nDA:8,0\nend_of_record\n'
[debug] "2020-07-12T23:47:34.125Z" 'user options undefined'
[debug] "2020-07-12T23:47:34.127Z" 'No valid .coveralls.yml file found'
[debug] "2020-07-12T23:47:34.147Z" {
service_name: 'circleci',
service_number: 'adbbae7c-9944-48db-8625-a90b97da6517',
service_job_number: '204',
run_at: '2020-07-12T23:47:34.126Z',
repo_token: '*********************************',
git: {
head: {
id: 'c49e0350d031834b87ac5d2236fd605281193dc5',
committer_name: 'James Kessler',
committer_email: '[email protected]',
message: 'Finish coveralls setup.',
author_name: 'James Kessler',
author_email: '[email protected]'
},
branch: 'circle-test-2',
remotes: [ [Object] ]
}
}
[debug] "2020-07-12T23:47:34.148Z" 'in: ' ''
[info] "2020-07-12T23:47:34.150Z" 'sending this to coveralls.io: '
'{ "source_files": [
{
"name":"lib/class_one.rb",
"source":"class ClassOne\\n\\n def self.covered\\n \\"covered\\"\\n end\\n\\n def self.uncovered\\n \\"uncovered\\"\\n end\\n\\nend\\n",
"coverage": [1,null,1,1,null,null,1,0,null,null,null,null],
"branches": []
}
],
"git": {
"head": {
"id":"c49e0350d031834b87ac5d2236fd605281193dc5",
"committer_name":"James Kessler",
"committer_email":"[email protected]",
"message":"Finish coveralls setup.",
"author_name":"James Kessler",
"author_email":"[email protected]"
},
"branch":"circle-test-2",
"remotes": [
{
"name":"origin",
"url":"[email protected]:afinetooth/coveralls-demo-ruby.git"
}
]
},
"run_at":"2020-07-12T23:47:34.126Z",
"service_name":"circleci",
"service_number":"adbbae7c-9944-48db-8625-a90b97da6517",
"service_job_number":"204",
"repo_token":"*********************************"
}'
[debug] "2020-07-12T23:47:34.819Z" 200
[debug] "2020-07-12T23:47:34.819Z" '{"message":"Job #adbbae7c-9944-48db-8625-a90b97da6517.4","url":"https://coveralls.io/jobs/64975270"}'
Note the url in the final line—https://coveralls.io/jobs/64975270—which you can pop right into a browser to view the corresponding job at coveralls.io.
How to invoke verbose mode:
There are different ways to invoke verbose mode. The exact method depends on the integration you’re using.
CLI flags:
If you’re using the node-coveralls integration, you can pass the CLI flag, -v, or ––verbose, with the coveralls command when you call it, like so:
nyc report --reporter=text-lcov | coveralls --verbose
Environment variables:
If you’re using an integration that doesn’t provide a CLI, you can usually set an environment variable. (Which one depends on the library you’re using):
If you’re using the coveralls-ruby integration, you can set an environment variable called COVERALLS_NOISY in your CI environment, like this:
# travis.yml
env:
global:
- COVERALLS_NOISY=true
Or set it from the command line, like this:
COVERALLS_NOISY=true bundle exec rspec
Dedicated params:
Finally, some Coveralls integrations have built-in parameters to toggle verbose mode.
The Coveralls Orb for CircleCI specifies a verbose parameter for its coveralls/upload command, configured like this:
# circleci/config.yml
version: 2.1
orbs:
coveralls: coveralls/[email protected]
jobs:
build:
docker:
- image: 'circleci/node:10.0.0'
steps:
- checkout
- run:
name: Install and Make
command: npm install && make test-coverage
- coveralls/upload
verbose: true
Pro tip:
It’s a wise move to learn up front how to invoke verbose mode for any Coveralls Integration you’re considering using. It could save you hours—and it makes for interesting reading while dialing in your builds!
Even if you end up needing support, it’s always a good move to include your verbose build log with your support request.
What if my integration has no verbose mode?
Then you might be struggling a little longer to find the root cause of your issue.
Super pro tip?:
Here’s a great idea: If your integration doesn’t provide a verbose mode, why not add one yourself? It should be straightforward to add some debug output to the projec’t existing source code. Why not submit your changes as a PR so other users can benefit?
In fact, if you’ve done this, email us at [email protected] and let us know. We’ll show our appreciation with a free month of service!
Any problems, questions or comments about this doc? Let us know.