Skip to content

Fix heisenbug#5027

Merged
style95 merged 7 commits into
apache:masterfrom
style95:increase-assert-interval
Jan 18, 2021
Merged

Fix heisenbug#5027
style95 merged 7 commits into
apache:masterfrom
style95:increase-assert-interval

Conversation

@style95

@style95 style95 commented Nov 17, 2020 •
edited
Loading

Copy link
Copy Markdown
Member

This is to fix the intermittent test failures.

Description

Intermittent test failures due to heisenbugs increase the CI time and make authors run tests multiple times.

Related issue and scope

My changes affect the following components

  • API
  • Controller
  • Message Bus (e.g., Kafka)
  • Loadbalancer
  • Invoker
  • Intrinsic actions (e.g., sequences, conductors)
  • Data stores (e.g., CouchDB)
  • Tests
  • Deployment
  • CLI
  • General tooling
  • Documentation

Types of changes

  • Bug fix (generally a non-breaking change which closes an issue).
  • Enhancement or new feature (adds new functionality).
  • Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

  • I signed an Apache CLA.
  • I reviewed the style guides and followed the recommendations (Travis CI will check :).
  • I added tests to cover my changes.
  • My changes require further changes to the documentation.
  • I updated the documentation where necessary.

@style95 style95 added the wip label Nov 17, 2020
@style95 style95 force-pushed the increase-assert-interval branch from cc1c766 to 8cc41e8 Compare November 19, 2020 14:03
@codecov-io

codecov-io commented Nov 19, 2020 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

Merging #5027 (8f8d40f) into master (a202538) will decrease coverage by 6.66%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #5027      +/-   ##
==========================================
- Coverage   83.52%   76.85%   -6.67%     
==========================================
  Files         195      202       +7     
  Lines        9553     9818     +265     
  Branches      413      416       +3     
==========================================
- Hits         7979     7546     -433     
- Misses       1574     2272     +698
Impacted Files Coverage Δ
...core/database/cosmosdb/RxObservableImplicits.scala 0.00% <0.00%> (-100.00%) ⬇️
...ore/database/cosmosdb/cache/CacheInvalidator.scala 0.00% <0.00%> (-100.00%) ⬇️
...e/database/cosmosdb/cache/ChangeFeedConsumer.scala 0.00% <0.00%> (-100.00%) ⬇️
...core/database/cosmosdb/CosmosDBArtifactStore.scala 0.00% <0.00%> (-95.85%) ⬇️
...sk/core/database/cosmosdb/CosmosDBViewMapper.scala 0.00% <0.00%> (-93.90%) ⬇️
...tabase/cosmosdb/cache/CacheInvalidatorConfig.scala 0.00% <0.00%> (-92.31%) ⬇️
...enwhisk/connector/kafka/KamonMetricsReporter.scala 0.00% <0.00%> (-83.34%) ⬇️
...e/database/cosmosdb/cache/KafkaEventProducer.scala 0.00% <0.00%> (-78.58%) ⬇️
...whisk/core/database/cosmosdb/CosmosDBSupport.scala 0.00% <0.00%> (-74.08%) ⬇️
...ore/database/azblob/AzureBlobAttachmentStore.scala 11.53% <0.00%> (-60.58%) ⬇️
... and 23 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a202538...8f8d40f. Read the comment docs.

style95
style95 commented Nov 22, 2020
View reviewed changes
Comment thread tests/src/test/scala/org/apache/openwhisk/core/limits/ActionLimitsTests.scala

for (a <- 1 to 10) {
val run = wsk.action.invoke(name, Map("payload" -> "128".toJson))
val run = wsk.action.invoke(name, Map("payload" -> "128".toJson), blocking = true)

@style95 style95 Nov 22, 2020

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is supposed to reuse a container, so I change them to blocking calls.

@style95 style95 changed the title [WIP] Fix heisenbug Fix heisenbug Nov 23, 2020
@style95 style95 removed the wip label Nov 23, 2020
@style95

style95 commented Nov 23, 2020

Copy link
Copy Markdown
Member Author

I would re-run all tests multiple times and merge this if there is no further issue.

@style95

style95 commented Nov 23, 2020

Copy link
Copy Markdown
Member Author

Handling

org.apache.openwhisk.core.limits.ConcurrencyTests > Action concurrency limits should execute activations sequentially when concurrency = 1 STANDARD_ERROR
    org.scalatest.exceptions.TestFailedException: 502 was not equal to 200
    	at org.scalatest.MatchersHelper$.indicateFailure(MatchersHelper.scala:343)
    	at org.scalatest.Matchers$AnyShouldWrapper.shouldBe(Matchers.scala:6919)
    	at common.rest.RunRestCmd.validateStatusCode(WskRestOperations.scala:1165)
    	at common.rest.RunRestCmd.validateStatusCode$(WskRestOperations.scala:1162)
    	at common.rest.RestActionOperations.validateStatusCode(WskRestOperations.scala:244)
    	at common.rest.RunRestCmd.invokeAction(WskRestOperations.scala:1358)
    	at common.rest.RunRestCmd.invokeAction$(WskRestOperations.scala:1331)
    	at common.rest.RestActionOperations.invoke(WskRestOperations.scala:388)
    	at org.apache.openwhisk.core.limits.ConcurrencyTests.$anonfun$new$21(ConcurrencyTests.scala:137)
    	at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
    	at scala.util.Success.$anonfun$map$1(Try.scala:255)
    	at scala.util.Success.map(Try.scala:213)
    	at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
    	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
    	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
    	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
    	at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    	at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:92)
    	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
    	at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:92)
    	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
    	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:49)
    	at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    	at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    	at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    	at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
org.apache.openwhisk.core.limits.ConcurrencyTests > Action concurrency limits should execute activations sequentially when concurrency = 1 FAILED
    org.scalatest.exceptions.TestFailedException: 502 was not equal to 200
        at org.scalatest.MatchersHelper$.indicateFailure(MatchersHelper.scala:343)
        at org.scalatest.Matchers$AnyShouldWrapper.shouldBe(Matchers.scala:6919)
        at common.rest.RunRestCmd.validateStatusCode(WskRestOperations.scala:1165)
        at common.rest.RunRestCmd.validateStatusCode$(WskRestOperations.scala:1162)
        at common.rest.RestActionOperations.validateStatusCode(WskRestOperations.scala:244)
        at common.rest.RunRestCmd.invokeAction(WskRestOperations.scala:1358)
        at common.rest.RunRestCmd.invokeAction$(WskRestOperations.scala:1331)
        at common.rest.RestActionOperations.invoke(WskRestOperations.scala:388)
        at org.apache.openwhisk.core.limits.ConcurrencyTests.$anonfun$new$21(ConcurrencyTests.scala:137)

@style95 style95 changed the title Fix heisenbug [WIP] Fix heisenbug Dec 1, 2020
@style95 style95 force-pushed the increase-assert-interval branch from bf128ad to 8f8d40f Compare January 15, 2021 05:30
@style95 style95 changed the title [WIP] Fix heisenbug Fix heisenbug Jan 18, 2021
@style95 style95 merged commit 2d0c8a7 into apache:master Jan 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rabbah rabbah rabbah approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@style95 @codecov-io @rabbah