Skip to content

Treat action code as attachments for created/updated actions#3286

Closed
dubee wants to merge 1 commit into
apache:masterfrom
dubee:code-attachments-all
Closed

Treat action code as attachments for created/updated actions#3286
dubee wants to merge 1 commit into
apache:masterfrom
dubee:code-attachments-all

Conversation

@dubee

@dubee dubee commented Feb 14, 2018

Copy link
Copy Markdown
Member

Supersedes #2847.

@dubee dubee added the wip label Feb 14, 2018
@markusthoemmes

Copy link
Copy Markdown
Contributor

@dubeejw can we close this PR? You can setup travis on your own repository if you want to have full travis coverage.

@dubee dubee force-pushed the code-attachments-all branch 3 times, most recently from 8be9ae8 to 94b4684 Compare April 19, 2018 17:09
@dubee dubee force-pushed the code-attachments-all branch from 94b4684 to 47d5657 Compare May 1, 2018 16:05
@dubee dubee changed the title Treat action code as attachments for all runtimes (Ignore for now) Treat action code as attachments for all runtimes and cache attachments (Ignore for now) May 1, 2018
@dubee dubee changed the title Treat action code as attachments for all runtimes and cache attachments (Ignore for now) Treat action code as attachments for all runtimes and cache attachments May 1, 2018
@dubee

dubee commented May 1, 2018

Copy link
Copy Markdown
Member Author

@rabbah, @markusthoemmes, a cache concurrency test fails on this one. I have figured out the failure is due the need of two DB writes for saving attachments. Where the first write is for the DB document and the second write is for the DB document attachment.

When only caching Java actions as done in #2855 and implementing the same concurrency test for Java actions, the test will fail as well.

Failure output shown here. The test fails because update does two writes as stated above, so a concurrent get request can fail as update does not finish as quickly for attachments.

whisk.core.database.test.CacheConcurrencyTests > the cache should support concurrent CRUD without bogus residual cache entries, iter 1 FAILED
    org.scalatest.exceptions.TestFailedException: update+get: failed for testy4 404 was not equal to 200

Any suggestions?

I did some hacking to only do a put on the DB document if it did not already exist. This allowed the test above to pass, but does not account for the document itself being updated. For instance, if a memory limit is updated from a REST call the document wouldn't be updated.

@dubee

dubee commented May 1, 2018

Copy link
Copy Markdown
Member Author

I suppose another option would be to change the test to accept a 200 or 404 from the get requests.

@codecov-io

codecov-io commented May 1, 2018

Copy link
Copy Markdown

Codecov Report

Merging #3286 into master will decrease coverage by 0.13%.
The diff coverage is 90%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3286      +/-   ##
==========================================
- Coverage   74.59%   74.45%   -0.14%     
==========================================
  Files         126      126              
  Lines        5982     5961      -21     
  Branches      397      384      -13     
==========================================
- Hits         4462     4438      -24     
- Misses       1520     1523       +3
Impacted Files Coverage Δ
...in/scala/whisk/core/database/DocumentFactory.scala 89.47% <100%> (ø) ⬆️
...src/main/scala/whisk/core/entity/WhiskAction.scala 77.62% <100%> (ø) ⬆️
.../scala/src/main/scala/whisk/core/entity/Exec.scala 79.37% <81.81%> (-4.5%) ⬇️
...rc/main/scala/whisk/core/controller/RestAPIs.scala 41.89% <0%> (-4.95%) ⬇️
...cala/src/main/scala/whisk/http/ErrorResponse.scala 88.5% <0%> (-1.27%) ⬇️
...rc/main/scala/whisk/core/controller/Triggers.scala 79.85% <0%> (-0.87%) ⬇️
...rc/main/scala/whisk/core/controller/Packages.scala 88.88% <0%> (-0.44%) ⬇️
...n/scala/whisk/core/database/CouchDbRestStore.scala 69.37% <0%> (-0.38%) ⬇️
... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6a3d6bd...b415146. Read the comment docs.

@dubee dubee added review Review for this PR has been requested and yet needs to be done. and removed wip labels May 2, 2018
@dubee dubee force-pushed the code-attachments-all branch from 3407e26 to 94b72b2 Compare May 10, 2018 14:06
@dubee dubee changed the title Treat action code as attachments for all runtimes and cache attachments Treat action code as attachments for created/updated actions May 10, 2018
@dubee

dubee commented May 10, 2018

Copy link
Copy Markdown
Member Author

Closing #2847 in favor of this PR as it was easier to rebase with the merged caching changes.

@dubee dubee force-pushed the code-attachments-all branch 2 times, most recently from 9c14a00 to faccc80 Compare May 11, 2018 15:05
@dubee dubee force-pushed the code-attachments-all branch from faccc80 to b415146 Compare May 11, 2018 16:56
@dubee

dubee commented Jun 11, 2018

Copy link
Copy Markdown
Member Author

@chetanmeh, do you think these changes will be useful in the future?

@chetanmeh

Copy link
Copy Markdown
Member

@dubee Yes this would be useful as document size limitation is impacting nodejs kind also for us. So would like to convert all types to attachments. With support for Inlined attachments #3709 switching to attachment should not have negative impact on small functions.

@chetanmeh

chetanmeh commented Jul 7, 2018

Copy link
Copy Markdown
Member

@dubee Do you plan to work on this otherwise I can look into it. Given recent changes in master the ArtifactStore does not make use of attachmentName attribute from manifest. So was thinking to do away with manifest entry for name attribute.

For attachment type I am not sure - For e.g. if its a simple code snippet then type can be text/plain otherwise if its a zip (say js code packaged as zip) then it would be application/octet-stream.

Further with inlining support now we can always store the code for all kinds as attachments by design and let ArtifactStore inline the binary if its small. So no need to differentiate between attachment for various kinds.

Any such change should be done in a backward compatible way to ensure existing actions work as expected

@chetanmeh

Copy link
Copy Markdown
Member

@dubee Ping

@markusthoemmes

Copy link
Copy Markdown
Contributor

@dubee any news on this one?

@chetanmeh chetanmeh self-assigned this Aug 6, 2018
@chetanmeh

Copy link
Copy Markdown
Member

PR #3945 supersedes this one

@chetanmeh chetanmeh closed this Aug 6, 2018
@chetanmeh chetanmeh mentioned this pull request Aug 7, 2018
26 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review Review for this PR has been requested and yet needs to be done.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants