Skip to content
This repository was archived by the owner on Nov 1, 2023. It is now read-only.

Add extra_output container, rename extra container#3064

Merged
Porges merged 40 commits into
mainfrom
extra-rw-container
Jun 15, 2023
Merged

Add extra_output container, rename extra container#3064
Porges merged 40 commits into
mainfrom
extra-rw-container

Conversation

@Porges

@Porges Porges commented Apr 26, 2023

Copy link
Copy Markdown
Member

Summary of the Pull Request

  • Breaking (but as far as I know this feature is not yet in use): rename the extra_container to extra_setup_container.
  • Add: the extra_output_container, which pushes its outputs continually.
    • We may also want a type of container which both pushes & pulls? See discussion below.
  • Improved: if onefuzz-task fails upon launch, we will log its output for diagnosis (might close Failures happening before the task channel initialization are not reported #3113)

Some thoughts for the future:

We might want to redesign the containers so that we have something like the following which is passed to the agent, and the agent doesn't need to know the specifics of the containers supplied:

{
    // ...
    "containers": {
        "extra_setup_dir": {
            "mode": "pull",
            "container_name": "yyy",
        },
        "extra_output_dir": {
            "mode": "push",
            "continuous": true, // keep pushing while job is running
            "container_name": "xxx"
        }
    }
}

At the moment the agent needs to know what each container is for, for each task type. A more generic and flexible method might be simpler overall.

@codecov-commenter

codecov-commenter commented Apr 26, 2023

Copy link
Copy Markdown

Codecov Report

Merging #3064 (6ec83ae) into main (630b083) will decrease coverage by 0.05%.
The diff coverage is 4.01%.

@@            Coverage Diff             @@
##             main    #3064      +/-   ##
==========================================
- Coverage   29.62%   29.57%   -0.05%     
==========================================
  Files         332      332              
  Lines       38948    39056     +108     
==========================================
+ Hits        11537    11552      +15     
- Misses      27411    27504      +93     
Impacted Files Coverage Δ
src/ApiService/ApiService/OneFuzzTypes/Enums.cs 32.39% <ø> (ø)
src/ApiService/ApiService/onefuzzlib/Config.cs 0.00% <0.00%> (ø)
src/ApiService/ApiService/onefuzzlib/Defs.cs 0.00% <0.00%> (ø)
src/ApiService/ApiService/onefuzzlib/Scheduler.cs 14.51% <0.00%> (-0.66%) ⬇️
src/agent/onefuzz-agent/src/debug.rs 0.00% <0.00%> (ø)
src/agent/onefuzz-agent/src/setup.rs 0.00% <0.00%> (ø)
src/agent/onefuzz-agent/src/validations.rs 0.00% <0.00%> (ø)
src/agent/onefuzz-task/src/local/common.rs 0.00% <0.00%> (ø)
...rc/agent/onefuzz-task/src/local/libfuzzer_merge.rs 0.00% <0.00%> (ø)
...ent/onefuzz-task/src/local/libfuzzer_test_input.rs 0.00% <0.00%> (ø)
... and 28 more

@Porges Porges changed the title Adding "extra_rw" container Draft: Adding extra_rw container Apr 26, 2023
@Porges Porges force-pushed the extra-rw-container branch from f6688c5 to 72d76a5 Compare April 27, 2023 02:15
@Porges Porges force-pushed the extra-rw-container branch from 72d76a5 to c04872b Compare May 16, 2023 21:26
@Porges Porges force-pushed the extra-rw-container branch from c04872b to ef375c3 Compare May 16, 2023 22:02
@Porges Porges changed the title Draft: Adding extra_rw container Draft: Adding extra_synced container May 21, 2023
@Porges Porges changed the title Draft: Adding extra_synced container Draft: Adding extra_output container May 24, 2023
@Porges Porges marked this pull request as ready for review May 24, 2023 20:56
@Porges Porges changed the title Draft: Adding extra_output container Add extra_output container, rename extra container May 24, 2023
@Porges

Porges commented May 25, 2023

Copy link
Copy Markdown
Member Author

Code changes are ready for looking at but this needs its own integration test to be fully complete.

Comment thread src/ApiService/ApiService/OneFuzzTypes/Model.cs Outdated
Comment thread src/agent/onefuzz/src/expand.rs
@Porges Porges enabled auto-merge (squash) June 15, 2023 02:15
@Porges Porges merged commit aa54a15 into main Jun 15, 2023
@Porges Porges deleted the extra-rw-container branch June 15, 2023 21:01
@AdamL-Microsoft AdamL-Microsoft mentioned this pull request Jun 20, 2023
@kananb kananb mentioned this pull request Sep 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Failures happening before the task channel initialization are not reported

3 participants