-
Notifications
You must be signed in to change notification settings - Fork 465
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Patch: hf-pt-train-1.13 for gevent and deepseed #4358
Open
malav-shastri
wants to merge
10
commits into
aws:master
Choose a base branch
from
malav-shastri:hf-pt-train-1.13
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aws-deep-learning-containers-ci
bot
added
build
Reflects file change in build folder
huggingface
Reflects file change in huggingface folder
Size:XS
Determines the size of the PR
labels
Oct 21, 2024
This reverts commit 69ae0f2.
bencrabtree
previously approved these changes
Oct 21, 2024
JGuinegagne
previously approved these changes
Oct 21, 2024
malav-shastri
dismissed stale reviews from bencrabtree and JGuinegagne
via
October 25, 2024 19:27
6096a59
aws-deep-learning-containers-ci
bot
added
sagemaker_tests
test
Reflects file change in test folder
labels
Oct 25, 2024
Captainia
previously approved these changes
Oct 25, 2024
This PR has been marked stale as a result of being open for 30 days without activity or updates. Please remove the stale label or comment in order to keep this open, otherwise the PR will be closed in 5 days. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
build
Reflects file change in build folder
huggingface
Reflects file change in huggingface folder
no-pr-activity
sagemaker_tests
Size:XS
Determines the size of the PR
test
Reflects file change in test folder
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
GitHub Issue #, if available:
Note:
If merging this PR should also close the associated Issue, please also add that Issue # to the Linked Issues section on the right.
All PR's are checked weekly for staleness. This PR will be closed if not updated in 30 days.
Description
Tests run
tested with AIAlgo scanner
bb remotescan --image
NOTE: By default, docker builds are disabled. In order to build your container, please update dlc_developer_config.toml and specify the framework to build in "build_frameworks"
Confused on how to run tests? Try using the helper utility...
Assuming your remote is called
origin
(you can find out more withgit remote -v
)...python src/prepare_dlc_dev_environment.py -b </path/to/buildspec.yml> -cp origin
python src/prepare_dlc_dev_environment.py -b </path/to/buildspec.yml> -t sanity_tests -cp origin
python src/prepare_dlc_dev_environment.py -rcp origin
NOTE: If you are creating a PR for a new framework version, please ensure success of the standard, rc, and efa sagemaker remote tests by updating the dlc_developer_config.toml file:
Expand
sagemaker_remote_tests = true
sagemaker_efa_tests = true
sagemaker_rc_tests = true
Additionally, please run the sagemaker local tests in at least one revision:
sagemaker_local_tests = true
Formatting
black -l 100
on my code (formatting tool: https://black.readthedocs.io/en/stable/getting_started.html)DLC image/dockerfile
Builds to Execute
Expand
Fill out the template and click the checkbox of the builds you'd like to execute
Note: Replace with <X.Y> with the major.minor framework version (i.e. 2.2) you would like to start.
build_pytorch_training_<X.Y>_sm
build_pytorch_training_<X.Y>_ec2
build_pytorch_inference_<X.Y>_sm
build_pytorch_inference_<X.Y>_ec2
build_pytorch_inference_<X.Y>_graviton
build_tensorflow_training_<X.Y>_sm
build_tensorflow_training_<X.Y>_ec2
build_tensorflow_inference_<X.Y>_sm
build_tensorflow_inference_<X.Y>_ec2
build_tensorflow_inference_<X.Y>_graviton
Additional context
PR Checklist
Expand
NEURON/GRAVITON Testing Checklist
dlc_developer_config.toml
in my PR branch by settingneuron_mode = true
orgraviton_mode = true
Benchmark Testing Checklist
dlc_developer_config.toml
in my PR branch by settingec2_benchmark_tests = true
orsagemaker_benchmark_tests = true
Pytest Marker Checklist
Expand
@pytest.mark.model("<model-type>")
to the new tests which I have added, to specify the Deep Learning model that is used in the test (use"N/A"
if the test doesn't use a model)@pytest.mark.integration("<feature-being-tested>")
to the new tests which I have added, to specify the feature that will be tested@pytest.mark.multinode(<integer-num-nodes>)
to the new tests which I have added, to specify the number of nodes used on a multi-node test@pytest.mark.processor(<"cpu"/"gpu"/"eia"/"neuron">)
to the new tests which I have added, if a test is specifically applicable to only one processor typeBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.