47 Pull requests merged by 30 people

• Introducing TensorRT lazy export and caching option with trt_compile()

  #11266 merged Nov 27, 2024

• Fix strategies saving unsharded optimizer states

  #11392 merged Nov 27, 2024

• data modules for llava_next

  #11400 merged Nov 27, 2024

• Fix vllm test issue when run_accuracy is enabled

  #11413 merged Nov 26, 2024

• Adding LLava-Next model class

  #11399 merged Nov 26, 2024

• [NeMo-UX] Support load_strictness

  #10612 merged Nov 26, 2024

• Rewire tokenizer exception handling in model resume

  #11375 merged Nov 26, 2024

• Remove logic to skip checkpoint save if checkpoint exists

  #11362 merged Nov 26, 2024

• Micro-optimizations with ~16% speedup Canary training on 1GPU

  #11370 merged Nov 26, 2024

• ci: Add HF cache

  #11398 merged Nov 26, 2024

• capitalize HF as HF instead of Hf

  #11384 merged Nov 26, 2024

• Add a fix for single-GPU nsys.

  #11354 merged Nov 26, 2024

• Add Tiktoken support for TRTLLM

  #10306 merged Nov 26, 2024

• Minor fix

  #11353 merged Nov 26, 2024

• Fix selective restore by explicitly verifying keys

  #11377 merged Nov 25, 2024

• Add sample generate to PTQ for NeMo 2.0

  #11339 merged Nov 25, 2024

• Fix environment variables in torchrun executor

  #11363 merged Nov 25, 2024

• Lhotse support for transcribe_speech_parallel

  #11249 merged Nov 25, 2024

• Update llama32 vision (mllama) use attention bias

  #11316 merged Nov 25, 2024

• Fix DDP unused param error when TE is enabled in NeMo Lite

  #11364 merged Nov 24, 2024

• Enable packed dataset for validation; add a2a_experimental argument

  #11378 merged Nov 23, 2024

• calculate metrics for nemo2 sftpeft notebook

  #11381 merged Nov 22, 2024

• mlm conversion & tiktokenizer support

  #11349 merged Nov 22, 2024

• Add llama 3.2 1b and 3b

  #11335 merged Nov 22, 2024

• Fix transcribe speech

  #11379 merged Nov 22, 2024

• Causal Codec decoder implementation

  #11380 merged Nov 22, 2024

• update SquadDataModule to use run.config

  #11358 merged Nov 22, 2024

• Add missing test to CICD needed list

  #11376 merged Nov 22, 2024

• add fix to recipe

  #11368 merged Nov 22, 2024

• Revert "update hypothesis when passed through cfg"

  #11373 merged Nov 22, 2024

• Fix Gemma2 Attention Args

  #11365 merged Nov 21, 2024

• update hypothesis when passed through cfg

  #11366 merged Nov 21, 2024

• Add dora recipes

  #11330 merged Nov 21, 2024

• nemo2 peft merge

  #11017 merged Nov 21, 2024

• Export & deploy updates (part II)

  #11344 merged Nov 21, 2024

• Add PP support in NeVA along with few bug fixes

  #11170 merged Nov 21, 2024

• Add torchrun local executor to recipes

  #11342 merged Nov 21, 2024

• fix typo

  #11351 merged Nov 21, 2024

• Fix linear layer replacement

  #11356 merged Nov 21, 2024

• pass trust_remote_code to AutoTokenizer

  #11343 merged Nov 21, 2024

• Adding alinger export

  #11269 merged Nov 21, 2024

• Add support for restoring from 2.0 checkpoint in 1.0

  #11347 merged Nov 20, 2024

• Making TDT models support all-positive durations (previously duration must contain 0)

  #9656 merged Nov 20, 2024

• Fix CLIP transformer layer api

  #11337 merged Nov 20, 2024

• Use NCCL bootsrap backend for TP communication overlaps

  #10622 merged Nov 20, 2024

• More robust tar file loading from AIStore

  #11323 merged Nov 20, 2024

• Leave target_module as default in PEFT Recipes

  #11334 merged Nov 20, 2024

33 Pull requests opened by 21 people

• Llama3 conversion from Megatron DCP checkpoints to HF [NeMo 1.0]

  #11345 opened Nov 20, 2024

• Remove default mutable arguments from AbstractEmbModel constructor

  #11348 opened Nov 20, 2024

• Use explicit subpaths in io for exporting a checkpoint

  #11352 opened Nov 20, 2024

• chore(beep boop 🤖): Bump `MCORE_TAG=81fee9b...` (2024-11-21)

  #11355 opened Nov 21, 2024

• Fix deploy conflicts in llm.api

  #11367 opened Nov 21, 2024

• Deprecate old preemption callback

  #11369 opened Nov 21, 2024

• chore(beep boop 🤖): Bump `MCORE_TAG=ddd920f...` (2024-11-22)

  #11371 opened Nov 22, 2024

• add hindi tn/itn coverage

  #11382 opened Nov 22, 2024

• Add VLM scripts and CI tests

  #11383 opened Nov 22, 2024

• chore(beep boop 🤖): Bump `MCORE_TAG=a9d040c...` (2024-11-23)

  #11385 opened Nov 23, 2024

• chore(beep boop 🤖): Bump `MCORE_TAG=c10721e...` (2024-11-24)

  #11387 opened Nov 24, 2024

• Huvu/t5 nemo2.0 nemoci 3b11b

  #11388 opened Nov 24, 2024

• chore(beep boop 🤖): Bump `MCORE_TAG=9a75c72...` (2024-11-25)

  #11389 opened Nov 25, 2024

• Add vlm nemo run scripts

  #11394 opened Nov 25, 2024

• Update Dockerfile.ci to pre failure

  #11395 opened Nov 25, 2024

• Update Dockerfile.ci to offending MCore

  #11396 opened Nov 25, 2024

• Adding changes to asr documentation

  #11397 opened Nov 25, 2024

• [Scripts] Remove fixed seed for adding noise

  #11401 opened Nov 26, 2024

• chore(beep boop 🤖): Bump `MCORE_TAG=081ab4d...` (2024-11-26)

  #11402 opened Nov 26, 2024

• [audio] Keep input directory structure when saving processed files

  #11403 opened Nov 26, 2024

• Nemo run recipe's and example scripts for Llava Next

  #11405 opened Nov 26, 2024

• Add tests for resiliency feature integration

  #11406 opened Nov 26, 2024

• Removing unnecessary lines

  #11408 opened Nov 26, 2024

• [Draft] Move FP8 TE export logic to mcore.export

  #11409 opened Nov 26, 2024

• Bug fix on Lhotse Speech LLM dataloader

  #11410 opened Nov 26, 2024

• Fix the fake parallel states init with moe parallel folding.

  #11411 opened Nov 26, 2024

• Fix checkpoint loading for None values in io_unflatten_object

  #11412 opened Nov 26, 2024

• Handle exception when importing RetroGPTChunkDatasets

  #11415 opened Nov 26, 2024

• Nemo 2.0 canonical lora

  #11416 opened Nov 26, 2024

• chore(beep boop 🤖): Bump `MCORE_TAG=f5afc25...` (2024-11-27)

  #11417 opened Nov 27, 2024

• ci: Allow dry-run of release

  #11418 opened Nov 27, 2024

• fix dtype when init HF model from config

  #11420 opened Nov 27, 2024

• Adjust CLI support for PTQ

  #11421 opened Nov 27, 2024

11 Issues closed by 5 people

• Fine-tuning ASR Lightning Error

  #11386 closed Nov 27, 2024

• asr + diarization config setting problem

  #11393 closed Nov 25, 2024

• ASR - WER not decreasing after certain point (Finetuning hybrid_cache_aware_streaming model)

  #10578 closed Nov 24, 2024

• Modules fail for Dreambooth example

  #10888 closed Nov 24, 2024

• Link Not Found at Mamba Tutorial

  #10899 closed Nov 24, 2024

• global batch size at different sequence length

  #10905 closed Nov 24, 2024

• NeMO dependency issues on HuggingFace Hub (for ASR models)

  #10940 closed Nov 23, 2024

• Shape mismatch between fastcont_hybrid_bpe_large.yaml config and 110M TDT-CTC Model, when using custom 1024 size tokenizer

  #11361 closed Nov 22, 2024

• SFT stage use context parallel with flash attention error

  #10876 closed Nov 21, 2024

• Add Hydrarunner to oomptimizer

  #10882 closed Nov 21, 2024

• Converting trained llama 2 checkpoint to hf gives "invalid key" error

  #10884 closed Nov 21, 2024

6 Issues opened by 6 people

• Add support for instruct models in nemo 2

  #11414 opened Nov 26, 2024

• NeMo docker image 24.09 shows weird FileNotFoundError for shards FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpxoyazmhf/model_weights/model.decoder.layers.self_attention.core_attention._extra_state/shard_0_32.pt'

  #11374 opened Nov 22, 2024

• Drastic difference between .nemo and HF checkpoint

  #11360 opened Nov 21, 2024

• Doesn't have the shards in expected directory.

  #11359 opened Nov 21, 2024

• Sequence packing with ChatDataset

  #11357 opened Nov 21, 2024

• AttributeError: module 'threading' has no attribute '_Condition

  #11350 opened Nov 20, 2024

57 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Sortformer Diarizer 4spk v1 model PR Part 1: models, modules and dataloaders

  #11282 commented on Nov 27, 2024 • 97 new comments

• Add BERT Model To NeMo2.0

  #11333 commented on Nov 26, 2024 • 16 new comments

• Next Canary's prompt format

  #11058 commented on Nov 26, 2024 • 8 new comments

• Context Parallel SFT Support for dataset in THD format

  #10688 commented on Nov 27, 2024 • 7 new comments

• build: Move dependencies to `requirements*.txt` files

  #11324 commented on Nov 25, 2024 • 6 new comments

• NeMo-UX: use input_ids instead of tokens in HfAutoModelForCausalLM

  #11340 commented on Nov 20, 2024 • 2 new comments

• Sortformer Diarizer 4spk v1 model PR Part 2: Unit-tests for Sortformer Diarizer.

  #11336 commented on Nov 27, 2024 • 2 new comments

• Add different recipe examples to NeMo 2.0

  #11317 commented on Nov 22, 2024 • 2 new comments

• Add MCore FSDP2 support

  #11216 commented on Nov 22, 2024 • 1 new comment

• fix: regular torch optims (e.g., sgd) no longer error with closure spec

  #11189 commented on Nov 27, 2024 • 0 new comments

• adding canary docs

  #11176 commented on Nov 21, 2024 • 0 new comments

• add nemotron5 conversion

  #11171 commented on Nov 27, 2024 • 0 new comments

• Optimized Graph-Transducer Implementation

  #11169 commented on Nov 22, 2024 • 0 new comments

• Adding support for LLaVA NeXT

  #11150 commented on Nov 22, 2024 • 0 new comments

• add JitTransform

  #11131 commented on Nov 27, 2024 • 0 new comments

• Add StragglerDetection and FTlauncher to NeMo2.0

  #11117 commented on Nov 26, 2024 • 0 new comments

• Add support converting Llama-3.2 LLMs hf to nemo

  #11113 commented on Nov 23, 2024 • 0 new comments

• Add a checkpoint averaging script for the new .distcp checkpoint format

  #10462 commented on Nov 22, 2024 • 0 new comments

• chore: Bump version

  #11227 commented on Nov 23, 2024 • 0 new comments

• NeMo 2.0 documentation upgrade

  #11235 commented on Nov 22, 2024 • 0 new comments

• trainer strategy params update

  #11236 commented on Nov 23, 2024 • 0 new comments

• add drop layers support

  #11238 commented on Nov 23, 2024 • 0 new comments

• Karpnv/beamsearch1

  #11243 commented on Nov 24, 2024 • 0 new comments

• Add option to set cp_comm_type

  #11258 commented on Nov 27, 2024 • 0 new comments

• Aligner/nemotron5

  #11264 commented on Nov 27, 2024 • 0 new comments

• Add checklist for config validations

  #11265 commented on Nov 27, 2024 • 0 new comments

• Add option to change batch size if needed

  #11268 commented on Nov 27, 2024 • 0 new comments

• NeMo-UX: add Hf's AutoModelForImageTextToText

  #11321 commented on Nov 26, 2024 • 0 new comments

• Huvu/t5 nemo2.0 recipes update

  #11327 commented on Nov 23, 2024 • 0 new comments

• NeMo-UX: MegatronAutoModel

  #11341 commented on Nov 22, 2024 • 0 new comments

• AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?

  #9818 commented on Nov 22, 2024 • 0 new comments

• Sortformer Integration Release Inquiry

  #10491 commented on Nov 22, 2024 • 0 new comments

• NeuralDiarizer with the telephonic config mix speakers at the very beginning of shorter audio files (less than 2 minutes duration)

  #10988 commented on Nov 23, 2024 • 0 new comments

• Converting Mamba to tp4: RuntimeError: The size of tensor a (18560) must match the size of tensor b (4640) at non-singleton dimension 0

  #10966 commented on Nov 23, 2024 • 0 new comments

• RuntimeError: Function 'AcosBackward0' returned nan values in its 0th output.

  #11025 commented on Nov 24, 2024 • 0 new comments

• canary-1b is not exportable

  #11004 commented on Nov 24, 2024 • 0 new comments

• Unable to export MSDD model to pt or ONNX

  #10999 commented on Nov 24, 2024 • 0 new comments

• dim unmatch when doing sft with tensor parallel and sequence parallel and LoRA

  #10280 commented on Nov 25, 2024 • 0 new comments

• How to finetune the NEST model with CTC Loss for ASR task?

  #11163 commented on Nov 26, 2024 • 0 new comments

• Add CI Tests for Canary/AEDMultitask "lang_field"

  #10103 commented on Nov 26, 2024 • 0 new comments

• Deploy ASR STT Streaming model

  #11019 commented on Nov 26, 2024 • 0 new comments

• Flashlight and Pyctcdecode decoders

  #8428 commented on Nov 26, 2024 • 0 new comments

• Fixed chokepoint in diarization for longer audios

  #9114 commented on Nov 21, 2024 • 0 new comments

• NeVa::forward - remove device syncs (torch.where) and vectorize over batch dimensions

  #9689 commented on Nov 21, 2024 • 0 new comments

• Fix import path for CTCDecodingConfig

  #10286 commented on Nov 21, 2024 • 0 new comments

• Fix trascribe speech parralel with tarred datasets

  #10372 commented on Nov 22, 2024 • 0 new comments

• Add slice_with_offset and dry_run Support for Tar Dataset Creation; New Script for Partial Conversion

  #10511 commented on Nov 22, 2024 • 0 new comments

• DAPT with NeMo FW

  #10689 commented on Nov 22, 2024 • 0 new comments

• replace `SIGKILL` with `SIGTERM`

  #10777 commented on Nov 26, 2024 • 0 new comments

• [WIP] Migrate SpeechLM to NeMo 2.0

  #10808 commented on Nov 27, 2024 • 0 new comments

• Replace usage of np.sctypes with np.issubdtype

  #10839 commented on Nov 26, 2024 • 0 new comments

• EMMeTT support in SpeechLLM + tutorial for Lhotse Multimodal Dataloading

  #10927 commented on Nov 26, 2024 • 0 new comments

• CLIP Score Fusion model implementation

  #10929 commented on Nov 23, 2024 • 0 new comments

• Add vlm generation function

  #11063 commented on Nov 23, 2024 • 0 new comments

• [NeMo-UX] Add option to drop optimizer states

  #11089 commented on Nov 25, 2024 • 0 new comments

• [Bugfix] fix qwen tokenizer config when converting to nemo format

  #11098 commented on Nov 23, 2024 • 0 new comments

• Add scripts for importing a ckpt and running a forward step on it for nemo.collections.llm

  #11108 commented on Nov 27, 2024 • 0 new comments