转换为transformers的格式报错 #3
Comments
|
HuggingFace-Transformers对目前ELECTRA的支持是beta版状态。 其中small模型的electra_config.json为： {
"attention_probs_dropout_prob": 0.1,
"directionality": "bidi",
"hidden_act": "gelu",
"hidden_dropout_prob": 0.1,
"hidden_size": 256,
"initializer_range": 0.02,
"intermediate_size": 1024,
"max_position_embeddings": 512,
"num_attention_heads": 4,
"num_hidden_layers": 12,
"pooler_fc_size": 768,
"pooler_num_attention_heads": 12,
"pooler_num_fc_layers": 3,
"pooler_size_per_head": 128,
"pooler_type": "first_token_transform",
"vocab_size": 21128,
"embedding_size": 128
}base模型为： {
"attention_probs_dropout_prob": 0.1,
"directionality": "bidi",
"hidden_act": "gelu",
"hidden_dropout_prob": 0.1,
"hidden_size": 768,
"initializer_range": 0.02,
"intermediate_size": 3072,
"max_position_embeddings": 512,
"num_attention_heads": 12,
"num_hidden_layers": 12,
"pooler_fc_size": 768,
"pooler_num_attention_heads": 12,
"pooler_num_fc_layers": 3,
"pooler_size_per_head": 128,
"pooler_type": "first_token_transform",
"vocab_size": 21128,
"embedding_size": 768
}待后续Huggingface正式支持electra后，我们会更新相应版本的权重以及配置文件。谢谢。 |
|
多谢，已经成功跑通，并整合进了工具包里 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
英文原版的可以转换
转换脚本：https://github.com/huggingface/transformers/blob/electra/src/transformers/convert_electra_original_tf_checkpoint_to_pytorch.py
报错信息：
Traceback (most recent call last): packages/transformers/model_electra/modeling_electra.py", line 102, in load_tf_weights_in_electra assert pointer.shape == array.shape, original_name AssertionError: ('electra/embeddings/LayerNorm/beta', torch.Size([128]), (768,))The text was updated successfully, but these errors were encountered: