U %d@sddlmZddlmZmZmZddlZddlmZddl m Z m Z ddl m Z gZeGdd d ZeGd d d eZed d dddddddgdddddddddddddddddZde_edd dddddddgddddddddddddddde dd Zd!e_ed"d dddddddgddddddddddddddde dd#Zd$e_ed%d dddddddgddddddddddddddde dd#Zd&e_ed'd dddddddgdd(dddd)ddd*dddd+dddd,Zd-e_ed.d dddddddgdd(dddd)ddd*dddd+dde dd#Zd/e_ed0d dddddddgdd(dddd)ddd*dddd+dde dd#Zd1e_ed2d dddddddgdd(dddd)ddd*dddd+dde dd#Zd3e_ed4d5dddddddgd6d(dddd)ddd*ddd6ddddd,Zd7e_ed8d5dddddddgd6d(dddd)ddd*ddd6ddde dd#Zd9e_ed:d5dddddddgd6d(dddd)ddd*ddd6ddde dd#Zd;e_edd5dddddddgd6d(dddd)ddd*ddd6ddddd,Zd?e_ed@d dddddddgddddddddddddddddd,Z dAe _edBd5dddddddgdd(dddd)ddd*ddd6ddddd,Z!dCe!_edDd5dddddddgddEddddFdddGddd6ddddd,Z"dHe"_edId5dddddddgdd(dddd)ddd*ddd6ddde dd#Z#dJe#_edKd5dddddddgddEddddFdddGddd6ddde dd#Z$dLe$_edMd dddddddgddddddddddddddNde %ddOdPZ&dQe&_edRd dddddddgddddddddddddddSde 'ddTdPZ(dUe(_edVd dddddddgddddddddddddddWde )ddOdPZ*dXe*_edYd dddddddgddddddddddddddZde +dd#Z,d[e,_ed\d dddddddgdddddddddddddd]de -dd^dPZ.d_e._dS)`) dataclass)AnyDictTupleN)load_state_dict_from_url)wav2vec2_model Wav2Vec2Model)utilsc@s`eZdZUdZeed<eeefed<eed<e edddZ dd Z d d e dd d Z d S)Wav2Vec2Bundleutorchaudio.pipelines.Wav2Vec2Bundle() Data class that bundles associated information to use pretrained Wav2Vec2Model. This class provides interfaces for instantiating the pretrained model along with the information necessary to retrieve pretrained weights and additional data to be used with the model. Torchaudio library instantiates objects of this class, each of which represents a different pretrained model. Client code should access pretrained models via these instances. Please see below for the usage and the available values. Example - Feature Extraction >>> import torchaudio >>> >>> bundle = torchaudio.pipelines.HUBERT_BASE >>> >>> # Build the model and load pretrained weight. >>> model = bundle.get_model() Downloading: 100%|███████████████████████████████| 360M/360M [00:06<00:00, 60.6MB/s] >>> >>> # Resample audio to the expected sampling rate >>> waveform = torchaudio.functional.resample(waveform, sample_rate, bundle.sample_rate) >>> >>> # Extract acoustic features >>> features, _ = model.extract_features(waveform) _path_params _sample_rate)returncCs|jS)zUSample rate of the audio that the model is trained on. :type: float r)selfrG/tmp/pip-unpacked-wheel-lbdmvq91/torchaudio/pipelines/_wav2vec2/impl.py sample_rate3szWav2Vec2Bundle.sample_ratecCs,d|j}|dkrin|}t|f|}|S)Nz/https://download.pytorch.org/torchaudio/models/)r r)r dl_kwargsurl state_dictrrr_get_state_dict;s  zWav2Vec2Bundle._get_state_dictN)rcCs(tf|j}|||||S)aget_model(self, *, dl_kwargs=None) -> torchaudio.models.Wav2Vec2Model Construct the model and load the pretrained weight. The weight file is downloaded from the internet and cached with :func:`torch.hub.load_state_dict_from_url` Args: dl_kwargs (dictionary of keyword arguments): Passed to :func:`torch.hub.load_state_dict_from_url`. )rr Zload_state_dictreval)rrmodelrrr get_modelAs zWav2Vec2Bundle.get_model)__name__ __module__ __qualname____doc__str__annotations__rrfloatpropertyrrrrrrrrr s r csXeZdZUdZeeed<dZeeed<ddeeeddd Z fd d Z Z S) Wav2Vec2ASRBundleutorchaudio.pipelines.Wav2Vec2ASRBundle() Data class that bundles associated information to use pretrained Wav2Vec2Model. This class provides interfaces for instantiating the pretrained model along with the information necessary to retrieve pretrained weights and additional data to be used with the model. Torchaudio library instantiates objects of this class, each of which represents a different pretrained model. Client code should access pretrained models via these instances. Please see below for the usage and the available values. Example - ASR >>> import torchaudio >>> >>> bundle = torchaudio.pipelines.HUBERT_ASR_LARGE >>> >>> # Build the model and load pretrained weight. >>> model = bundle.get_model() Downloading: 100%|███████████████████████████████| 1.18G/1.18G [00:17<00:00, 73.8MB/s] >>> >>> # Check the corresponding labels of the output. >>> labels = bundle.get_labels() >>> print(labels) ('-', '|', 'E', 'T', 'A', 'O', 'N', 'I', 'H', 'S', 'R', 'D', 'L', 'U', 'M', 'W', 'C', 'F', 'G', 'Y', 'P', 'B', 'V', 'K', "'", 'X', 'J', 'Q', 'Z') >>> >>> # Resample audio to the expected sampling rate >>> waveform = torchaudio.functional.resample(waveform, sample_rate, bundle.sample_rate) >>> >>> # Infer the label probability distribution >>> emissions, _ = model(waveform) >>> >>> # Pass emission to decoder >>> # `ctc_decode` is for illustration purpose only >>> transcripts = ctc_decode(emissions, labels) _labelsr _remove_aux_axis-)blank)r+rcCs |f|jS)aThe output class labels (only applicable to fine-tuned bundles) The first is blank token, and it is customizable. Args: blank (str, optional): Blank token. (default: ``'-'``) Returns: Tuple[str]: For models fine-tuned on ASR, returns the tuple of strings representing the output class labels. Example >>> import torchaudio >>> torchaudio.models.HUBERT_ASR_LARGE.get_labels() ('-', '|', 'E', 'T', 'A', 'O', 'N', 'I', 'H', 'S', 'R', 'D', 'L', 'U', 'M', 'W', 'C', 'F', 'G', 'Y', 'P', 'B', 'V', 'K', "'", 'X', 'J', 'Q', 'Z') )r%)rr+rrr get_labelsszWav2Vec2ASRBundle.get_labelscsPt|}jrLdD]4}||tfddtdD||<q|S)N)z aux.weightzaux.biascsg|]}|jkr|qSr)r)).0irtrr s z5Wav2Vec2ASRBundle._get_state_dict..r)superrr)torchstackrangesize)rrrkey __class__r/rrs   *z!Wav2Vec2ASRBundle._get_state_dict) rrrrrr r!r)intr,r __classcell__rrr8rr$Ss (  r$zwav2vec2_fairseq_base_ls960.pthZ group_norm) )r<r(r')r<r'r'Fig? i gg?)Zextractor_modeZextractor_conv_layer_configZextractor_conv_biasZencoder_embed_dimZencoder_projection_dropoutZencoder_pos_conv_kernelZencoder_pos_conv_groupsZencoder_num_layersZencoder_num_headsZencoder_attention_dropoutZencoder_ff_interm_featuresZencoder_ff_interm_dropoutZencoder_dropoutZencoder_layer_norm_firstZencoder_layer_dropZ aux_num_outi>)r r rawav2vec 2.0 model with "Base" configuration. Pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset [:footcite:`7178964`] (the combination of "train-clean-100", "train-clean-360", and "train-other-500"). Not fine-tuned. Originally published by the authors of *wav2vec 2.0* [:footcite:`baevski2020wav2vec`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. z)wav2vec2_fairseq_base_ls960_asr_ll10m.pth)r r r%ra;Build "base" wav2vec2 model with an extra linear module Pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset [:footcite:`7178964`] (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on 10 minutes of transcribed audio from *Libri-Light* dataset [:footcite:`librilight`] ("train-10min" subset). Originally published by the authors of *wav2vec 2.0* [:footcite:`baevski2020wav2vec`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z)wav2vec2_fairseq_base_ls960_asr_ls100.pth)r%ra Build "base" wav2vec2 model with an extra linear module Pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset [:footcite:`7178964`] (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on 100 hours of transcribed audio from "train-clean-100" subset. Originally published by the authors of *wav2vec 2.0* [:footcite:`baevski2020wav2vec`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z)wav2vec2_fairseq_base_ls960_asr_ls960.pthaBuild "base" wav2vec2 model with an extra linear module Pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset [:footcite:`7178964`] (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on the same audio with the corresponding transcripts. Originally published by the authors of *wav2vec 2.0* [:footcite:`baevski2020wav2vec`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z wav2vec2_fairseq_large_ls960.pthiig?raBuild "large" wav2vec2 model. Pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset [:footcite:`7178964`] (the combination of "train-clean-100", "train-clean-360", and "train-other-500"). Not fine-tuned. Originally published by the authors of *wav2vec 2.0* [:footcite:`baevski2020wav2vec`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. z*wav2vec2_fairseq_large_ls960_asr_ll10m.ptha<Build "large" wav2vec2 model with an extra linear module Pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset [:footcite:`7178964`] (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on 10 minutes of transcribed audio from *Libri-Light* dataset [:footcite:`librilight`] ("train-10min" subset). Originally published by the authors of *wav2vec 2.0* [:footcite:`baevski2020wav2vec`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z*wav2vec2_fairseq_large_ls960_asr_ls100.ptha!Build "large" wav2vec2 model with an extra linear module Pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset [:footcite:`7178964`] (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on 100 hours of transcribed audio from the same dataset ("train-clean-100" subset). Originally published by the authors of *wav2vec 2.0* [:footcite:`baevski2020wav2vec`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z*wav2vec2_fairseq_large_ls960_asr_ls960.pthaBuild "large" wav2vec2 model with an extra linear module Pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset [:footcite:`7178964`] (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on the same audio with the corresponding transcripts. Originally published by the authors of *wav2vec 2.0* [:footcite:`baevski2020wav2vec`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z wav2vec2_fairseq_large_lv60k.pthZ layer_normTacBuild "large-lv60k" wav2vec2 model. Pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset [:footcite:`librilight`]. Not fine-tuned. Originally published by the authors of *wav2vec 2.0* [:footcite:`baevski2020wav2vec`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. z*wav2vec2_fairseq_large_lv60k_asr_ll10m.pthaBuild "large-lv60k" wav2vec2 model with an extra linear module Pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset [:footcite:`librilight`], and fine-tuned for ASR on 10 minutes of transcribed audio from the same dataset ("train-10min" subset). Originally published by the authors of *wav2vec 2.0* [:footcite:`baevski2020wav2vec`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z*wav2vec2_fairseq_large_lv60k_asr_ls100.pthaBuild "large-lv60k" wav2vec2 model with an extra linear module Pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset [:footcite:`librilight`], and fine-tuned for ASR on 100 hours of transcribed audio from *LibriSpeech* dataset [:footcite:`7178964`] ("train-clean-100" subset). Originally published by the authors of *wav2vec 2.0* [:footcite:`baevski2020wav2vec`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z*wav2vec2_fairseq_large_lv60k_asr_ls960.ptha-Build "large-lv60k" wav2vec2 model with an extra linear module Pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* [:footcite:`librilight`] dataset, and fine-tuned for ASR on 960 hours of transcribed audio from *LibriSpeech* dataset [:footcite:`7178964`] (the combination of "train-clean-100", "train-clean-360", and "train-other-500"). Originally published by the authors of *wav2vec 2.0* [:footcite:`baevski2020wav2vec`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z!wav2vec2_fairseq_large_xlsr53.ptha wav2vec 2.0 model with "Base" configuration. Trained on 56,000 hours of unlabeled audio from multiple datasets ( *Multilingual LibriSpeech* [:footcite:`Pratap_2020`], *CommonVoice* [:footcite:`ardila2020common`] and *BABEL* [:footcite:`Gales2014SpeechRA`]). Not fine-tuned. Originally published by the authors of *Unsupervised Cross-lingual Representation Learning for Speech Recognition* [:footcite:`conneau2020unsupervised`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zhubert_fairseq_base_ls960.pthaHuBERT model with "Base" configuration. Pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset [:footcite:`7178964`] (the combination of "train-clean-100", "train-clean-360", and "train-other-500"). Not fine-tuned. Originally published by the authors of *HuBERT* [:footcite:`hsu2021hubert`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zhubert_fairseq_large_ll60k.pthapHuBERT model with "Large" configuration. Pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset [:footcite:`librilight`]. Not fine-tuned. Originally published by the authors of *HuBERT* [:footcite:`hsu2021hubert`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zhubert_fairseq_xlarge_ll60k.pthi0iavHuBERT model with "Extra Large" configuration. Pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset [:footcite:`librilight`]. Not fine-tuned. Originally published by the authors of *HuBERT* [:footcite:`hsu2021hubert`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. z(hubert_fairseq_large_ll60k_asr_ls960.pthaHuBERT model with "Large" configuration. Pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset [:footcite:`librilight`], and fine-tuned for ASR on 960 hours of transcribed audio from *LibriSpeech* dataset [:footcite:`7178964`] (the combination of "train-clean-100", "train-clean-360", and "train-other-500"). Originally published by the authors of *HuBERT* [:footcite:`hsu2021hubert`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z)hubert_fairseq_xlarge_ll60k_asr_ls960.ptha%HuBERT model with "Extra Large" configuration. Pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset [:footcite:`librilight`], and fine-tuned for ASR on 960 hours of transcribed audio from *LibriSpeech* dataset [:footcite:`7178964`] (the combination of "train-clean-100", "train-clean-360", and "train-other-500"). Originally published by the authors of *HuBERT* [:footcite:`hsu2021hubert`] under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_de.pt )r r'r(#)r%rr)awav2vec 2.0 model with "Base" configuration. Pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset [:footcite:`voxpopuli`] ("10k" subset, consisting of 23 languages). Fine-tuned for ASR on 282 hours of transcribed audio from "de" subset. Originally published by the authors of *VoxPopuli* [:footcite:`voxpopuli`] under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_en.pt)r r'r(awav2vec 2.0 model with "Base" configuration. Pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset [:footcite:`voxpopuli`] ("10k" subset, consisting of 23 languages). Fine-tuned for ASR on 543 hours of transcribed audio from "en" subset. Originally published by the authors of *VoxPopuli* [:footcite:`voxpopuli`] under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_es.ptrFawav2vec 2.0 model with "Base" configuration. Pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset [:footcite:`voxpopuli`] ("10k" subset, consisting of 23 languages). Fine-tuned for ASR on 166 hours of transcribed audio from "es" subset. Originally published by the authors of *VoxPopuli* [:footcite:`voxpopuli`] under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_fr.pt+awav2vec 2.0 model with "Base" configuration. Pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset [:footcite:`voxpopuli`] ("10k" subset, consisting of 23 languages). Fine-tuned for ASR on 211 hours of transcribed audio from "fr" subset. Originally published by the authors of *VoxPopuli* [:footcite:`voxpopuli`] under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_it.pt%r&awav2vec 2.0 model with "Base" configuration. Pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset [:footcite:`voxpopuli`] ("10k" subset, consisting of 23 languages). Fine-tuned for ASR on 91 hours of transcribed audio from "it" subset. Originally published by the authors of *VoxPopuli* [:footcite:`voxpopuli`] under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :func:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. )/Z dataclassesrtypingrrrr3Ztorchaudio._internalrZtorchaudio.modelsrrr __all__r r$Z WAV2VEC2_BASErZ_get_en_labelsZWAV2VEC2_ASR_BASE_10MZWAV2VEC2_ASR_BASE_100HZWAV2VEC2_ASR_BASE_960HZWAV2VEC2_LARGEZWAV2VEC2_ASR_LARGE_10MZWAV2VEC2_ASR_LARGE_100HZWAV2VEC2_ASR_LARGE_960HZWAV2VEC2_LARGE_LV60KZWAV2VEC2_ASR_LARGE_LV60K_10MZWAV2VEC2_ASR_LARGE_LV60K_100HZWAV2VEC2_ASR_LARGE_LV60K_960HZWAV2VEC2_XLSR53Z HUBERT_BASEZ HUBERT_LARGEZ HUBERT_XLARGEZHUBERT_ASR_LARGEZHUBERT_ASR_XLARGEZ_get_de_labelsZVOXPOPULI_ASR_BASE_10K_DEZ_get_vp_en_labelsZVOXPOPULI_ASR_BASE_10K_ENZ_get_es_labelsZVOXPOPULI_ASR_BASE_10K_ESZ_get_fr_labelsZVOXPOPULI_ASR_BASE_10K_FRZ_get_it_labelsZVOXPOPULI_ASR_BASE_10K_ITrrrrsr   DX