U %d~ @sddlZddlZddlZddlZddlZddlZddlZddlZddlm Z m Z m Z m Z ddl mZdee eeee dddZdeee ee eeeedd d d Zde eeedddZdee eee edddZdS)N)AnyIterableListOptional)tqdmT)url start_byte block_size progress_barreturnc cstjj|dd}tj|}t|dd}W5QRX||krHdStj|}|rhd||jd<tj|\}t dd d || d <}d } | |} | sq| V| t | 7} | t | qW5QRXW5QRXdS) aDStream url by chunk Args: url (str): Url. start_byte (int or None, optional): Start streaming at that point (Default: ``None``). block_size (int, optional): Size of chunks to stream (Default: ``32 * 1024``). progress_bar (bool, optional): Display a progress bar (Default: ``True``). HEADmethodContent-LengthNz bytes={}-ZRangeBTi)unitZ unit_scaleZ unit_divisortotaldisabler) urllibrequestRequesturlopenintinfogetformatheadersrreadlenupdate) rr r r reqresponseZurl_sizeZupointerZpbar num_byteschunkr&=/tmp/pip-unpacked-wheel-lbdmvq91/torchaudio/datasets/utils.py stream_urls.    r(sha256F)rdownload_folderfilename hash_value hash_typer resumer c Csbtdtjj|dd}tj|}|p@|p@tj |}tj ||} |rrtj | rrd} tj | } n(|stj | rtd| nd} d} |r| t|dd krt| d  } t| ||rW5QRdSW5QRXtd | t| | &} t|| |d D]}| |q W5QRXt| d (} |rTt| ||sTtd | W5QRXdS) aPDownload file to disk. Args: url (str): Url. download_folder (str): Folder to download file. filename (str or None, optional): Name of downloaded file. If None, it is inferred from the url (Default: ``None``). hash_value (str or None, optional): Hash for url (Default: ``None``). hash_type (str, optional): Hash type, among "sha256" and "md5" (Default: ``"sha256"``). progress_bar (bool, optional): Display a progress bar (Default: ``True``). resume (bool, optional): Enable resuming download (Default: ``False``). zDdownload_url is deprecated and will be removed in the v0.12 release.r rabz6{} already exists. Delete the file manually and retry.wbNrrrbzBThe hash of {} does not match. Delete the file manually and retry.)r r )warningswarnrrrrr get_filenameospathbasenamejoinexistsgetsize RuntimeErrorrrropen validate_filer(write)rr*r+r,r-r r.r"Zreq_infofilepathmodeZ local_sizefile_objZfpointerr%r&r&r' download_url7s.     rB)rAr,r-r cCsP|dkrt}n|dkr$t}nt|d}|s8qD||q(||kS)a8Validate a given file object with its hash. Args: file_obj: File object to read from. hash_value (str): Hash for url. hash_type (str, optional): Hash type, among "sha256" and "md5" (Default: ``"sha256"``). Returns: bool: return True if its a valid file, else False. r)md5i)hashlibr)rC ValueErrorrr! hexdigest)rAr,r-Z hash_funcr%r&r&r'r=ls     r=) from_pathto_path overwriter c Csj|dkrtj|}zt|d}td|g}|D]T}tj||j }| r| |tj |rtd||sq<| ||q<|W5QRWSQRXWntjk rYnXzt|dh}td||}|D]D}tj||}tj |r(td||s(q| ||qW5QRX|WStjk r\YnXtddS)aExtract archive. Args: from_path (str): the path of the archive. to_path (str or None, optional): the root path of the extraced files (directory of from_path) (Default: ``None``) overwrite (bool, optional): overwrite existing files (Default: ``False``) Returns: List[str]: List of paths to extracted files even if not overwritten. Examples: >>> url = 'http://www.quest.dcs.shef.ac.uk/wmt16_files_mmt/validation.tar.gz' >>> from_path = './validation.tar.gz' >>> to_path = './' >>> torchaudio.datasets.utils.download_from_url(url, from_path) >>> torchaudio.datasets.utils.extract_archive(from_path, to_path) NrzOpened tar file {}.z{} already extracted.zOpened zip file {}.z7We currently only support tar.gz, tgz, and zip achives.)r5r6dirnametarfiler<loggingrrr8nameisfileappendr9extract ReadErrorzipfileZipFilenamelist BadZipFileNotImplementedError)rGrHrItarfilesfile_ file_pathzfiler&r&r'extract_archivesB   r])NrT)NNr)TF)r))NF)rDrMr5rLrurllib.requestr2rStypingrrrrZtorch.utils.model_zoorstrrboolr(rBr=r]r&r&r&r'sH  , 5