U d@s.UdZddlmZddlmZmZmZmZddlZgZ ee e d<GdddZ e Z eejje fZejedd d Zejedd d Zejedd dZeejeddddZeeeddddZeejdddZeeddddZejeddddZeedddZeejjddd ZdS)!zWUtilities for eliminating boilerplate code to handle abstract streams with CPU device. )contextmanager) GeneratorListUnioncastN__all__c@s eZdZdS) CPUStreamTypeN)__name__ __module__ __qualname__r r J/tmp/pip-unpacked-wheel-ua33x9lu/torch/distributed/pipeline/sync/stream.pyrsr)devicereturncCs|jdkrtStj|S)z3Creates a new stream for either CPU or CUDA device.cuda)type CPUStreamtorchrStreamrr r r new_streams rcCs|jdkrtStj|S)z@:func:`torch.cuda.current_stream` for either CPU or CUDA device.r)rrrrcurrent_streamrr r r r$s rcCs|jdkrtStj|S)z@:func:`torch.cuda.default_stream` for either CPU or CUDA device.r)rrrrdefault_streamrr r r r+s r)NNNc cs6|jdkrdVdStj| dVW5QRXdS)z8:func:`torch.cuda.device` for either CPU or CUDA device.rN)rrrrrr r r use_device2s  r)streamrc cs8t|sdVdStjt| dVW5QRXdS)z8:func:`torch.cuda.stream` for either CPU or CUDA stream.N)is_cudarrras_cudarr r r use_stream=s rcCst|rt|jStdS)z(Gets the device from CPU or CUDA stream.cpu)rrrrrr r r get_deviceHs r )sourcetargetrcCs4t|r0t|r$t|t|n t|dS)z:meth:`torch.cuda.Stream.wait_stream` for either CPU or CUDA stream. It makes the source stream wait until the target stream completes work queued. N)rr wait_streamZ synchronize)r!r"r r r r#Osr#)tensorrrcCs0t|r,|dg|}|t|dS)zA:meth:`torch.Tensor.record_stream` for either CPU or CUDA stream.rN)rZ new_emptyset_Zstorage record_streamr)r$rr r r r&^s r&cCs|tk S)zs&