U ‰džã@sœUdZddlmZmZddlZddlmZddlmZgZee e d<eeeefdœd d „Z Gd d „d ej j ƒZeeed œdd„ZGdd„dej j ƒZdS)z0Arbitrary dependency between two autograd lanes.é)ÚListÚTupleN)ÚTensoré)Ú get_phonyÚ__all__)ÚinputÚreturncCs4t ¡r|jrt |¡\}}nt|jdd}||fS)z7Branches out from an autograd lane of the given tensor.F©Ú requires_grad)ÚtorchÚis_grad_enabledr ÚForkÚapplyrÚdevice©rÚphony©rúN/tmp/pip-unpacked-wheel-ua33x9lu/torch/distributed/pipeline/sync/dependency.pyÚforksrc@sBeZdZedeeeefdœdd„ƒZedeeedœdd„ƒZdS)r)Úctxrr cCst|jdd}| ¡| ¡fS)NFr )rrÚdetach©rrrrrrÚforwardsz Fork.forward)rÚ grad_inputÚ grad_gradr cCs|S©Nr)rrrrrrÚbackward"sz Fork.backwardN)Ú__name__Ú __module__Ú __qualname__Ú staticmethodrrrrrrrrrsr)rrr cCs$t ¡r |js|jr t ||¡}|S)zMerges two autograd lanes.)r r r ÚJoinrrrrrÚjoin's r#c@sBeZdZedeeedœdd„ƒZedeeedfdœdd„ƒZdS)r")rrrr cCs| ¡Sr)rrrrrr0sz Join.forwardN)rrr cCs|dfSrr)rrrrrr4sz Join.backward)rrr r!rrrrrrrrr"/sr")Ú__doc__Útypingrrr rrrrÚstrÚ__annotations__rZautogradZFunctionrr#r"rrrrÚs