U ds@sddlZddlmZmZddlmZmZddlZddl Z ddl m Z ddl m Z ddlmZdd lmZmZmZmZmZmZmZmZmZmZed e d ZGd d d e ZGddde ZGddde ZGddde ZGddde Z Gddde Z!dS)N) OrderedDictabc)chainislice)Module) Parameter)_copy_to_script_wrapper) AnyDictIterableIteratorMappingOptionaloverloadTupleTypeVarUnionT)boundcs$eZdZeddfdd ZZS) ContainerN)kwargsreturnc s:tt|td|D]\}}|||q dS)Nzmnn.Container is deprecated. All of it's functionality is now implemented in nn.Module. Subclass that instead.)superr__init__warningswarnitems add_module)selfrkeyvalue __class__>/tmp/pip-unpacked-wheel-ua33x9lu/torch/nn/modules/container.pyrs zContainer.__init__)__name__ __module__ __qualname__r r __classcell__r%r%r#r&rsrcseZdZUdZeeefed<eeddddZ edddd dZ fd dZ e d d d Z e e de fd ddZeeddddZe eefddddZe ed ddZe fddZe eed ddZddZedddd ZZS)! SequentialaA sequential container. Modules will be added to it in the order they are passed in the constructor. Alternatively, an ``OrderedDict`` of modules can be passed in. The ``forward()`` method of ``Sequential`` accepts any input and forwards it to the first module it contains. It then "chains" outputs to inputs sequentially for each subsequent module, finally returning the output of the last module. The value a ``Sequential`` provides over manually calling a sequence of modules is that it allows treating the whole container as a single module, such that performing a transformation on the ``Sequential`` applies to each of the modules it stores (which are each a registered submodule of the ``Sequential``). What's the difference between a ``Sequential`` and a :class:`torch.nn.ModuleList`? A ``ModuleList`` is exactly what it sounds like--a list for storing ``Module`` s! On the other hand, the layers in a ``Sequential`` are connected in a cascading way. Example:: # Using Sequential to create a small model. When `model` is run, # input will first be passed to `Conv2d(1,20,5)`. The output of # `Conv2d(1,20,5)` will be used as the input to the first # `ReLU`; the output of the first `ReLU` will become the input # for `Conv2d(20,64,5)`. Finally, the output of # `Conv2d(20,64,5)` will be used as input to the second `ReLU` model = nn.Sequential( nn.Conv2d(1,20,5), nn.ReLU(), nn.Conv2d(20,64,5), nn.ReLU() ) # Using Sequential with OrderedDict. This is functionally the # same as the above code model = nn.Sequential(OrderedDict([ ('conv1', nn.Conv2d(1,20,5)), ('relu1', nn.ReLU()), ('conv2', nn.Conv2d(20,64,5)), ('relu2', nn.ReLU()) ])) _modulesN)argsrcGsdSNr%)r r-r%r%r&rJszSequential.__init__zOrderedDict[str, Module])argrcCsdSr.r%)r r/r%r%r&rNscsrtt|t|dkrLt|dtrL|dD]\}}|||q4n"t|D]\}}|t ||qTdS)Nrr) rr+rlen isinstancerrr enumeratestr)r r-r!moduleidxr#r%r&rRs rcCsPt|}t|}| |kr(|ks8ntd|||;}tt||dS)z#Get the idx-th item of the iteratorindex {} is out of rangeN)r0operatorindex IndexErrorformatnextr)r iteratorr5sizer%r%r&_get_item_by_idx[s  zSequential._get_item_by_idxcCs<t|tr&|tt|j|S||j|SdSr.) r1slicer$rlistr,rr?valuesr r5r%r%r& __getitem__ds zSequential.__getitem__r5r4rcCs||j|}t|||Sr.)r?r,keyssetattr)r r5r4r!r%r%r& __setitem__kszSequential.__setitem__r5rcCsNt|tr.t|j|D]}t||qn||j|}t||dSr.)r1r@rAr,rFdelattrr?)r r5r!r%r%r& __delitem__os  zSequential.__delitem__cCs t|jSr.r0r,r r%r%r&__len__wszSequential.__len__cs tt|}dd|D}|S)NcSsg|]}|s|qSr%isdigit.0r!r%r%r& ~sz&Sequential.__dir__..)rr+__dir__r rFr#r%r&rT{szSequential.__dir__cCst|jSr.iterr,rBrMr%r%r&__iter__szSequential.__iter__cCs|D] }||}q|Sr.r%)r inputr4r%r%r&forwards zSequential.forwardr4rcCs|tt|||S)zkAppends a given module to the end. Args: module (nn.Module): module to append rr3r0r r4r%r%r&appendszSequential.append)r'r(r)__doc__r r3r__annotations__rrrr?r rrDintrHr@rKrNrTrrXrZr^r*r%r%r#r&r+s( ,  r+cs(eZdZUdZeeefed<d%ee eddfdd Z ddZ e e eedfd d d Ze edd d dZee efdd ddZe e dddZe eedddZe eddddZe eddddZe fddZe eddddZedd d!d"Ze eddd#d$ZZS)& ModuleListaHolds submodules in a list. :class:`~torch.nn.ModuleList` can be indexed like a regular Python list, but modules it contains are properly registered, and will be visible by all :class:`~torch.nn.Module` methods. Args: modules (iterable, optional): an iterable of modules to add Example:: class MyModule(nn.Module): def __init__(self): super(MyModule, self).__init__() self.linears = nn.ModuleList([nn.Linear(10, 10) for i in range(10)]) def forward(self, x): # ModuleList can act as an iterable, or be indexed using ints for i, l in enumerate(self.linears): x = self.linears[i // 2](x) + l(x) return x r,Nmodulesrcs"tt||dk r||7}dSr.)rrbrr rdr#r%r&rszModuleList.__init__cCsTt|}t| |kr(t|ks8ntd||dkrL|t|7}t|Sz.Get the absolute index for the list of modulesr7rr8r9r0r:r;r3rCr%r%r&_get_abs_string_indexs    z ModuleList._get_abs_string_indexrIcCs6t|tr"|t|j|S|j||SdSr.)r1r@r$rAr,rBrhrCr%r%r&rDs zModuleList.__getitem__rEcCs||}t|t||Sr.)rhrGr3)r r5r4r%r%r&rHs zModuleList.__setitem__cCsxt|tr2tt|j|D]}t|t|qnt|||ddtt|jD}tt t ||j |_dS)NcSsg|] }t|qSr%r3rRir%r%r&rSsz*ModuleList.__delitem__..) r1r@ranger0r,rJr3rhrrAziprB)r r5kZ str_indicesr%r%r&rKs  zModuleList.__delitem__r6cCs t|jSr.rLrMr%r%r&rNszModuleList.__len__cCst|jSr.rVrMr%r%r&rXszModuleList.__iter__cCs ||Sr.extendrer%r%r&__iadd__szModuleList.__iadd__otherrcCs2t}tt||D]\}}|t||q|Sr.)rbr2rrr3)r rsZcombinedrkr4r%r%r&__add__szModuleList.__add__cs tt|}dd|D}|S)NcSsg|]}|s|qSr%rOrQr%r%r&rSsz&ModuleList.__dir__..)rrbrTrUr#r%r&rTszModuleList.__dir__)r9r4rcCsFtt|j|dD] }|jt|d|jt|<q||jt|<dS)zInsert a given module before a given index in the list. Args: index (int): index to insert. module (nn.Module): module to insert rN)rlr0r,r3)r r9r4rkr%r%r&insertszModuleList.insertr[cCs|tt|||S)zwAppends a given module to the end of the list. Args: module (nn.Module): module to append r\r]r%r%r&r^szModuleList.appendcCsPt|tjstdt|jt|}t|D]\}}|t |||q.|S)zAppends modules from a Python iterable to the end of the list. Args: modules (iterable): iterable of modules to append z=ModuleList.extend should be called with an iterable, but got ) r1container_abcsr TypeErrortyper'r0r2rr3)r rdoffsetrkr4r%r%r&rps zModuleList.extend)N)r'r(r)r_r r3rr`rr rrhr rarrDrHr@rKrNrrXrqrtrTrvr^rpr*r%r%r#r&rbs&     rbcs:eZdZUdZeeefed<d"ee eefddfdd Z e eeddd Z eedd d d Z eddd dZe edddZe eedddZe eedddZddddZeedddZe eedddZe eeeefdddZe eedddZe eefddd d!ZZS)# ModuleDicta2Holds submodules in a dictionary. :class:`~torch.nn.ModuleDict` can be indexed like a regular Python dictionary, but modules it contains are properly registered, and will be visible by all :class:`~torch.nn.Module` methods. :class:`~torch.nn.ModuleDict` is an **ordered** dictionary that respects * the order of insertion, and * in :meth:`~torch.nn.ModuleDict.update`, the order of the merged ``OrderedDict``, ``dict`` (started from Python 3.6) or another :class:`~torch.nn.ModuleDict` (the argument to :meth:`~torch.nn.ModuleDict.update`). Note that :meth:`~torch.nn.ModuleDict.update` with other unordered mapping types (e.g., Python's plain ``dict`` before Python version 3.6) does not preserve the order of the merged mapping. Args: modules (iterable, optional): a mapping (dictionary) of (string: module) or an iterable of key-value pairs of type (string, module) Example:: class MyModule(nn.Module): def __init__(self): super(MyModule, self).__init__() self.choices = nn.ModuleDict({ 'conv': nn.Conv2d(10, 10, 3), 'pool': nn.MaxPool2d(3) }) self.activations = nn.ModuleDict([ ['lrelu', nn.LeakyReLU()], ['prelu', nn.PReLU()] ]) def forward(self, x, choice, act): x = self.choices[choice](x) x = self.activations[act](x) return x r,Nrccs$tt||dk r ||dSr.)rr{rupdaterer#r%r&r?szModuleDict.__init__r!rcCs |j|Sr.r,r r!r%r%r&rDDszModuleDict.__getitem__)r!r4rcCs|||dSr.)r)r r!r4r%r%r&rHHszModuleDict.__setitem__cCs |j|=dSr.r~rr%r%r&rKKszModuleDict.__delitem__r6cCs t|jSr.rLrMr%r%r&rNNszModuleDict.__len__cCs t|jSr.)rWr,rMr%r%r&rXRszModuleDict.__iter__cCs ||jkSr.r~rr%r%r& __contains__VszModuleDict.__contains__cCs|jdS)z.Remove all items from the ModuleDict. N)r,clearrMr%r%r&rZszModuleDict.clearcCs||}||=|S)zRemove key from the ModuleDict and return its module. Args: key (string): key to pop from the ModuleDict r%r r!vr%r%r&pop_szModuleDict.popcCs |jS)z3Return an iterable of the ModuleDict keys. )r,rFrMr%r%r&rFiszModuleDict.keyscCs |jS)z>Return an iterable of the ModuleDict key/value pairs. )r,rrMr%r%r&roszModuleDict.itemscCs |jS)z5Return an iterable of the ModuleDict values. )r,rBrMr%r%r&rBuszModuleDict.valuescCst|tjstdt|jt|tttjfrL| D]\}}|||<q8n|t |D]r\}}t|tjstdt |dt|jt |dkst dt |dt t |d|d||d<qTd S) a!Update the :class:`~torch.nn.ModuleDict` with the key-value pairs from a mapping or an iterable, overwriting existing keys. .. note:: If :attr:`modules` is an ``OrderedDict``, a :class:`~torch.nn.ModuleDict`, or an iterable of key-value pairs, the order of new elements in it is preserved. Args: modules (iterable): a mapping (dictionary) from string to :class:`~torch.nn.Module`, or an iterable of key-value pairs of type (string, :class:`~torch.nn.Module`) zPModuleDict.update should be called with an iterable of key/value pairs, but got z$ModuleDict update sequence element # should be Iterable; isr has length ; 2 is requiredrrN)r1rwr rxryr'rr{rrr2r3r0 ValueError)r rdr!r4jmr%r%r&r|{s6     zModuleDict.update)N)r'r(r)r_r r3rr`rrrr rDrHrKrarNrrXboolrrrr rFrrrBr|r*r%r%r#r&r{s, +" r{cseZdZdZd$eeeddfdd ZddZe e edd d Z e e e e d d d Z d d Z e eddddZe dddZeedddZeeddddZfddZeddddZeeddddZedd d!Zd"d#ZZS)% ParameterListaDHolds parameters in a list. :class:`~torch.nn.ParameterList` can be used like a regular Python list, but Tensors that are :class:`~torch.nn.Parameter` are properly registered, and will be visible by all :class:`~torch.nn.Module` methods. Note that the constructor, assigning an element of the list, the :meth:`~torch.nn.ParameterDict.append` method and the :meth:`~torch.nn.ParameterDict.extend` method will convert any :class:`~torch.Tensor` into :class:`~torch.nn.Parameter`. Args: parameters (iterable, optional): an iterable of elements to add to the list. Example:: class MyModule(nn.Module): def __init__(self): super(MyModule, self).__init__() self.params = nn.ParameterList([nn.Parameter(torch.randn(10, 10)) for i in range(10)]) def forward(self, x): # ParameterList can act as an iterable, or be indexed using ints for i, p in enumerate(self.params): x = self.params[i // 2].mm(x) + p.mm(x) return x N)rBrcs(tt|d|_|dk r$||7}dS)Nr)rrr_size)r rBr#r%r&rszParameterList.__init__cCsTt|}t| |kr(t|ks8ntd||dkrL|t|7}t|SrfrgrCr%r%r&rhs    z#ParameterList._get_abs_string_indexrIcCsdSr.r%rCr%r%r&rDszParameterList.__getitem__)r r5rcCsdSr.r%rCr%r%r&rDscCsft|trJ|t|\}}}|}t|||D]}|||q2|S||}t|t |SdSr.) r1r@indicesr0r$rlr^rhgetattrr3)r r5startstopstepoutrkr%r%r&rDs  )r5paramrcCs8||}t|tjr(t|ts(t|}t|t||Sr.)rhr1torchTensorr rGr3)r r5rr%r%r&rHs zParameterList.__setitem__r6cCs|jSr.)rrMr%r%r&rNszParameterList.__len__cstfddttDS)Nc3s|]}|VqdSr.r%rjrMr%r& sz)ParameterList.__iter__..)rWrlr0rMr%rMr&rXszParameterList.__iter__ parametersrcCs ||Sr.ror rr%r%r&rqszParameterList.__iadd__cs tt|}dd|D}|S)NcSsg|]}|s|qSr%rOrQr%r%r&rSsz)ParameterList.__dir__..)rrrTrUr#r%r&rTszParameterList.__dir__)r"rcCs"t|}|jd7_|||<|S)znAppends a given value at the end of the list. Args: value (Any): value to append r)r0r)r r"Znew_idxr%r%r&r^szParameterList.appendcCsBt|tjrt|tjr*tdt|j|D]}||q.|S)zAppends values from a Python iterable to the end of the list. Args: values (iterable): iterable of values to append z@ParameterList.extend should be called with an iterable, but got ) r1rwr rrrxryr'r^)r rBr"r%r%r&rps zParameterList.extendcCsg}t|D]\}}t|tjrddd|D}|jsBdn d|}dt|t rbdndt |||}| d t |d |q | d t |d t |jq d |}|S) Nxcss|]}t|VqdSr.rirRr>r%r%r&rsz+ParameterList.extra_repr.. (GPU {}) {} containing: [{} of size {}{}]r r (): ): Object of type:  )r2r1rrjoinr>is_cudar; get_devicer typenamer^r3ryr'r Z child_linesrnpZsize_strZ device_strZparastrZtmpstrr%r%r& extra_reprs " zParameterList.extra_reprcOs tddS)Nz#ParameterList should not be called. RuntimeError)r r-rr%r%r&__call__!szParameterList.__call__)N)r'r(r)r_rr r rrhrrarDrr@rHrNrrXrqrTr^rpr3rrr*r%r%r#r&rs"     rcseZdZdZdeeeedd$d%Zd?eeeedd&d'd(Zeedd)d*Zeeeefdd+d,Zeedd-d.Zee eefdfddd/d0Z!edd1d2Z"d3d4Z#ddd5d6d7Z$ddd5d8d9Z%ddd5d:d;Z&Z'S)@ ParameterDictaHolds parameters in a dictionary. ParameterDict can be indexed like a regular Python dictionary, but Parameters it contains are properly registered, and will be visible by all Module methods. Other objects are treated as would be done by a regular Python dictionary :class:`~torch.nn.ParameterDict` is an **ordered** dictionary. :meth:`~torch.nn.ParameterDict.update` with other unordered mapping types (e.g., Python's plain ``dict``) does not preserve the order of the merged mapping. On the other hand, ``OrderedDict`` or another :class:`~torch.nn.ParameterDict` will preserve their ordering. Note that the constructor, assigning an element of the dictionary and the :meth:`~torch.nn.ParameterDict.update` method will convert any :class:`~torch.Tensor` into :class:`~torch.nn.Parameter`. Args: values (iterable, optional): a mapping (dictionary) of (string : Any) or an iterable of key-value pairs of type (string, Any) Example:: class MyModule(nn.Module): def __init__(self): super(MyModule, self).__init__() self.params = nn.ParameterDict({ 'left': nn.Parameter(torch.randn(5, 10)), 'right': nn.Parameter(torch.randn(5, 10)) }) def forward(self, x, choice): x = self.params[choice].mm(x) return x Nrcs*tt|i|_|dk r&||dSr.)rrr_keysr|rr#r%r&rJszParameterDict.__init__r}cCs*t|ts"tdt|jdn|SdS)NzUIndex given to ParameterDict cannot be used as a key as it is not a string (type is 'z8'). Open an issue on github if you need non-string keys.)r1r3rxryr'rr%r%r& _key_to_attrPs zParameterDict._key_to_attrcCs||}t||Sr.)rrr r!attrr%r%r&rDYs zParameterDict.__getitem__)r!r"rcCsBd|j|<||}t|tjr2t|ts2t|}t|||dSr.)rrr1rrr rG)r r!r"rr%r%r&rH]s   zParameterDict.__setitem__cCs |j|=||}t||dSr.)rrrJrr%r%r&rKis zParameterDict.__delitem__r6cCs t|jSr.)r0rrMr%r%r&rNnszParameterDict.__len__cCs t|jSr.)rWrrMr%r%r&rXqszParameterDict.__iter__cCstt|jSr.)reversedrArrMr%r%r& __reversed__tszParameterDict.__reversed__csttfddjDS)zJReturns a copy of this :class:`~torch.nn.ParameterDict` instance. c3s|]}||fVqdSr.r%rRrnrMr%r&r|sz%ParameterDict.copy..)rrrrMr%rMr&copywszParameterDict.copycCs ||jkSr.rrr%r%r&r~szParameterDict.__contains__)r!defaultrcCs||kr|||<||S)a&If key is in the ParameterDict, return its value. If not, insert `key` with a parameter `default` and return `default`. `default` defaults to `None`. Args: key (string): key to set default for default (Any): the parameter set to the key r%r r!rr%r%r& setdefaults zParameterDict.setdefaultcCs|jD] }||=q dS)z1Remove all items from the ParameterDict. N)rr)r rnr%r%r&rszParameterDict.clearcCs||}||=|S)zRemove key from the ParameterDict and return its parameter. Args: key (string): key to pop from the ParameterDict r%rr%r%r&rszParameterDict.popcCs.|j\}}d|j|<||}||=||fS)zcRemove and return the last inserted `(key, parameter)` pair from the ParameterDict N)rpopitem)r rn_valr%r%r&rs  zParameterDict.popitemcCs||kr||S|S)aReturn the parameter associated with key if present. Otherwise return default if provided, None if not. Args: key (string): key to get from the ParameterDict default (Parameter, optional): value to return if key not present r%rr%r%r&getszParameterDict.get)rFrrcstfdd|DS)zReturn a new ParameterDict with the keys provided Args: keys (iterable, string): keys to make the new ParameterDict from default (Parameter, optional): value to set for all keys c3s|]}|fVqdSr.r%rrr%r&rsz)ParameterDict.fromkeys..)r)r rFrr%rr&fromkeysszParameterDict.fromkeyscCs |jS)z6Return an iterable of the ParameterDict keys. )rrFrMr%r%r&rFszParameterDict.keyscsfddjDS)zAReturn an iterable of the ParameterDict key/value pairs. c3s|]}||fVqdSr.r%rrMr%r&rsz&ParameterDict.items..rrMr%rMr&rszParameterDict.itemscsfddjDS)z8Return an iterable of the ParameterDict values. c3s|]}|VqdSr.r%rrMr%r&rsz'ParameterDict.values..rrMr%rMr&rBszParameterDict.valuescCst|tjstdt|jt|ttfrH|D]\}}|||<q4nt|tj rtt |D]\}}|||<q`n|t |D]r\}}t|tjstdt |dt|jt |dkstdt |dt t |d|d||d<q|d S) aCUpdate the :class:`~torch.nn.ParameterDict` with the key-value pairs from a mapping or an iterable, overwriting existing keys. .. note:: If :attr:`parameters` is an ``OrderedDict``, a :class:`~torch.nn.ParameterDict`, or an iterable of key-value pairs, the order of new elements in it is preserved. Args: parameters (iterable): a mapping (dictionary) from string to :class:`~torch.nn.Parameter`, or an iterable of key-value pairs of type (string, :class:`~torch.nn.Parameter`) zTParametersDict.update should be called with an iterable of key/value pairs, but got z'ParameterDict update sequence element #rrrrrrN)r1rwr rxryr'rrrrsortedr2r3r0r)r rr! parameterrrr%r%r&r|s<       zParameterDict.updatecCsg}|D]\}}t|tjrddd|D}|jsBdn d|}dt|t rbdndt |||}| d t |d |q | d t |d t |jq d |}|S) Nrcss|]}t|VqdSr.rirr%r%r&rsz+ParameterDict.extra_repr..rrrr rrrrr)rr1rrrr>rr;rr rr^r3ryr'rr%r%r&rs " zParameterDict.extra_reprcCs tddS)Nz#ParameterDict should not be called.r)r rYr%r%r&rszParameterDict.__call__rrcCs|}|||Sr.rr|r rsrr%r%r&__or__s zParameterDict.__or__cCs|}|||Sr.rrr%r%r&__ror__s zParameterDict.__ror__cCs|||Sr.)r|)r rsr%r%r&__ior__ s zParameterDict.__ior__)N)N)N)N)(r'r(r)r_r rr3rrDrHrKrarNrrXrrrrrrrrrrrr rrFrrBrrr|rrrrrr*r%r%r#r&r%s4$       %r)"r collectionsrrrw itertoolsrrr8rr4rrr Ztorch._jit_internalr typingr r r rrrrrrrrrr+rbr{rrr%r%r%r&s"   0  }y