U dA@sddlZddlZGdddejjZGdddejjZGdddejjZGdd d ejjZGd d d ejjZGd d d ejj Z dS)Ncs>eZdZdZd fdd ZddZddZed d ZZ S) ReLU6aApplies the element-wise function: :math:`\text{ReLU6}(x) = \min(\max(x_0, x), q(6))`, where :math:`x_0` is the zero_point, and :math:`q(6)` is the quantized representation of number 6. Args: inplace: can optionally do the operation in-place. Default: ``False`` Shape: - Input: :math:`(N, *)` where `*` means, any number of additional dimensions - Output: :math:`(N, *)`, same shape as the input .. image:: ../scripts/activation_images/ReLU6.png Examples:: >>> m = nn.quantized.ReLU6() >>> input = torch.randn(2) >>> input = torch.quantize_per_tensor(input, 1.0, 0, dtype=torch.qint32) >>> output = m(input) Fcstt||||_dSN)superr__init__inplace)selfr __class__I/tmp/pip-unpacked-wheel-ua33x9lu/torch/nn/quantized/modules/activation.pyrszReLU6.__init__cCstjj||jSr)torchops quantizedZrelu6rrinputr r r forwardsz ReLU6.forwardcCsdS)NZQuantizedReLU6r rr r r _get_name"szReLU6._get_namecCs t|jSr)rr)modr r r from_float%szReLU6.from_float)F) __name__ __module__ __qualname____doc__rrr staticmethodr __classcell__r r rr rs rcsHeZdZdZfddZddZddZedd Ze d d Z Z S) HardswishzThis is the quantized version of :class:`~torch.nn.Hardswish`. Args: scale: quantization scale of the output tensor zero_point: quantization zero point of the output tensor cstt|||_||_dSr)rrrscale zero_point)rrrrr r r0szHardswish.__init__cCstjjjj||j|jdS)N)rr)r nnr functionalZ hardswishrrrr r r r5s  zHardswish.forwardcCsdS)NZQuantizedHardswishr rr r r r9szHardswish._get_namecCs |j\}}tt|t|Sr)activation_post_processcalculate_qparamsrfloatintrrrr r r r<szHardswish.from_floatcCs|t|t|Sr)r#r$clsrrrr r r from_referenceAszHardswish.from_reference rrrrrrrrr classmethodr(rr r rr r)s  rcsJeZdZdZd fdd ZddZddZed d Ze d d Z Z S)ELUzThis is the quantized equivalent of :class:`~torch.nn.ELU`. Args: scale: quantization scale of the output tensor zero_point: quantization zero point of the output tensor alpha: the alpha constant ?cs tt||||_||_dSr)rr+rrr)rrralpharr r rMsz ELU.__init__cCstjjj||j|j|jSr)r rrr Zelurrr-rr r r rRs  z ELU.forwardcCsdS)NZ QuantizedELUr rr r r rVsz ELU._get_namecCs$|j\}}tt|t||jSr)r!r"r+r#r$r-r%r r r rYszELU.from_floatcCs|t|t||jSr)r#r$r-r&r r r r(^szELU.from_reference)r,r)r r rr r+Es r+csXeZdZdZdeeeeddfdd Zdd Zd d Z e d d Z e ddZ Z S) LeakyReLUaThis is the quantized equivalent of :class:`~torch.nn.LeakyReLU`. Args: scale: quantization scale of the output tensor zero_point: quantization zero point of the output tensor negative_slope: Controls the angle of the negative slope. Default: 1e-2 {Gz?FN)rrnegative_sloperreturncsH||d}t|||dtj|f||dtj|f|dS)N)devicedtyperr)rrZregister_bufferr Ztensor)rrrr0rr2r3Zfactory_kwargsrr r rjs zLeakyReLU.__init__cCstjj||j|j|j|jSr)r r rZ leaky_relur0rrrrr r r rqszLeakyReLU.forwardcCsdS)NZQuantizedLeakyReLUr rr r r ruszLeakyReLU._get_namecCs(|j\}}|t|t||j|jSr)r!r"r#r$r0rr&r r r rxszLeakyReLU.from_floatcCs|t|t||j|jSr)r#r$r0rr&r r r r(}szLeakyReLU.from_reference)r/FNN)rrrrr#r$boolrrrr*rr(rr r rr r.bs r.cs<eZdZdZeedfdd ZddZeddZ Z S) SigmoidzThis is the quantized equivalent of :class:`~torch.nn.Sigmoid`. Args: scale: quantization scale of the output tensor zero_point: quantization zero point of the output tensor ) output_scaleoutput_zero_pointcst||_||_dSr)rrr6r7)rr6r7rr r rs zSigmoid.__init__cCstjj||j|jSr)r r rZsigmoidr6r7rr r r rszSigmoid.forwardcCs |j\}}|t|t|Sr)r!r"r#r$)r'rr6r7r r r rszSigmoid.from_float) rrrrr#r$rrr*rrr r rr r5s r5csJeZdZdZdfdd ZddZd d Zed d Ze d dZ Z S)Softmaxa,This is the quantized version of :class:`~torch.nn.Softmax`. Args: dim: A dimension along which Softmax will be computed (so every slice along dim will sum to 1). scale: quantization scale of the output tensor zero_point: quantization zero point of the output tensor Nr,rcs t||_||_||_dSr)rrdimrr)rr9rrrr r rs zSoftmax.__init__cCs@|j}|dkr(d}tjjd||}tjj|||j|j S)Nsoftmax) r9r rr Z_get_softmax_dimr rr;rr)rrr9 stacklevelr r r rszSoftmax.forwardcCsdS)NZQuantizedSoftmaxr rr r r rszSoftmax._get_namecCs$|j\}}t|jt|t|Sr)r!r"r8r9r#r$r%r r r rszSoftmax.from_floatcCs||jt|t|Sr)r9r#r$r&r r r r(szSoftmax.from_reference)Nr,rr)r r rr r8s  r8) r Ztorch.nn.quantized.functionalrZReLUrrr+r.r5r8r r r r s%