U dNZ @s^dZddlZddlmZddlmZmZmZmZm Z m Z m Z ddl Z ddl mZmZddlmZmZdedd d Zdedd d Zdedd dZdedddZGdddeeZGdddeZGdddeZGdddeZejeddejejddZejeddej ej!ddZ"ejeddejdd Z#eje d!Z$eje d!Z%e$Z&e%Z'ejeddej ej(ddd"Z)ejeej*ejddddd#Z+ejeej*dej,dd$Z-ejeddejejddZ.ejeddejd%Z/ejeddej ej!d&Z0ejeddej ej(d&Z1ejed'dej ej!d(d)Z2ejed'dej ej(d(d)Z3d*d+Z4d,d-Z5d.d/Z6d0d1Z7d2d3Z8dS)4zX This module implements modules which are used to perform fake quantization during QAT. N)Module)MovingAverageMinMaxObserverHistogramObserver%MovingAveragePerChannelMinMaxObserverFixedQParamsObserver)default_fixed_qparams_range_0to1_observer,default_fixed_qparams_range_neg1to1_observer _with_args)ABCabstractmethod)AnyTuplez torch.qscheme)qschemereturncCs|tjtjtjfkSN)torchper_channel_symmetricZper_channel_affine per_channel_affine_float_qparamsrrG/tmp/pip-unpacked-wheel-ua33x9lu/torch/ao/quantization/fake_quantize.py_is_per_channelsrcCs|tjtjfkSr)rper_tensor_symmetricper_tensor_affinerrrr_is_per_tensorsrcCs|tjtjfkSr)rrrrrrr_is_symmetric_quantsrcCs |tjfkSr)rrrrrr_is_float_qparamssrcseZdZUdZejed<ejed<fddZeddZ edd Z ej j de d d d dZej j ddZej j de d d ddZej j ddZeeZZS)FakeQuantizeBasea Base fake quantize module Any fake quantize implementation should derive from this class. Concrete fake quantize module should follow the same API. In forward, they will update the statistics of the observed Tensor and fake quantize the input. They should also provide a `calculate_qparams` function that computes the quantization parameters given the collected statistics. fake_quant_enabledobserver_enabledcsBt|dtjdgtjd|dtjdgtjddS)Nrdtyper)super__init__register_bufferrtensorZuint8self __class__rrr$/s zFakeQuantizeBase.__init__cCsdSrr)r(xrrrforward7szFakeQuantizeBase.forwardcKsdSrr)r(kwargsrrrcalculate_qparams;sz"FakeQuantizeBase.calculate_qparamsTN)enabledrcCs|rdnd|jd<dSNr r)rr(r/rrrenable_fake_quant?sz"FakeQuantizeBase.enable_fake_quantcCs|ddSNF)r2r'rrrdisable_fake_quantCsz#FakeQuantizeBase.disable_fake_quantcCs|rdnd|jd<dSr0)rr1rrrenable_observerGsz FakeQuantizeBase.enable_observercCs|ddSr3)r5r'rrrdisable_observerKsz!FakeQuantizeBase.disable_observer)T)T)__name__ __module__ __qualname____doc__rTensor__annotations__r$r r,r.jitexportboolr2r4r5r6 classmethodr with_args __classcell__rrr)rr!s"        rcs~eZdZUdZejed<ejed<eddffdd Zej j ddZ d d Z ej j d d Z fd dZfddZZS) FakeQuantizea Simulate the quantize and dequantize operations in training time. The output of this module is given by:: x_out = ( clamp(round(x/scale + zero_point), quant_min, quant_max) - zero_point ) * scale * :attr:`scale` defines the scale factor used for quantization. * :attr:`zero_point` specifies the quantized value to which 0 in floating point maps to * :attr:`fake_quant_enabled` controls the application of fake quantization on tensors, note that statistics can still be updated. * :attr:`observer_enabled` controls statistics collection on tensors * :attr:`dtype` specifies the quantized dtype that is being emulated with fake-quantization, allowable values are torch.qint8 and torch.quint8. Args: observer (module): Module for observing statistics on input tensors and calculating scale and zero-point. observer_kwargs (optional): Arguments for the observer module Attributes: activation_post_process (Module): User provided module that collects statistics on the input tensor and provides a method to calculate scale and zero-point. scale zero_pointNc stt|dk r|dk r||ks*td|dtj}t|dr^tt|didid|}t|j |ksvtd|t|j kstd| ||d|f||_ |j j |_ |j j|_t|j jrtj}ntj}|dtjd gtjd |d tjd g|d |j j|_|j j|_t|j d r4|j jnd|_t|jsdt|jsdtdt|jt|j|_dS)Nz1quant_min must be less than or equal to quant_maxr"pkeywordszquant_min out of boundzquant_max out of bound) quant_min quant_maxrDg?r!rErch_axiszYOnly per channel and per tensor quantization are supported in fake quantize got qscheme: )r#r$AssertionErrorgetrquint8hasattrgetattrZiinfominmaxupdateactivation_post_processrHrIrrfloatintr%r&r"rJrrstris_per_channel)r(observerrHrIobserver_kwargsr"Zzero_point_dtyper)rrr$usH           zFakeQuantize.__init__cCs |jSrrTr.r'rrrr.szFakeQuantize.calculate_qparamscCs|jddkr|||\}}||jj||jj}}|jj|jkrp|j |j|j |j|j ||j ||j ddkr|j rt ||j|j|j|jj|jj}nt ||j|j|jj|jj}|S)Nrr )rrTdetachr.torDZdevicerEshaperesize_copy_rrXrZ fake_quantize_per_channel_affinerJrHrIZfake_quantize_per_tensor_affine)r(XZ_scaleZ _zero_pointrrrr,s6   zFakeQuantize.forwardc Cs0d|j|j|jj|jj|j|j|j|j |j S)Nzfake_quant_enabled={}, observer_enabled={}, quant_min={}, quant_max={}, dtype={}, qscheme={}, ch_axis={}, scale={}, zero_point={}) formatrrrTrHrIr"rrJrDrEr'rrr extra_reprszFakeQuantize.extra_reprcs4tt|||||j||d<|j||d<dSNrDrE)r#rC_save_to_state_dictrDrE)r(Z destinationprefixZ keep_varsr)rrresz FakeQuantize._save_to_state_dictc sddg}|D]} || } | |kr|| } | dkr@|j| jn| dksLt|j| jtjr| dkrz|j| q| dkst|j| q |r | | q t t | |||||||dSrd) rDr_r^rLrErr=Z is_scriptingr`appendr#rC_load_from_state_dict) r(Z state_dictrfZlocal_metadatastrictZ missing_keysZunexpected_keysZ error_msgsZ local_statenamekeyvalr)rrrhs*    z"FakeQuantize._load_from_state_dict)r7r8r9r:rr;r<rr$r=r>r.r,rcrerhrBrrr)rrCQs   %   rCcs@eZdZdZfddZejjddZejjddZ Z S)FixedQParamsFakeQuantizez Simulate quantize and dequantize with fixed quantization parameters in training time. Only per tensor quantization is supported. csltj|dt|jtks2td|jjtjf||_|jj |_ |jj |_ t |j shtdt |j dS)NrYz%s's observer must be a %szWOnly per tensor quantization is supported FixedQParamsFakeQuantize module, got qscheme:)r#r$typerTrrLr*r7Z _observer_ctrrDrErrrW)r(rYr)rrr$s  z!FixedQParamsFakeQuantize.__init__cCs |j|jfSr)rDrEr'rrrr.sz*FixedQParamsFakeQuantize.calculate_qparamsc Cs,d|j|j|j|j|j|jj|jj|j S)Nzufake_quant_enabled={}, observer_enabled={}, scale={}, zero_point={}, dtype={}, quant_min={}, quant_max={}, qscheme={}) rbrrrDrEr"rTrHrIrr'rrrrcsz#FixedQParamsFakeQuantize.extra_repr) r7r8r9r:r$rr=r>r.rcrBrrr)rrms   rmcseZdZdZeddfeeeeddfdd Zej j e ej ej fdd d Z ej j edd d Zej ej d ddZZS)FusedMovingAvgObsFakeQuantizeaFused module that is used to observe the input tensor (compute min/max), compute scale/zero_point and fake_quantize the tensor. This module uses calculation similar MovingAverageMinMaxObserver for the inputs, to compute the min/max values in order to compute the scale/zero_point. The qscheme input in the observer is used to differentiate between symmetric/affine quantization scheme. The output of this module is given by x_out = (clamp(round(x/scale + zero_point), quant_min, quant_max)-zero_point)*scale Similar to :class:`~torch.ao.quantization.FakeQuantize`, and accepts the same attributes as the base class. rN)rYrHrIrZrc srtj|||f|t|jttfs,td|dtj dgtj d|dtj dgtj dt |jj |_ dS)NzLFused observer+fake_quant module only works with MovingAverageMinMaxObserverrr r!r)r#r$ isinstancerTrrrLr%rr&longrris_symmetric_quant)r(rYrHrIrZr)rrr$sz&FusedMovingAvgObsFakeQuantize.__init__)rcCs |jSrr[r'rrrr.sz/FusedMovingAvgObsFakeQuantize.calculate_qparamsc Cs2d|j|j|j|j|j|jj|jj|j |jj S)Nzfake_quant_enabled={}, observer_enabled={}, scale={}, zero_point={}, dtype={}, quant_min={}, quant_max={}, qscheme={}, reduce_range={}) rbrrrDrEr"rTrHrIr reduce_ranger'rrrrc#sz(FusedMovingAvgObsFakeQuantize.extra_repr)rarcCsDt||j|j|jj|jj|j|j|jj |jj |jj |j |j |j Sr)rZfused_moving_avg_obs_fake_quantrrrTZmin_valZmax_valrDrEaveraging_constantrHrIrJrXrt)r(rarrrr,4sz%FusedMovingAvgObsFakeQuantize.forward)r7r8r9r:rr rVr$rr=r>r r;r.rWrcr,rBrrr)rrps rprqT)rYrHrIr"rruiFr )rYrHrIr"rvrn)rYrHrIr"rrurJ)rYrr"rHrIrJrv)rYrrJr"rv)rYrHrIr")rYrHrIr"rig0?)rYrHrIr"rZepscCsDt|tjjr@|jjddd}tdd|}|dkp>|dkSdS)zM Returns true if given mod is an instance of FakeQuantize script module. .r z\.___torch_mangle_\d+z0torch.ao.quantization.fake_quantize.FakeQuantizezAtorch.ao.quantization.fake_quantize.FusedMovingAvgObsFakeQuantizeF) rrrr=ZRecursiveScriptModuleZ_cZqualified_namesplitresub)modsuffixrjrrr_is_fake_quant_script_modulesrcCst|tst|r|dS)z Disable fake quantization for this module, if applicable. Example usage:: # model is any PyTorch model model.apply(torch.ao.quantization.disable_fake_quant) N)rrrrr4r}rrrr4sr4cCst|tst|r|dS)z Enable fake quantization for this module, if applicable. Example usage:: # model is any PyTorch model model.apply(torch.ao.quantization.enable_fake_quant) N)rrrrr2rrrrr2sr2cCst|tst|r|dS)z Disable observation for this module, if applicable. Example usage:: # model is any PyTorch model model.apply(torch.ao.quantization.disable_observer) N)rrrrr6rrrrr6sr6cCst|tst|r|dS)z Enable observation for this module, if applicable. Example usage:: # model is any PyTorch model model.apply(torch.ao.quantization.enable_observer) N)rrrrr5rrrrr5sr5)9r:rZtorch.nnrZtorch.ao.quantization.observerrrrrrrr r{abcr r typingr r r?rrrrrrCrmrprArNrZdefault_fake_quantZqint8rZdefault_weight_fake_quantZdefault_dynamic_fake_quantZ.default_fixed_qparams_range_neg1to1_fake_quantZ+default_fixed_qparams_range_0to1_fake_quantZ*default_symmetric_fixed_qparams_fake_quantZ'default_affine_fixed_qparams_fake_quantrZ%default_per_channel_weight_fake_quantrZdefault_embedding_fake_quantZquint4x2Z!default_embedding_fake_quant_4bitZdefault_histogram_fake_quantZdefault_fused_act_fake_quantZdefault_fused_wt_fake_quantZ'default_fused_per_channel_wt_fake_quantZ(fused_wt_fake_quant_range_neg_127_to_127Z4fused_per_channel_wt_fake_quant_range_neg_127_to_127rr4r2r6r5rrrrs $ 0D