U dgI @s"ddlZddlZddlZddlmZddlmmmZddl mmZ ej jZ ddl mZmZmZmZmZmZmZddlmZddlmZddlmZmZddlmZddlmZdd l m!Z!m"Z"Gd d d ej#Z$eeeee%ee!fee$e$fd d dZ&eeee%ee!feeeej'e(feej'e)ffdddZ*eeedddZ+eee)dddZ,eee)dddZ-eee%dddZ.e"e%e"dddZ/e"ddd d!Z0d"d#Z1e1ej'ej'ej'd$d%d&Z2e1ej'ej'ej'd$d'd(Z3e1ej'ej'ej'd$d)d*Z4ee5dd+d,Z6dS)-N)TupleCallableDictSetListOptionalUnion) GraphModule)Node) ObserverBaseFakeQuantizeBase)getattr_from_fqn)is_activation_post_process)NSNodeTargetType NSResultsTypec@s4eZdZeZeZeZeZeZ dS)NodeInputOrOutputTypeN) __name__ __module__ __qualname__enumautoFP32INT8FP16UNKNOWNZ FP32_OR_INT8rr8/tmp/pip-unpacked-wheel-ua33x9lu/torch/ao/ns/fx/utils.pyrs r)nodegm logger_clsnode_type_to_io_type_mapreturncs|d}|d}|d}|d}|d}|d} |d} |d} |jd kr|j|kr`tjtjfS|j|krvtjtjfS|j|krtjtjfS|j|kr|jd } t| tst t | |||\} }||fStj tj fSn|jd kr|jd kst t|jt st t ||jtfd d | D}t|ttfs>|rr|jd } t| tsXt t | |||\} }||fStfdd |D}tfdd | D}|rtjtjfS|rtjtjfStj tj fSn |jdkr|jdkr |jd }t|tst t ||||\} }|tjfS|jdkr|jd }t|tsFt t ||||\} }|jd}|tjks|t |d|tjfS|j| kr|jd } t| tst t | |||\} }||fStj tj fStj tj fSdS)NZfuns_io_type_fp32Zfuns_io_type_fp16Zfuns_io_type_int8Zfuns_io_type_fp32_or_int8Zmods_io_type_fp32Zmods_io_type_int8mods_io_type_fp32_or_int8Zmeths_io_type_fp32_or_int8 call_functionr call_modulec3s|]}t|VqdSN isinstance.0 target_typemodrr Osz7get_node_first_input_and_output_type..c3s|]}t|VqdSr&r'r)r,rrr.asc3s|]}t|VqdSr&r'r)r,rrr.ds call_method dequantizetorz handling needs to be added)optargetrrrrargsr(r AssertionError$get_node_first_input_and_output_typerstrr anyr r torchZfloat16)rrr r!ZFUNS_IO_TYPE_FP32ZFUNS_IO_TYPE_FP16ZFUNS_IO_TYPE_INT8ZFUNS_IO_TYPE_FP32_OR_INT8ZMODS_IO_TYPE_FP32ZMODS_IO_TYPE_INT8MODS_IO_TYPE_FP32_OR_INT8ZMETHS_IO_TYPE_FP32_OR_INT8Z first_argZ_prev_node_input_typeZprev_node_output_type"is_known_fp32_or_int8_input_moduleZis_known_fp32_input_moduleZis_known_int8_input_module prev_nodeZcur_node_dtype_targetrr,rr6&s                            r6)rrr!r"csD|jd}t|tsdS|d}dd}|jdkrx|jtjkrL|||ddS|jtjtj tj tj fkrt|||dd SdS|jd kr@t|jt st t||jttjtjtjtjtjtjtjtjtjtjtjtjtjtjtjtj tj!tj"tj#tj$tj%tjtj&tj'frj(j)fSt*fd d |D}|r@t+|||SdS) z{ Returns the qparams (scale, zero_point) of the first input to `node`, if they can be inferred from the graph. rNr#cSsj|j||j|}}t|tr,t|jts0tt|trFt|jtsJtt||j}t||j}||fSr&)r4r(r r3r7r5r )rrZ scale_arg_idxZ zp_arg_idxZ scale_nodeZzp_nodeZ scale_objZzp_objrrr _get_scale_zp_from_function_argss   z@get_node_input_qparams.._get_scale_zp_from_function_argsr$rr%c3s|]}t|VqdSr&r'r)Z module_objrrr.sz)get_node_input_qparams..),r4r(r r2r3r9Zquantize_per_tensortoqaddZadd_relumulZmul_relur7r5r nnqZLinearZConv1dZConv2dnniqZ ConvReLU2dZConv3dZ BatchNorm2dZ BatchNorm3dZConvTranspose1dZConvTranspose2dZELUZ GroupNormZInstanceNorm1dZInstanceNorm2dZInstanceNorm3dZ LayerNormZ HardswishZ LeakyReLUZReLU6ZBNReLU2dZBNReLU3dZ ConvReLU1dZ ConvReLU3dZ LinearReLUZscaleZ zero_pointr8get_node_input_qparams)rrr!r<r:r=r;rr@rrFsb         rF)rrr"cCs|jdkrt||j}t|rt|jdks0tt|jdtsDt|jd}t|jt s^tt||j}t|rt|jdkstt|jdtst|jd}|S)a If node is not an observer, returns it. If node is an observer, navigates up the graph and returns the first parent which is not an observer. For example, graph: (node_non_obs), node = node_non_obs : returns node_non_obs graph: (node_non_obs -> obs0), node = obs0 : returns node_non_obs graph: (node_non_obs -> obs0 -> fq0), node = fq0 : returns node_non_obs r%rr) r2r r3rlenr4r5r(r r7rrZnode_objrrrreturn_first_non_observer_nodes     rIcCs*|jdkr&t||j}t|tjr&dSdS)aO Assumes that all non-param args occur first. Returns the number of non-param args expected for a node. For example, for F.linear(x, weight, bias) Returns 1, because x is a non-param arg and weight and bias are params. For lstm_mod(x, hid) Returns 2, because both x and hid are non-param args. r%r>r)r2r r3r(nnZLSTMrHrrrget_number_of_non_param_argss    rK)rr"cCst|jdkrgS|jdkr|jtjtjjjtjfksP|jtj tjjj tj fkrg}t dD] }t |j|t kr\| |q\|SdgS)a- Returns the indices of args of the node which we should attach loggers to, if input logging is enabled. For example, * for (x + y), returns [0, 1] * for (1 + y), returns [1] * for (x + 1), returns [0] * for (linear(x, w, b)) returns [0] * by default, returns [0] rr$r>)rGr4r2r3r9rBops quantizedoperatorrCrangetyper append)rresultirrr get_arg_indices_of_inputs_to_log*s    rTcCsPd}|jdkrt|j}n0|jdkrLt|jts6tt||j}t|}|S)z Returns a string representation of the type of the function or module pointed to by this node, or '' for other node types. )r$r/r%)r2r9typenamer3r(r7r5r )rrr+Z target_modrrrget_target_type_strEs    rW)results model_namer"c Csi}|D]n\}}d}|D]>\}}|D],\}} ||kr0t| sLt| dd}q0q0q0q |dk rr|||<q |||<q |S)a  Rekeys the layer name of a results dictionary to use node names from `model_name`. For example, transforms {'base_op_1_0': {'node_output': {'model_a': [{'ref_node_name': 'linear1', ...}]}}} into {'linear1': {'node_output': {'model_a': [{'ref_node_name': 'linear1', ...}]}}} Note: we cannot use these node names directly because they are not guaranteed to be consistent across models. This is why we extract the results first and rekey afterwards. NrZ ref_node_name)itemsrGr5) rXrYZ new_resultsZold_layer_nameresult_type_to_resultsZnew_layer_nameZ _result_typemodel_name_to_resultsZcur_model_nameZlist_of_resultsrrr'rekey_logger_info_on_node_name_of_modelTs   r])rXr"c Csd}|D]X\}}|D]B\}}|D],\}}t|dkr,|dddk r,|}qZq,q`qqfq |r|D]h\}}|D]V\}}||}|D]<\}}||krqtt|D]} || d} | || d<qqqqrdS)ay If `fqn` entries are filled in for one of the models in `results`, copies them over to any models which do not have them filled out. A common use case benefitting from this is comparing a model prepared by quantization to a quantized model. In this case, the model prepared by quantization would have `fqn` entries, and the quantized model would not. Nrfqn)rZrGrO) rXZmodel_name_with_fqnsZ layer_namer[Z result_typer\rYZ model_resultsZref_model_resultsrSr^rrrmaybe_add_missing_fqns{s(   r_csfddS)Nc s|^}}}t|trt|ts2t|trjt|trjg}t||D]$\}}||f|}|||q@|St|tjrt|tjr|jr|}|jr|}|j tj ks|j tj krdS||f|}||Sr&) r(tuplelistziprQr9TensorZ is_quantizedr0Zdtypefloat) r4kwargsZa0Za1Za_otherrXZel0Zel1new_argsfinnerrrris(   zGmaybe_dequantize_first_two_tensor_args_and_handle_tuples..innerr)rhrrgr8maybe_dequantize_first_two_tensor_args_and_handle_tuplessrj)xyr"cCs*t|}t||}dt||S)z Computes the SQNR between `x` and `y`. Args: x: Tensor or tuple of tensors y: Tensor or tuple of tensors Return: float or tuple of floats )r9Znormlog10)rkrlZPsZPnrrr compute_sqnrs rocCs"t||d|dS)z Computes the normalized L2 error between `x` and `y`. Args: x: Tensor or tuple of tensors y: Tensor or tuple of tensors Return: float or tuple of floats r>)r9sqrtsumrkrlrrrcompute_normalized_l2_errors rscCs(|dd}|dd}tjj||S)z Computes the cosine similarity between `x` and `y`. Args: x: Tensor or tuple of tensors y: Tensor or tuple of tensors Return: float or tuple of floats r)Zreshaper9rJZ functionalZcosine_similarityrrrrrcompute_cosine_similaritys  rucCs4|jdkr0|jtjtjtjtjtjtjfkr0dSdS)Nr$FT)r2r3r9rBrCrNcatstack)rrrrop_type_supports_shadowings "rx)7rrNr9Ztorch.nnrJZtorch.nn.intrinsic.quantizedZ intrinsicrMrEZtorch.nn.quantizedrDrLrAtypingrrrrrrrZtorch.fxr Ztorch.fx.graphr Ztorch.ao.quantizationr r Ztorch.ao.quantization.utilsr Ztorch.ao.quantization.quantizerZns_typesrrEnumrr7r6rcrdintrFrIrKrTrWr]r_rjrorsruboolrxrrrrs` $     }" R   '"