U d@sPddlmZmZddlZddlmZGdddejjZd ddZ d d d Z dS) )OptionalListN)_NnapiSerializercseZdZUdZeejjje d<e ej e d<e ej e d<ej j ej e ej e ee edfdd Zejje ej dd d Ze ej e ej d d d ZZS) NnapiModulezTorch Module that wraps an NNAPI Compilation. This module handles preparing the weights, initializing the NNAPI TorchBind object, and adjusting the memory formats of all inputs and outputs. compweights out_templates)shape_compute_module ser_modelr inp_mem_fmts out_mem_fmtscs8t||_||_||_||_||_g|_d|_dSN) super__init__r r rr r rr)selfr r rr r  __class__A/tmp/pip-unpacked-wheel-ua33x9lu/torch/backends/_nnapi/prepare.pyrs zNnapiModule.__init__)argscCsX|jdkst|j|j||_dd|jD|_tjj }| |j|j||_dS)NcSsg|] }|qSr) contiguous).0wrrr )sz$NnapiModule.init..) rAssertionErrorr preparer rrtorchclasses_nnapi Compilationinit)rrrrrrr %s  zNnapiModule.init)rreturnc Cs4|jdkr|||j}|dk s&tdd|jD}t|t|jksLtg}tt|D]Z}|j|}|dkr|||q\|dkr||| ddddq\t dq\| ||t|t|j ksttt|jD]F}|j |}|dkrq|dkr&|| dddd||<qt dq|S) NcSsg|]}t|qSr)rZ empty_like)routrrrr3sz'NnapiModule.forward..rzInvalid mem_fmt)rr$) rr rrlenr rangeappendrZpermute Exceptionrunr )rrrZoutsZ fixed_argsidxfmtrrrforward.s0           zNnapiModule.forward)__name__ __module__ __qualname____doc__rrrrr__annotations__rTensornnModuleintrjitexportr r- __classcell__rrrrrs rFc Cst|||||\}}}}} } t||||| } Gdddtjj} | | } tj| }dddtt |D}| dkr~d}ndd dt| D}| d |d |d |d |S)Ncs eZdZdZfddZZS)z5convert_model_to_nnapi..NnapiInterfaceWrappera0NNAPI list-ifying and de-list-ifying wrapper. NNAPI always expects a list of inputs and provides a list of outputs. This module allows us to accept inputs as separate arguments. It returns results as either a single tensor or tuple, matching the original module. cst||_dSr )rrmod)rr:rrrrbs z>convert_model_to_nnapi..NnapiInterfaceWrapper.__init__)r.r/r0r1rr9rrrrNnapiInterfaceWrapperZsr;z, css|]}d|VqdS)Zarg_Nrrr+rrr isz)convert_model_to_nnapi..rz retvals[0]css|]}d|dVqdS)zretvals[z], Nrr<rrrr=mszdef forward(self, z): retvals = self.mod([z]) return  ) process_for_nnapirrr4r5r7scriptjoinr'r&define)modelinputs serializer return_shapesuse_int16_for_qint16r ser_model_tensor used_weightsr r retval_countZ nnapi_modelr;Zwrapper_model_pyZ wrapper_modelarg_listZret_exprrrrconvert_model_to_nnapiOs*   rMcCstj|}t|tjr|g}|p,td|d}||||\}}}}} } tj|tjd} Gdddtj j } tj | } dgdd| D}| d || | |||| fS) N)configrH)Zdtypec@seZdZdZdS)z-process_for_nnapi..ShapeComputeModulezCode-gen-ed module for tensor shape computation module.prepare will mutate ser_model according to the computed operand shapes, based on the shapes of args. Returns a list of output templates. N)r.r/r0r1rrrrShapeComputeModulesrOz\def prepare(self, ser_model: torch.Tensor, args: List[torch.Tensor]) -> List[torch.Tensor]: cSsg|]}d|dqS)z r?r)rlinerrrrsz%process_for_nnapi..r>)rr7freeze isinstancer3rZserialize_modelZtensorZint32r4r5rArCrB)rDrErFrGrHr rJr r Zshape_compute_linesrKrIrOr Zreal_shape_compute_linesrrrr@us0   r@)NNF)NNF) typingrrrZ torch.backends._nnapi.serializerrr4r5rrMr@rrrrs  H &