U 9%e@sdZddlZddlZddlZddlmZddlmZmZm Z m Z m Z ddl Z ddl mmZddl mZmZddlmZddlmZmZmZmZmZmZdd lmZdd lmZdd lm Z dd l!m"Z"m#Z#m$Z$d dl%m&Z&ddl'm(Z(e#)e*Z+erddl,m-Z-e"rddl.m/Z/er0ddl0m1Z1e#)e*Z+dZ2dZ3dgZ4eGdddeZ5eGdddeZ6eGdddeZ7ddZ8dndd Z9Gd!d"d"ej:Z;d#d$Ze e?d'd(d)Z@Gd*d+d+ej:ZAGd,d-d-ej:ZBd.d/ZCeeeeed0d1d2ZDGd3d4d4ej:ZEGd5d6d6ej:ZFGd7d8d8ej:ZGGd9d:d:ej:ZHGd;d<dd>eZJd?ZKd@ZLGdAdBdBeJZMGdCdDdDeJZNedEeKGdFdGdGeJZOedHeKGdIdJdJeJZPdKdLZQdpeReRdNdOdPZSGdQdRdRej:ZTGdSdTdTej:ZUGdUdVdVej:ZVeedWdXdYZWeedZd[d\ZXd]d^ZYd_d`ZZdadbZ[Gdcdddde\Z]e je?eRe?dedfdgZ^dhdiZ_Gdjdkdkej:Z`Gdldmdmej:ZadS)qz PyTorch DETA model.N) dataclass)DictListOptionalTupleUnion)Tensornn)ACT2FN) ModelOutputadd_start_docstrings%add_start_docstrings_to_model_forwardis_scipy_availableis_vision_availablereplace_return_docstrings)BaseModelOutput)PreTrainedModel)meshgrid)is_torchvision_availableloggingrequires_backends) AutoBackbone) DetaConfig)center_to_corners_format) batched_nmslinear_sum_assignmentrzjozhang97/deta-swin-large-o365c@s~eZdZUdZdZejed<dZejed<dZ ejed<dZ e e ejed<dZ e e ejed<dZe e ejed<dS) DetaDecoderOutputa Base class for outputs of the DetaDecoder. This class adds two attributes to BaseModelOutputWithCrossAttentions, namely: - a stacked tensor of intermediate decoder hidden states (i.e. the output of each decoder layer) - a stacked tensor of intermediate reference points. Args: last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the model. intermediate_hidden_states (`torch.FloatTensor` of shape `(batch_size, config.decoder_layers, num_queries, hidden_size)`): Stacked intermediate hidden states (output of each layer of the decoder). intermediate_reference_points (`torch.FloatTensor` of shape `(batch_size, config.decoder_layers, sequence_length, hidden_size)`): Stacked intermediate reference points (reference points of each layer of the decoder). hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` and `config.add_cross_attention=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the weighted average in the cross-attention heads. Nlast_hidden_stateintermediate_hidden_statesintermediate_reference_points hidden_states attentionscross_attentions)__name__ __module__ __qualname____doc__r!torch FloatTensor__annotations__r"r#r$rrr%r&r.r.e/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/transformers/models/deta/modeling_deta.pyr Ds r c@seZdZUdZdZejed<dZejed<dZ ejed<dZ ejed<dZ e e ejed<dZe e ejed<dZe e ejed <dZe ejed <dZe e ejed <dZe e ejed <dZe ejed <dZe ejed<dS)DetaModelOutputa Base class for outputs of the Deformable DETR encoder-decoder model. Args: init_reference_points (`torch.FloatTensor` of shape `(batch_size, num_queries, 4)`): Initial reference points sent through the Transformer decoder. last_hidden_state (`torch.FloatTensor` of shape `(batch_size, num_queries, hidden_size)`): Sequence of hidden-states at the output of the last layer of the decoder of the model. intermediate_hidden_states (`torch.FloatTensor` of shape `(batch_size, config.decoder_layers, num_queries, hidden_size)`): Stacked intermediate hidden states (output of each layer of the decoder). intermediate_reference_points (`torch.FloatTensor` of shape `(batch_size, config.decoder_layers, num_queries, 4)`): Stacked intermediate reference points (reference points of each layer of the decoder). decoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, num_queries, hidden_size)`. Hidden-states of the decoder at the output of each layer plus the initial embedding outputs. decoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, num_queries, num_queries)`. Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the self-attention heads. cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_queries, num_heads, 4, 4)`. Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the weighted average in the cross-attention heads. encoder_last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder of the model. encoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the encoder at the output of each layer plus the initial embedding outputs. encoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_queries, num_heads, 4, 4)`. Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the self-attention heads. enc_outputs_class (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.num_labels)`, *optional*, returned when `config.with_box_refine=True` and `config.two_stage=True`): Predicted bounding boxes scores where the top `config.two_stage_num_proposals` scoring bounding boxes are picked as region proposals in the first stage. Output of bounding box binary classification (i.e. foreground and background). enc_outputs_coord_logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, 4)`, *optional*, returned when `config.with_box_refine=True` and `config.two_stage=True`): Logits of predicted bounding boxes coordinates in the first stage. Ninit_reference_pointsr!r"r#decoder_hidden_statesdecoder_attentionsr&encoder_last_hidden_stateencoder_hidden_statesencoder_attentionsenc_outputs_classenc_outputs_coord_logits)r'r(r)r*r1r+r,r-r!r"r#r2rrr3r&r4r5r6r7r8r.r.r.r/r0js *r0c@sDeZdZUdZdZeejed<dZ ee ed<dZ ejed<dZ ejed<dZ eee ed<dZeejed<dZeejed <dZeejed <dZeejed <dZeeejed <dZeeejed <dZeeejed<dZeejed<dZeeejed<dZeeejed<dZeed<dZeed<dS)DetaObjectDetectionOutputa Output type of [`DetaForObjectDetection`]. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` are provided)): Total loss as a linear combination of a negative log-likehood (cross-entropy) for class prediction and a bounding box loss. The latter is defined as a linear combination of the L1 loss and the generalized scale-invariant IoU loss. loss_dict (`Dict`, *optional*): A dictionary containing the individual losses. Useful for logging. logits (`torch.FloatTensor` of shape `(batch_size, num_queries, num_classes + 1)`): Classification logits (including no-object) for all queries. pred_boxes (`torch.FloatTensor` of shape `(batch_size, num_queries, 4)`): Normalized boxes coordinates for all queries, represented as (center_x, center_y, width, height). These values are normalized in [0, 1], relative to the size of each individual image in the batch (disregarding possible padding). You can use [`~DetaProcessor.post_process_object_detection`] to retrieve the unnormalized bounding boxes. auxiliary_outputs (`list[Dict]`, *optional*): Optional, only returned when auxilary losses are activated (i.e. `config.auxiliary_loss` is set to `True`) and labels are provided. It is a list of dictionaries containing the two above keys (`logits` and `pred_boxes`) for each decoder layer. last_hidden_state (`torch.FloatTensor` of shape `(batch_size, num_queries, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the decoder of the model. decoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, num_queries, hidden_size)`. Hidden-states of the decoder at the output of each layer plus the initial embedding outputs. decoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, num_queries, num_queries)`. Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the self-attention heads. cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_queries, num_heads, 4, 4)`. Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the weighted average in the cross-attention heads. encoder_last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder of the model. encoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the encoder at the output of each layer plus the initial embedding outputs. encoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, sequence_length, num_heads, 4, 4)`. Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the self-attention heads. intermediate_hidden_states (`torch.FloatTensor` of shape `(batch_size, config.decoder_layers, num_queries, hidden_size)`): Stacked intermediate hidden states (output of each layer of the decoder). intermediate_reference_points (`torch.FloatTensor` of shape `(batch_size, config.decoder_layers, num_queries, 4)`): Stacked intermediate reference points (reference points of each layer of the decoder). init_reference_points (`torch.FloatTensor` of shape `(batch_size, num_queries, 4)`): Initial reference points sent through the Transformer decoder. enc_outputs_class (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.num_labels)`, *optional*, returned when `config.with_box_refine=True` and `config.two_stage=True`): Predicted bounding boxes scores where the top `config.two_stage_num_proposals` scoring bounding boxes are picked as region proposals in the first stage. Output of bounding box binary classification (i.e. foreground and background). enc_outputs_coord_logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, 4)`, *optional*, returned when `config.with_box_refine=True` and `config.two_stage=True`): Logits of predicted bounding boxes coordinates in the first stage. Nloss loss_dictlogits pred_boxesauxiliary_outputsr1r!r"r#r2r3r&r4r5r6r7r8)r'r(r)r*r:rr+r,r-r;rr<r=r>rr1r!r"r#r2rr3r&r4r5r6r7r8r.r.r.r/r9s$ ; r9cstfddt|DS)Ncsg|]}tqSr.)copydeepcopy).0imoduler.r/ sz_get_clones..)r ModuleListrange)rDNr.rCr/ _get_clonessrIh㈵>cCs8|jddd}|j|d}d|j|d}t||S)NrrminmaxrL)clampr+log)xepsx1Zx2r.r.r/inverse_sigmoids rTcs4eZdZdZfddZfddZddZZS)DetaFrozenBatchNorm2dz BatchNorm2d where the batch statistics and the affine parameters are fixed. Copy-paste from torchvision.misc.ops with added eps before rqsrt, without which any other models than torchvision.models.resnet[18,34,50,101] produce nans. csVt|dt||dt||dt||dt|dS)Nweightbias running_mean running_var)super__init__Zregister_bufferr+oneszeros)selfn __class__r.r/r[ s  zDetaFrozenBatchNorm2d.__init__c s2|d}||kr||=t|||||||dS)NZnum_batches_tracked)rZ_load_from_state_dict) r^Z state_dictprefixZlocal_metadatastrictZ missing_keysZunexpected_keysZ error_msgsZnum_batches_tracked_keyr`r.r/rbsz+DetaFrozenBatchNorm2d._load_from_state_dictcCst|jdddd}|jdddd}|jdddd}|jdddd}d}|||}|||}|||S)NrrJ)rVreshaperWrYrXZrsqrt)r^rQrVrWrYrXepsilonscaler.r.r/forwards zDetaFrozenBatchNorm2d.forward)r'r(r)r*r[rbri __classcell__r.r.r`r/rUs  rUcCs|D]\}}t|tjrpt|j}|jj|j|j j|j |j j|j |j j|j ||j |<t t|dkrt|qdS)z Recursively replace all `torch.nn.BatchNorm2d` with `DetaFrozenBatchNorm2d`. Args: model (torch.nn.Module): input model rN)Znamed_children isinstancer BatchNorm2drUZ num_featuresrVdataZcopy_rWrXrYZ_moduleslenlistchildrenreplace_batch_norm)modelnamerDZ new_moduler.r.r/rq*s   rqcs4eZdZdZfddZejejdddZZS)#DetaBackboneWithPositionalEncodingsz Backbone model with positional embeddings. nn.BatchNorm2d layers are replaced by DetaFrozenBatchNorm2d as defined above. c stt|j}tt|W5QRX||_|jj |_ |jj dkr|j D]*\}}d|krXd|krXd|krX| dqXt||_dS)NZresnetzstages.1zstages.2zstages.3F)rZr[r from_configZbackbone_configr+no_gradrqrrZchannelsintermediate_channel_sizesZ model_typenamed_parametersrequires_grad_build_position_encodingposition_embedding)r^configbackbonersZ parameterr`r.r/r[Hs      z,DetaBackboneWithPositionalEncodings.__init__) pixel_values pixel_maskc Cs||j}g}g}|D]^}tjj|d|jdddtj d}| |||j }| ||f| |q||fS)z Outputs feature maps of latter stages C_3 through C_5 in ResNet if `config.num_feature_levels > 1`, otherwise outputs feature maps of C_5. Nsizer) rrZ feature_mapsr functional interpolatefloatshapetor+boolr{dtypeappend) r^r~rfeaturesoutposZ feature_mapmaskposition_embeddingsr.r.r/riYs . z+DetaBackboneWithPositionalEncodings.forward) r'r(r)r*r[r+rrirjr.r.r`r/rtAs rt)rr target_lencCsf|\}}|dk r|n|}|ddddddf|d|||}d|}||t|jS)zs Expands attention_mask from `[batch_size, seq_len]` to `[batch_size, 1, target_seq_len, source_seq_len]`. Nr?)rexpandr masked_fillrr+finforL)rrr batch_size source_lenZ expanded_maskZ inverted_maskr.r.r/ _expand_maskos  *rcs*eZdZdZd fdd Zdd ZZS) DetaSinePositionEmbeddingz This is a more standard version of the position embedding, very similar to the one used by the Attention is all you need paper, generalized to work on images. @'FNcsPt||_||_||_|dk r4|dkr4td|dkrFdtj}||_dS)NFz+normalize should be True if scale is passedr) rZr[ embedding_dim temperature normalize ValueErrormathpirh)r^rrrrhr`r.r/r[s  z"DetaSinePositionEmbedding.__init__c Cs|dkrtd|jdtjd}|jdtjd}|jrd}|d|ddddddf||j}|d|ddddddf||j}tj|jtj|jd}|j dtj |dd d |j}|dddddddf|}|dddddddf|}tj |ddddddd ddf |dddddddddf fd d d}tj |ddddddd ddf |dddddddddf fd d d}tj||fdd d ddd} | S)NzNo pixel mask providedrrrgư>?rerdevicefloorZ rounding_moderdimr )rcumsumr+float32rrharangerrrdivstacksincosflattencatpermute) r^r~rZy_embedZx_embedrRdim_tZpos_xZpos_yrr.r.r/ris ,,   \\z!DetaSinePositionEmbedding.forward)rrFNr'r(r)r*r[rirjr.r.r`r/r~s rcs,eZdZdZdfdd Zd ddZZS) DetaLearnedPositionEmbeddingzN This module learns positional embeddings up to a fixed maximum size. cs*ttd||_td||_dS)N2)rZr[r Embeddingrow_embeddingscolumn_embeddings)r^rr`r.r/r[s z%DetaLearnedPositionEmbedding.__init__Nc Cs|jdd\}}tj||jd}tj||jd}||}||}tj|d|dd|dd|dgdd} | ddd} | d} | |jdddd} | S)Nrrrrrerr) rr+rrrrr unsqueezerepeatr) r^r~rheightwidthZ width_valuesZ height_valuesZx_embZy_embrr.r.r/ris  2 z$DetaLearnedPositionEmbedding.forward)r)Nrr.r.r`r/rsrcCsJ|jd}|jdkr"t|dd}n$|jdkr6t|}ntd|j|S)NrZsineT)rZlearnedzNot supported )d_modelZposition_embedding_typerrr)r|Zn_stepsr{r.r.r/rzs    rz)valuevalue_spatial_shapessampling_locationsattention_weightsreturncCs,|j\}}}}|j\}}}} } }|jdd|Ddd} d|d} g} t|D]~\}\}}| |ddd|||||}| dddddd|fdddd}tjj||ddd d }| |qP|dd||d|| | }t j | d dd | d  ||||}|ddS) NcSs g|]\}}||qSr.)item)rArrr.r.r/rEsz4multi_scale_deformable_attention..rrrrZbilinearr]F)modeZ padding_modeZ align_cornersrre)rsplit enumerater transposerfr rZ grid_samplerr+rsumview contiguous)rrrrr_ num_heads hidden_dim num_queriesZ num_levelsZ num_pointsZ value_listZsampling_gridsZsampling_value_listZlevel_idrrZvalue_l_Zsampling_grid_l_Zsampling_value_l_outputr.r.r/ multi_scale_deformable_attentions@ $,  rcsneZdZdZeeeedfdd ZddZeje eddd Z deje eje eje d d dZ Z S)!DetaMultiscaleDeformableAttentionzI Multiscale deformable attention as proposed in Deformable DETR.  embed_dimrn_levelsn_pointscst||dkr*td|d|||}||d@dkrJ|dksTtdd|_||_||_||_||_ t ||||d|_ t |||||_ t |||_t |||_|dS)Nrz