MjUddlZddlmcmZddlmZddlmZgZe e e d<ejjGddZy)N)Tensor)2_scripted_functional_optimizer_deprecation_warning__all__ceZdZ ddeedededededededed ed ed ed ed efdZdeedzfdZy)_FunctionalAdagradparamslrlr_decay weight_decayinitial_accumulator_valuewarmup_lr_multiplierwarmup_num_iterseps coalesce_gradforeachfusedmaximize_allow_empty_param_listctd|||||||d|_| |_| |_| |_| |_t jjtt jttt jffi|_ t|dk(r | s tdd|i|_|jdD]E}t j |j"|t j$dd|j|<Gy) N) stacklevel)r r rr r r rrz%optimizer got an empty parameter listr)sumstep)rdefaultsrrrrtorchjitannotatedictrstrstatelen ValueError param_group full_likedatatensor)selfrr r r r r rrrrrrrps i/var/www/html/ia-enhanced/venv/lib/python3.12/site-packages/torch/distributed/optim/functional_adagrad.py__init__z_FunctionalAdagrad.__init__s ;aH ()B$8 0  +    YY''U\\4U\\@Q;R-R(SUWX v;! $;DE E%f-!!(+ Aqvv/HI S)DJJqM  gradientsNcD|jd}g}g}g}g}t|t|k7r*tddt|dzdt|zd\}}t|jd|D]\} } |  || jz}|t j | z}|j| |j| |j| } |j| d|j| dt j5tj|||||jd |jd |jd |jd ||j|j||jdd dddy#1swYyxYw)NrzEthe gradients passed in does not equal to the size of the parameters!zParams length: z. zGradients length: )FFrrr r r r) r r r rhas_sparse_gradrr has_complexr grad_scale found_inf)r$r"r#zip is_sparser is_complexappendr!no_gradFadagradrrrr) r(r-rparams_with_gradgrads state_sums state_stepsr/r0paramgradientr!s r*rz_FunctionalAdagrad.stepHs!!(+ $& v;#i. (W#CK=34&s9~&678  (4$"4#3#3H#=yI 2OE8#8#5#55u//66  ''. X& 5)!!%,/""5=1 2]]_  II==&!]]>:z2MM%( / 'jj    s A7FF) g{Gz?rrrg?rg|=TFFFF) __name__ __module__ __qualname__listrfloatboolr+rr,r*rrs !+.&)"%"(--V - - -  - $) -$- ------"&-^*d6D=1*r,r)rtorch.optim._functionaloptim _functionalr8r,torch.distributed.optim._deprecation_warningrrrCr __annotations__rscriptrrFr,r*rMsN ## cZZZr,