Sj <ddlZddlmZddlmZmZddlZddlmZddl m Z ddl m Z ddgZ d ejeezeed fzeeefzd ejdzfd Z dd ee deedeeeefdzdeeej*zdzdzd eef dZy)N)Sequence)Anycast)ExceptionWrapper)_get_device_index)Module get_a_varparallel_applyobj.returncpt|tjr|St|ttfr3t t |D] }t|tjs|cSt|trAt t |jD] }t|tjs|cSy)N) isinstancetorchTensorlisttuplemapr dictitems)r results _/var/www/html/ia-enhanced/venv/lib/python3.12/site-packages/torch/nn/parallel/parallel_apply.pyr r s#u||$ #e}%)S) F&%,,/  #t)SYY[1 F&%,,/   modulesinputs kwargs_tupdevicescxt|t|k7r#tdt|dt||:t|t|k7rMtdt|dt|tttt fift|z}|:t|t|k7r2tdt|dt|dgt|z}|Dcgc]}t |d}}|Dcgc]!}tjj|#}}tjjs tdtjjjtjitjtj c ddt"d t$d t d ttt fd t"tj&zdzd tj(dzddffd }t|dkDrt+t-|||||dD cgc](\}\}} } } } tj.|||| | | | f*} } } } }}} | D]}|j1| D]}|j3n|d|d|d|d|d|dg}t5t|D]8}|}t7|t8r|j;|j=|:|Scc}wcc}wcc} } } } }}w)aApply each `module` in :attr:`modules` in parallel on each of :attr:`devices`. Args: modules (Module): modules to be parallelized inputs (tensor): inputs to the modules devices (list of int or torch.device): CUDA devices :attr:`modules`, :attr:`inputs`, :attr:`kwargs_tup` (if given), and :attr:`devices` (if given) should all have same length. Moreover, each element of :attr:`inputs` can either be a single object as the only argument to a module, or a collection of positional arguments. zThe number of modules z& is not equal to the number of inputs Nz* is not equal to the number of kwargs_tup z' is not equal to the number of devices TzNo available accelerator found.imoduleinputkwargsdevicestreamr cHtj |xYw#1swYyxYw#t$r0 5td|d| |<dddYy#1swYYyxYwwxYw)Nz in replica zQ, no device was provided and no tensor input was found; device cannot be resolved)where)enabledz on device )rset_grad_enabledr r get_devicerr"index acceleratorcurrent_stream device_indexampautocastrr Exception) rrr r!r"r#toutputautocast_enabled device_type grad_enabledlockresultss r_workerzparallel_apply.._workerRs |, >% Ay!1 +A3/44"GAJ \\^F fell +\\F >&&55f=F !!..v6 2 2 "";8H"I 2 "%$7"HE1&1 2 2 2 $#  $ $+  2 2 2 2 2 2 $ $  -'s+fX>     sD,E(<E?"E!"D8E E E(E#E(,D58E =EE EEE(E%!E(%E(( F!3F F!F F!F!)strict)targetargsrNN)lenAssertionErrorrrstrrrrr*r+ is_availablecurrent_acceleratortype threadingLockis_grad_enabledis_autocast_enabledintrr"Stream enumeratezipThreadstartjoinrangerrreraiseappend)rrrrxstreamsr7rrr r!r"r#threadsthreadoutputsr1r2r3r4r5r6s @@@@@rr r sF$ 7|s6{"$S\N3$$'K= 2   w<3z? * (W7,,/ O+<>  4S>2.03w<?  w<3w< ' (W7)),W8  &3w<'3:;a D);G;??##779>>K >> DG  !!##L"-1&* ' '''S#X ' ell"T) '  t# ' ''R 7|a ?HGVZ'$O?   ;:FE666   a%O    F LLN  F KKM  71:vay*Q-WQZPG 3v;  f. / NN v  NY<Dj s5L(&L--L2r<)rCcollections.abcrtypingrrr torch._utilsrtorch.cuda._utilsrtorch.nn.modulesr__all__rrrrr r?rGr"r rrr]s$ )/# ( ) S !E#s(O 3d38n D \\D(37:> s f s SMsc3h(4/scELL(4/ 04 7 s  #Y sr