Sj$ddlmZmZddlmZddlZddlZddlmZddl m Z ddgZ da ejZ dd lmZd Zer3dd Zdd Z ddej*j,ezdej.ezdedej2j4dzdedzf dZ ddej*j,ezdej.ezdededzdedzdej2j4dzdej*j,ezdzfdZ ddej*j,ezdej.ezdedej2j4dzdej*j,ezdzf dZyy#e$rdZdZed YKwxYw))Anycast)CallableN)CompileCounterWithBackend)Timer bench_allbenchmark_compileF)tabulateTzJtabulate is not installed, please pip install tabulate to use this utilitycPtjjrtjjjj durWtjj dk\r5tjdtstdtddayyyyy)NF)rhighzYour GPU supports tensor coreszWwe will enable it automatically by setting `torch.set_float32_matmul_precision('high')`T) torchcuda is_availablebackendsmatmul allow_tf32get_device_capabilityset_float32_matmul_precision_warned_tensor_coresprintb/var/www/html/ia-enhanced/venv/lib/python3.12/site-packages/torch/utils/benchmark/utils/compile.py_enable_tensor_coresrs :: " " $~~""))44=%**BbBbBdhnBn226:+:;st+/(,Co= %rc6tjty)N)rr_default_float_32_precisionrrr_disable_tensor_coresr's **+FGrmodel sample_input num_iters optimizerloss_fnc|r|rd}nd}t|||||d}|j|}|jdz}t|dS)Nz output = model(sample_input) loss = loss_fn(output) if loss_fn else output.sum() loss.backward() optimizer.step() optimizer.zero_grad() zmodel(sample_input))rr r"r#)stmtglobals)numberi)rtimeitmeanround) rr r!r"r#r%timerresultavg_times r bench_loopr/*sc D)D#\PYfmn  Y/;;%Xq!!rbackendmodec|r tjjt|}tj|||}t ||d||} t |||||} |j dk(r td|j dkDr tdn|}d} t |||||} | r t| d nd} | r t| d nd} | | fS#t$r&} t| td|d|Yd} ~ y d} ~ wwxYw) z= Use this utility to benchmark torch.compile )r0r1rz,No compilation occurred during benchmarking.z+Recompilation occurred during benchmarking.zFailed to compile z with mode N)NNr() r_dynamoresetrcompiler/ frame_count RuntimeError Exceptionrr+) rr r!r0r1r"r#compile_counter_with_backend opt_modelcompilation_time running_timees rr r Ls  " ##%/H/Q,!MM%9U\`a $.iq)U\#] ))\9iY`a /;;q@&'UVV/;;a?&'TUU@I# %iy)U\]L9I5!115t1=u\1-4  -- "a*7);tfEF! "sBC C>C99C>c gd}g}d}tjjt|||dd|\}}|j |rdndddd|dgtjj D]w} | dk(rt ttdzttjjjdgz} | D]} | d k(r tjj tjjr tt|||| | ||\} } tjjst|j |rdnd| r| nd| | nd| r| d nd| r| d ndgtjjt|||| d||\} } | T|j |rdnd| d| d xsd| d gzt!||d S#tjjr;t|j |rdnd| r| nd| | nd r| d nd r| d ndgwwxYw) a This is a simple utility that can be used to benchmark torch.compile In particular it ensures that your GPU is setup to use tensor cores if it supports its It also tries out all the main backends and prints a table of results so you can easily compare them all Many of the backendds have their own optional dependencies so please pip install them separately You will get one table for inference and another for training If you'd like to leverage this utility for training make sure to pass in a torch.optim.Optimizer The important warnings are Your GPU supports tensor cores we will enable it automatically by setting `torch.set_float32_matmul_precision('high')` If a compilation fails for any reason including the dependency not being included then we will print Failed to compile {backend} with mode {mode} )zTrain/InferenceBackendModezCompilation TimezAverage Running TimeNTraining InferenceEager-z msinductordefaultz ms github)headerstablefmt)rr4r5r append list_backendsrliststr _inductorlist_mode_optionskeysrrrrr )rr r!r"r# field_namestable eager_time_r0 mode_optionsr1r<r=s rrrysu.i   )%y$PTV_` : %j;#szlZ]L^ _ }}224$ G*$#Dt$4d5??;\;\;^;c;c;e6fgkojpp (Dy( MM'') ::224029J!<GT9V]:_6(,!::22413!LL/8k+2(,(8c=M#3"4D 9SV9E<. 53 *, ##%1B<GT9g2W. , +LL'0k+,D18S'.- "?$ P{XFF5!::22413!LL/8k+2(,(8c=M#3"4D 9SV9E<. 53 *5s :>AI)returnN)NN)rXNrGNN)typingrrcollections.abcrr torch._dynamotorch._dynamo.testingrtorch.utils.benchmarkr__all__rget_float32_matmul_precisionrr HAS_TABULATEModuleNotFoundErrorrrrnnModuleTensorintoptim Optimizerr/rNr rrrrrhs$ ;' + ,@e@@BX!L  0H 26#' "xx) "llS( " ";;((4/ " D "J"$266:*.xx)*.llS(*.*.t *. Dj *. ;;((4/ *.((//H,t3*.`266: JG(*JGllS(JGJG;;((4/ JG ((//H,t3 JG} XLH VWXsE::FF