Sj/J UdZddlZddlZddlmZddlmZddlZddlmZddl m Z m Z ddl m Z ddlmZd d lmZej$Zej&Zd Zeej,j.d fed <dagaeeegdfeefed<ej:Zej>Z e a!e"ej,drej,jFZ$nde%de%fdZ$e"ej,drej,jLZ'nde%de%fdZ'dZ(dZ)dZ*de+fdZ,dFdZ-Gdde.Z/de+fdZ0de+fdZ1dGdeddfdZ2de%fd Z3de%fd!Z4dGdedefd"Z5dGdedefd#Z6 dHd$edzd%ed&e%ddfd'Z7de8eeffd(Z9d)ee%e%e%e%gdfddfd*Z:dId+e+fd,Z;dGdedee%e%ffd-Zdeddfd1Z?dGdede8eeffd2Z@Gd3dZAGd4d5ZBd6ZCd/edzdeBfd7ZDdJdedefd8ZEdeefd9ZFdJd:ededdfd;ZGde%ddfd?ZId>e%ddfd@ZJde%fdAZKde%fdBZLde%fdCZMd dDlNd dDlOgdEZPy)KzH This package enables an interface for accessing MTIA backend in python N)Callable)Any)Tensor) is_fbcodeis_prod)_LazySeedTracker)Device)_get_device_index.default_generatorsF _queued_calls_mtia_exchangeDevicedevicereturnc$|dkrytdNrz)PyTorch was compiled without MTIA support RuntimeErrorrs R/var/www/html/ia-enhanced/venv/lib/python3.12/site-packages/torch/mtia/__init__.py_exchange_devicer) A:FGG_mtia_maybeExchangeDevicec$|dkrytdrrrs r_maybe_exchange_devicer3rrctyN) _lazy_initr rrinitr"9sLrc(txr t S)z9Return whether PyTorch's MTIA state has been initialized.) _initialized_is_in_bad_forkr rris_initializedr&=s  1 111rc t5tr|cdddS|jddr)tj |t j nd|jddr)tj|t j n)tj|t j fdddy#1swYyxYw)Nseed_allFseed) _initialization_lockr&get_lazy_seed_trackerqueue_seed_all traceback format_stack queue_seedrappend)callablekwargss r _lazy_callr4Bs  K  : K K zz*e,"11(ItjjSr )torch_C_mtia_isInBadForkr rrr%r%Ps 88 % % ''rc tsttdryt5tr dddyt r t dt s tdtr!tr ddl m }|jtjj!dt_t$j'dt(j+D t$D] \}} | t5tddadddy#t$rYwxYw#t,$r1}dt/|d d j1|}t3||d}~wwxYw#t5tdwxYw#1swYyxYw) Nis_initializingzwCannot re-initialize MTIA in forked subprocess. To use MTIA with multiprocessing, you must use the 'spawn' start methodzTorch not compiled with MTIA enabled. Ensure you have `import mtia.host_runtime.torch_mtia.dynamic_library` in your python src file and include `//mtia/host_runtime/torch_mtia:torch_mtia` as your target dependency!r)cxx_resource_managerTc3&K|] }|s| ywr r ).0callss r z_lazy_init..sXuRWUXsz6MTIA call failed lazily at initialization with error: z( MTIA call was originally invoked at: )r&hasattr_tlsr*r%r _is_compiledAssertionErrorrrlibfb.py.cxx_resourcesr;installModuleNotFoundErrorr6r7 _mtia_initr:rextendr, get_calls ExceptionstrjoinDeferredMtiaCallErrordelattrr$)r; queued_callorig_tracebackemsgs rr!r!Ts}74):; 7   77  I ~ *  ;79 G$,,.  $X0B0L0L0NXX -/< <+ ^<M < D+ , o77>'   !<PQTUVQWPXYCCE77>CZB[]04!; < D+ ,m77sq E7>E76D AE7$ E"1D%8E";E7 D"E7!D""E7% E.,EEE""E44E77Fc eZdZy)rNN)__name__ __module__ __qualname__r rrrNrNsrrNc>tjjS)z*Return true if compiled with MTIA support.)r6r7 _mtia_isBuiltr rrrCrCs 88 ! ! ##rc2tsytdkDS)z'Return true if MTIA device is availableFr)rC device_countr rr is_availabler\s > >A rctjj|5tjj cdddS#1swYyxYw)z?Waits for all jobs in all streams on a MTIA device to complete.N)r6mtiarr7_mtia_deviceSynchronizers r synchronizer`s9   6 "2xx//1222s AAc>tjjS)z,Return the number of MTIA devices available.)r6r7_mtia_getDeviceCountr rrr[r[s 88 ( ( **rc>tjjS)z0Return the index of a currently selected device.)r6r7_mtia_getDevicer rrcurrent_deviceres 88 # # %%rcZt|d}tjj|S)aSReturn the currently selected :class:`Stream` for a given device. Args: device (torch.device or int, optional): selected device. Returns the currently selected :class:`Stream` for the current device, given by :func:`~torch.mtia.current_device`, if :attr:`device` is ``None`` (default). Toptional)r r6r7_mtia_getCurrentStream)r device_idxs rcurrent_streamrks%#6D9J 88 * *: 66rcVtjjt|dS)a=Return the default :class:`Stream` for a given device. Args: device (torch.device or int, optional): selected device. Returns the default :class:`Stream` for the current device, given by :func:`~torch.mtia.current_device`, if :attr:`device` is ``None`` (default). Trg)r6r7_mtia_getDefaultStreamr rs rdefault_streamrns! 88 * *+tjjS)z4Return a dictionary of MTIA memory allocator history)r6r7_mtia_memorySnapshotr rrsnapshotrws 88 ( ( **robservercBtjj|y)z9Attach an out-of-memory observer to MTIA memory allocatorN)r6r7_mtia_attachOutOfMemoryObserver)rxs rattach_out_of_memory_observerr{s HH,,X6rincluding_emulationcy)zLReturn a bool indicating if the current MTIA device supports dtype bfloat16.Tr )r|s ris_bf16_supportedr~s rcVtjjt|dS)aReturn capability of a given device as a tuple of (major version, minor version). Args: device (torch.device or int, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). Trg)r6r7_mtia_getDeviceCapabilityr rs rget_device_capabilityr" 88 - -.?QU.V WWrc>tjjS)zEmpty the MTIA device cache.)r6r7_mtia_emptyCacher rr empty_cachers 88 $ $ &&rstreamcH|ytjj|y)aSet the current stream. This is a wrapper API to set the stream. Usage of this function is discouraged in favor of the ``stream`` context manager. Args: stream (Stream): selected stream. This function is a no-op if this argument is ``None``. N)r6r7_mtia_setCurrentStreamrs r set_streamrs~ HH##F+rcdt|}|dk\r tjj|yy)zSet the current device. Args: device (torch.device or int): selected device. This function is a no-op if this argument is negative. rN)r r6r7_mtia_setDevicers r set_devicers,v &F {   (rcVtjjt|dS)zReturn a dictionary of MTIA device properties Args: device (torch.device or int, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). Trg)r6r7_mtia_getDevicePropertiesr rs rget_device_propertiesrrrc6eZdZdZdefdZdZdededefdZy) rzContext-manager that changes the selected device. Args: device (torch.device or int): device index to select. It's a no-op if this argument is a negative integer or ``None``. c6t|d|_d|_y)NTrgr)r idxprev_idx)selfrs r__init__zdevice.__init__0s$Vd; rc`tjj|j|_yr )r6r7rrr)rs r __enter__zdevice.__enter__4s::488D rtypevaluer.c`tjj|j|_y)NF)r6r7rrr)rrrr.s r__exit__zdevice.__exit__8s8855dmmDrN)rUrVrW__doc__rrrrr rrrr(s3sESrcNeZdZUdZedzed<dedzfdZdZdeded efd Z y) StreamContextaContext-manager that selects a given stream. All MTIA kernels queued within its context will be enqueued on a selected stream. Args: Stream (Stream): selected stream. This manager is a no-op if it's ``None``. .. note:: Streams are per-device. N cur_streamrcd|_||_tdd|_tj j s|jd|_tj j sdntjjd|_ tj j sd|_ ytjjd|_ y)NTr) rrr rr6jit is_scriptingr^rnsrc_prev_streamdst_prev_stream)rrs rrzStreamContext.__init__Ls $T40yy%%'xx ..0Dejj6O6OPT6U  ..0D 6;jj6O6OPT6U rc|j}||jdk(rytjj d|_|j j |j k7rLt |j 5tjj |j |_dddtjj|y#1swY)xYwNr) rrr6r^rkrrrr)rrs rrzStreamContext.__enter__[s[[  R $zz88>    & &**;*; ; ))* T',zz'@'@ARAR'S$ T j) T Ts ;/CCrrr.c*|j}||jdk(ry|jj|jk7r)tj j |jtj j |jyr)rrrrr6r^rr)rrrr.rs rrzStreamContext.__exit__jsj[[  R     & &**;*; ; JJ ! !$"6"6 7 d223r) rUrVrWrStream__annotations__rrrrr rrrr>sB    v}   * 4S 4 4 4rrcFtjj|||y)zset stream specified by the stream id, device index and device type Args: stream_id (int): stream id in stream pool device_index (int): device index in topo device_type (int): enum device type N)r6r7_mtia_setStream) stream_id device_index device_types r_set_stream_by_idrxs HHY kBrct|S)a,Wrap around the Context-manager StreamContext that selects a given stream. Arguments: stream (Stream): selected stream. This manager is a no-op if it's ``None``. .. note:: In eager mode stream is of type Stream class while in JIT it doesn't support torch.mtia.stream )rrs rrrs   rcztt|d}| t}t|}|j S)a[Returns the random number generator state of the specified MTIA device as a ByteTensor. Args: device (torch.device or int, optional): The device to return the RNG state of. Default: ``'mtia'`` (i.e., ``torch.device('mtia')``, the current mtia device). .. warning:: This function eagerly initializes MTIA. Trg)r!r rer get_state)rrdefault_generators r get_rng_staters<L FT 2C {*3/  & & ((rcbttDcgc] }t|}}|Scc}w)zRReturns a list of ByteTensor representing the random number states of all devices.)ranger[r)iresultss rget_rng_state_allrs,).|~)>?A}Q?G? N@s, new_statectsGtjj5j tj dddt |d tfd}t|y#1swY4xYw)a8Sets the random number generator state of the specified MTIA device. Args: new_state (torch.ByteTensor): The desired state device (torch.device or int, optional): The device to set the RNG state. Default: ``'mtia'`` (i.e., ``torch.device('mtia')``, the current mtia device). ) memory_formatNTrgc:t}|jyr )r set_state)rrrs rcbzset_rng_state..cbs.s3##I.r) r&r6r7_DisableFuncTorchclonecontiguous_formatr rer4)rrrrs` @r set_rng_statersu   XX ' ' ) O"e6M6MNI O FT 2C {/rN O Os !A??B new_statescBt|D]\}}t||y)zSets the random number generator state of all devices. Args: new_states (Iterable of torch.ByteTensor): The desired state for each device. N) enumerater)rrstates rset_rng_state_allrs& j) 5eQ rr)c@tfd}t|dy)aSets the seed for generating random numbers for the current MTIA device. It's safe to call this function if MTIA is not available; in that case, it is silently ignored. Args: seed (int): The desired seed. .. warning:: If you are working with a multi-GPU model, this function is insufficient to get determinism. To seed all GPUs, use :func:`manual_seed_all`. cNt}t|}|jyr )rer manual_seed)rrr)s rrzmanual_seed..cbs$.s3%%d+rTr)Nintr4r)rs` rrrs t9D, rrc@tfd}t|dy)zSets the seed for generating random numbers on all MTIA devices. It's safe to call this function if MTIA is not available; in that case, it is silently ignored. Args: seed (int): The desired seed. cjttD]}t|}|jyr )rr[r r)rrr)s rrzmanual_seed_all..cbs2|~& 0A 21 5   ) )$ / 0rTr(Nrrs` rmanual_seed_allrs t9D0 rD!rc"d}t|dS)axSets the seed for generating random numbers to a random number for the current MTIA device. It's safe to call this function if MTIA is not available; in that case, it is silently ignored. .. warning:: If you are working with a multi-GPU model, this function will only initialize the seed on one GPU. To initialize all GPUs, use :func:`seed_all`. cHt}t|}|jSr )rer r))rrs rrzseed..cbs$.s3 %%''rTrr4rs rr)r)s( bt $$rc"d}t|dS)zSets the seed for generating random numbers to a random number on all MTIA devices. It's safe to call this function if MTIA is not available; in that case, it is silently ignored. cd}d}ttD]1}t|}|s|j}d}!|j |3|S)NrFT)rr[r r)r) random_seedseededrrs rrzseed_all..cbsY |~& ;A 21 5 /446 !--k:  ;rTrrrs rr(r(s  b4 ((rcXtt}t|jS)zReturns the current random seed of the current MTIA device. .. warning:: This function eagerly initializes MTIA. )r!rer initial_seed)rs rrrs% L  C c " / / 11r)*)#r"r\r&r`r[rerkrn memory_statsmax_memory_allocatedmemory_allocatedreset_peak_memory_statsrrrtrwr{rrrrrrrrrrrr)r(rr~ MTIAGraphgraphgraph_pool_handle)rNr )allpythonr)T)r^)Qr threadingr.collections.abcrtypingrr6rtorch._environmentrr torch._utilsr torch.typesr _utilsr Eventrr tupler7 Generatorrr$rlistrLlocalrBLockr*r,rArrrrrr"r&r4boolr%r!rKrNrCr\r`r[rerkrnrtdictrwr{r~rrrrrrrrrrrrrrrr)r(rmemory mtia_graph__all__r rrrs$ 1)%   68E%((,,c127 t (2t8 d3i '(y%y~~'%' 588+,xx44HHH  58801"XX??HsHsH 2 K((;| I $d$ d22$2 +c+ && 76 7V 7 U6 UV UMNE 4ZE),EFIE E&+$sCx.+ 7S#s+T127 74 X&XE#s(OX' ,v , )v )$ )X&XDcNX,7474tC!6D=!]!)&)f)$4< VV2 $v, 4 cd*"#"$""%c%")#)*2c2$ r