Zj6&Z dZddlmZddlZddlmZddlmZddlm Z m Z m Z ddl m Z e d Ze d Zeeej"d fZeedzej&d fZeedzej&d fZej-dddg dUdede ddfdZej-ddgddddddddddedeeej&d fdeeej&d fdeeej&d fd!eeej&d" fd#eeej&d$ fd%eeej&d& fd'eeej&d( fded)eedzej&d* fde d+eeej&d, fd-eeej&d. fd/eedzej&d0 fd1eedzej&d2 fd3eedzej&d4 fddf"d5Zej-dd6g dVdeeej&d fdedeedzej&d7 fdede ddf d8Z dWde ddfd9Zej-dd:geej-d;d<=eej;ed>?ej-d@g dUdedede ddfdAZej-dBg dXdededeedzej&d fdeedzej&d fd!eedzej&d" fd#eedzej&d$ fdeedzej&dC fdDeedzej&dE fd)eedzej&d* fd+eedzej&d, fd-eedzej&d. fd/eedzej&d0 fd1eedzej&d2 fd3eedzej&d4 fde ddf dFZej-dGg dYdededHee ej&dIdJ fde ddf dKZ!ej-dLg dUdedede ddfdMZ"ej-dNg dYdededOee ej&dPdQ fde ddf dRZ#ej-dSg dUdedede ddfdTZ$y)Zz2CLI commands for Hugging Face Inference Endpoints.) AnnotatedN)InferenceEndpointScalingMetric)HfHubHTTPError)TokenOpt get_hf_api typer_factory)outz(Manage Hugging Face Inference Endpoints.)helpz.Interact with the Inference Endpoints catalog.zEndpoint name.z_The namespace associated with the Inference Endpoint. Defaults to the current user's namespace.z list | lszhf endpoints lsz"hf endpoints ls --namespace my-org)examples namespacetokenreturnct|} |j||}g}|D]o}|j}|jdi}|jdi} |jdi} |jd i} |j|jd d t| tr| jd d nd t|tr|jd d nd t| tr| jdd nd t| tr| jdd nd t| tr| jdd nd t| tr| jdd nd t| tr| jdd nd drtj|d y#t$rH}tjd|t j |jj|d}~wwxYw)z6Lists all Inference Endpoints for the given namespace.r)r rzListing failed: codeNstatusmodelcomputeprovidername repositorystatetask framework instanceTypevendorregion)rrrrrinstancerr )id_key)rlist_inference_endpointsrr errortyperExitresponse status_coderawgetappend isinstancedicttable) r rapi endpointsr$resultsendpointr)rrrrs f/var/www/html/ia-enhanced/venv/lib/python3.12/site-packages/huggingface_hub/cli/inference_endpoints.pylsr4$s 5 !CE009E0R G ll2&$'')R(77:r*+8B5$8O<4UW5?5M&**Wb1SU1;E41H &"-b;EeT;RUYY{B7XZ?I'SW?XGKK;^`8B8T8R(,,x4XZ8B8T8R(,,x4XZ   $IIgf%/ E $UG,-jjenn889uDEsF00 H9AG<<HdeployzChf endpoints deploy my-endpoint --repo gpt2 --framework pytorch ...)rr )r rr min_replica max_replicascale_to_zero_timeoutscaling_metricscaling_thresholdrrepozeThe name of the model repository associated with the Inference Endpoint (e.g. 'openai/gpt-oss-120b').rz@The machine learning framework used for the model (e.g. 'vllm'). acceleratorz?The hardware accelerator to be used for inference (e.g. 'cpu'). instance_sizezNThe size or type of the instance to be used for hosting the model (e.g. 'x4'). instance_typezYThe cloud instance type where the Inference Endpoint will be deployed (e.g. 'intel-icl').r zTThe cloud region in which the Inference Endpoint will be created (e.g. 'us-east-1').rzVThe cloud provider or vendor where the Inference Endpoint will be hosted (e.g. 'aws').rzCThe task on which to deploy the model (e.g. 'text-classification').r6zVThe minimum number of replicas (instances) to keep running for the Inference Endpoint.r7zRThe maximum number of replicas (instances) to scale to for the Inference Endpoint.r8zFThe duration in minutes before an inactive endpoint is scaled to zero.r9z!The metric reference for scaling.r:zeThe scaling metric threshold used to trigger a scale up. Ignored when scaling metric is not provided.ct| }|j|||||||||| | | | ||| }tj|jy)z3Deploy an Inference Endpoint from a Hub repository.r)rrrr<r=r>r rr rrr6r7r9r:r8N)rcreate_inference_endpointr r-r))rr;rr<r=r>r rr rrr6r7r8r9r:r/r2s r3r5r5Gsfl 5 !C,, ## %+3!-H$HHX\\zChf endpoints catalog deploy --repo meta-llama/Llama-3.2-1B-InstructzPThe hardware accelerator to be used for inference (e.g. 'cpu', 'gpu', 'neuron').c0t|} |j|||||}tj|jy#t$rH}tjd|t j |jj|d}~wwxYw)z4Deploy an Inference Endpoint from the Model Catalog.r)repo_idrr<r rzDeployment failed: rN) r&create_inference_endpoint_from_catalogrr r$r%r&r'r(r-r))r;rr<r rr/r2r$s r3deploy_from_catalogrEs& 5 !C E==# > HHX\\ E 'w/0jjenn889uDEA B ABBct|} |j|}tjd|iy#t$rH}tjd|t j |jj|d}~wwxYw)zList available Catalog models.rzCatalog fetch failed: rNmodels) rlist_inference_catalogrr r$r%r&r'r(r-)rr/rHr$s r3 list_catalogrJs} 5 !CE++%+8 HHh  E *5'23jjenn889uDEs8 B ABB zhf endpoints catalog lsz list-catalogT)rhiddencatalogrz!hf endpoints describe my-endpointc,t|} |j|||}tj|jy#t$rH}tjd|t j |jj|d}~wwxYw)z+Get information about an existing endpoint.rrr rzFetch failed: rN) rget_inference_endpointrr r$r%r&r'r(r-r)rr rr/r2r$s r3describerRs 5 !CE--49TY-Z HHX\\ E N5'*+jjenn889uDEA B ABBz/hf endpoints update my-endpoint --min-replica 2zBThe machine learning framework used for the model (e.g. 'custom').revisionzrThe specific model revision to deploy on the Inference Endpoint (e.g. '6c0e6080953db56375760c0471a8c5f2929baf11').cDt|} |j|||||||||| | | | | |}tj|jy#t$rH}tjd|t j |jj|d}~wwxYw)zUpdate an existing endpoint.r)rr rrrTrr<r=r>r6r7r8r9r:rzUpdate failed: rN) rupdate_inference_endpointrr r$r%r&r'r(r-r))rr r;r<r=r>rrTrr6r7r8r9r:rr/r2r$s r3updaterWs^ 5 !CE00#''##"7)/1 (HHX\\ E OE7+,jjenn889uDEs A BABBzhf endpoints delete my-endpointyesz--yeszSkip confirmation prompts.cZtjd|d|t|} |j|||tjd |d | y#t$rH}tj d|t j|jj|d}~wwxYw) z)Delete an Inference Endpoint permanently.zDelete endpoint 'z'?)rXrrOzDelete failed: rNz Deleted 'z'.rM) r confirmrdelete_inference_endpointrr$r%r&r'r(result)rr rXrr/r$s r3deleter]dsKK#D6,#6 5 !CE %%49E%R JJ4&#$/ E OE7+,jjenn889uDEsA B*"AB%%B*zhf endpoints pause my-endpointc,t|} |j|||}tj|jy#t$rH}tjd|t j |jj|d}~wwxYw)zPause an Inference Endpoint.rrOzPause failed: rN) rpause_inference_endpointrr r$r%r&r'r(r-r)rQs r3pauser`{s 5 !CE//TYV[/\ HHX\\ E N5'*+jjenn889uDErSzhf endpoints resume my-endpointfail_if_already_runningz--fail-if-already-runningzWIf `True`, the method will raise an error if the Inference Endpoint is already running.c0t|} |j|||| }tj|jy#t$rH}tjd|t j |jj|d}~wwxYw)zResume an Inference Endpoint.r)rr r running_okzResume failed: rN) rresume_inference_endpointrr r$r%r&r'r(r-r))rr rarr/r2r$s r3resumeres 5 !C E0022 1 HHX\\ E OE7+,jjenn889uDErFz&hf endpoints scale-to-zero my-endpointc,t|} |j|||}tj|jy#t$rH}tjd|t j |jj|d}~wwxYw)z$Scale an Inference Endpoint to zero.rrOzScale To Zero failed: rN) r scale_to_zero_inference_endpointrr r$r%r&r'r(r-r)rQs r3 scale_to_zerorhs 5 !CE77TY^c7d HHX\\ E *5'23jjenn889uDErS)NN)NNNN)N)NNNNNNNNNNNNNN)NFN)%__doc__typingrr%$huggingface_hub._inference_endpointsrhuggingface_hub.errorsr _cli_utilsrrr _outputr ie_cli catalog_appstrArgumentNameArgOptionNameOpt NamespaceOptcommandr4intfloatr5rErJ add_typerrRrWboolr]r`rerhrAr3r}ss 8 O1;; F G!QR  ENN()+  $JELL&')  $JELL n  '8:^&_`"&& & &a&DX)n(op\#  eh h   x   h  S  h  R  h(  a  )h4  l  5h@   g   AhL   i   MhZ[h\  d  V   ]hh ihj  i  khv  e  whB% d  Y  ChN&- 4  OhZ!   x  [hf ghqhV (.s-tu "   x     d  c     !vF !  !  !L 0I/JKLY0N40>9-=>?#       @  KLM# Wd dd  d  x   d d  R  d d  a  d* d  l  +d6 d  U  7dB d  F  CdN  d  V   OdZ d  i  [df d  e  gdr% d  Y  sd~&- 4  dJ!   x  KdV WdX YdNdN;<=# 0 00   W#?@ B 0 0 0>0,:;<#       =  ;<=#  '  'j    >6BCD#       E rA