Zj0e$ dZddlZddlZddlmZddlZddlmZddlm Z m Z m Z m Z m Z ddlmZmZmZddlmZd d lmZmZmZmZd d lmZmZd d lmZmZej>e Z!ed Z"e"jGdgd ddee$ejJdfdee&ejNddfdeedzejNddfdee&ejNddfdeddf dZ(d e$de&fd!Z)e"jGd"gd# dd ee$dzejJd$fd%ee&ejNd&d'd(fd)ee&ejNd*d+fd,ee&ejNd-d.d/fd0ededdfd1Z*d2e$dzd0e$dzd%e&d)e&d,e&de$dzddfd3Z+d e$d%e&d)e&d,e&de$dzddf d4Z,e"jGd5d6d7g ddee$ejJd8fdeddfd9Z-e"jGd:gd; ddee$ejJd8fdd?fd@ee&ejNdAdBfdeddf dCZ.e"jGdDgdE dd ee$ejJdFfd,ee&ejNd-d.dGfdd?fdHee&ejNdIdJfdKee/e$dzejNdLfdMee/e$dzejNdNfdeddfdOZ0e"jGdPgdQ ddRee$ejJdSfdTee$ejJdUfdeddfdVZ1e"jGdWgdX ddYee$dzejJdZfd[ee$dzejJd\fd:ee&ejNd]fd^ee&ejNd_d`fdaee&ejNdbdcfddee$dzejNdefdfee$dzejNdgfdHee&ejNdIdhfdKee/e$dzejNdifdMee/e$dzejNdjfdkee$dzejNdlfdmee&ejNdndofdpee&ejNdqdrfdsee&ejNdtdudvfdeddf dwZ2e"jGdxgdy ddzee$ejJd{fd|ee$dzejJd}fdeddfd~Z3y)z7Contains commands to interact with buckets via the CLI.N) Annotated)logging) BUCKET_PREFIX BucketFile FilterMatcher_is_bucket_path_parse_bucket_uri)SoftTemporaryDirectorydisable_progress_bars is_hf_uri) REPO_REGIONS) SearchOptTokenOpt get_hf_api typer_factory) format_sizeprint_file_listing)OutputFormatWithAutooutz"Commands to interact with buckets.)helpcreate)zhf buckets create my-bucketz hf buckets create user/my-bucketz-hf buckets create hf://buckets/user/my-bucketz*hf buckets create user/my-bucket --privatez+hf buckets create user/my-bucket --exist-okz,hf buckets create user/my-bucket --region us)nameexamples bucket_idzTBucket ID: bucket_name, namespace/bucket_name, or hf://buckets/namespace/bucket_nameprivatez --privatezCreate a private bucket.regionz--regionzdCloud region in which to create the bucket. Can be one of 'us' or 'eu'. Requires Team plan or above.exist_okz --exist-okz3Do not raise an error if the bucket already exists.tokenreturncrt|}|jtrCt|}|jr t j d|dtd|j}|j||r|nd||}tjd|jj|jy) zCreate a new bucket.r z-Cannot specify a prefix for bucket creation: . Use namespace/bucket_name or namespace/bucket_name.N)rrrzBucket created)uriurl)r startswithrr path_in_repotyper BadParameterid create_bucketrresultr&to_urir')rrrrr apiparsed bucket_urls Z/var/www/html/ia-enhanced/venv/lib/python3.12/site-packages/huggingface_hub/cli/buckets.pyrr1sT 5 !CM*"9-   $$? {K11>?UW II """ #J JJZ^^%:%:%<*..Qargumentcd|jtr|ttd}d|vS|}d|vS)zFCheck if argument is a bucket ID (namespace/name) vs just a namespace.N/)r(rlen)r5paths r3 _is_bucket_idr:os?=)M*,- $; $;r4z list | ls) zhf buckets listzhf buckets list huggingfacez$hf buckets list --search "my-prefix"zhf buckets list user/my-bucketz!hf buckets list user/my-bucket -Rz!hf buckets list user/my-bucket -hz%hf buckets list user/my-bucket --treez(hf buckets list user/my-bucket --tree -hz+hf buckets list hf://buckets/user/my-bucketz%hf buckets list user/my-bucket/sub -RzyNamespace (user or org) to list buckets, or bucket ID (namespace/bucket_name(/prefix) or hf://buckets/...) to list files.human_readablez--human-readablez-hz$Show sizes in human readable format.as_treez--treez3List files in tree format (only for listing files). recursivez --recursivez-Rz0List files recursively (only for listing files).searchc|duxr t|}|r(|tjdt|||||yt ||||||y)zList buckets or files in a bucket. When called with no argument or a namespace, lists buckets. When called with a bucket ID (namespace/bucket_name), lists files in the bucket. Nz'Cannot use --search when listing files.)r5r;r<r=r ) namespacer>r;r<r=r )r:r*r+ _list_files _list_buckets)r5r;r<r=r>r is_file_modes r3list_cmdrDxsgr4'CM(,CL  $$%NO O)   )  r4r@c  |rtjd|rtjd|8|jtr#|t td}|j d}t |}|j||Dcgc]T}|j|j|rt|j|n |j|j|jdV}}tj|ddd  ycc}w) zList buckets in a namespace.z'Cannot use --tree when listing buckets.z,Cannot use --recursive when listing buckets.Nr7r#)r@r>)r,rsize total_files created_atright)rFrG) alignments)r*r+r(rr8rstripr list_bucketsr,rrrFrGrHrtable) r@r>r;r<r=r r0bucketitemss r3rBrBs  !JKK  !OPP!5!5m!Dc-023 $$S) 5 !C&&6&J   ))~~@NK ^?UW II I   !)-;;H/I_ a  KK:9+RHcR 5 !CiJ7JJ95r4z remove | rm)z)hf buckets remove user/my-bucket/file.txtz2hf buckets rm hf://buckets/user/my-bucket/file.txtz.hf buckets rm user/my-bucket/logs/ --recursivez:hf buckets rm user/my-bucket --recursive --include "*.tmp"z8hf buckets rm user/my-bucket/data/ --recursive --dry-runzBucket path: namespace/bucket_name/path or hf://buckets/namespace/bucket_name/path. With --recursive, namespace/bucket_name is also accepted to target all files.z0Remove files recursively under the given prefix.dry_runz --dry-runz8Preview what would be deleted without actually deleting.includezQInclude only files matching pattern (can specify multiple). Requires --recursive.excludezLExclude files matching pattern (can specify multiple). Requires --recursive.c:t|}|j|j}|dk(r|stjddd|s|r|stjdt |} |rt jd} g} | j|xsdd D]B} t| ts| j| | jd t| d D| jd t| d |s|r9t|| } | Dcgc] }| j!|j"s|"}}n| }|Dcgc]}|j"}}t%d|D}t'|d }|st j(dyt|d|}|sK|sIt j(dj+d|Dt j,d|ddd|rGt j(dj+fd|Dt j(d|dy| j/|t j0d|ddt||y|}|stjd |r7t j(d!t2d"|t j(d#yt j,d$|d%d|| j/|gt j0d&|'ycc}wcc}w)(zbRemove files from a bucket. To delete an entire bucket, use `hf buckets delete` instead. z?No file path specified. To remove files, provide a path (e.g. 'zd/FILE') or use --recursive to remove all files. To delete the entire bucket, use `hf buckets delete z`.z,--include and --exclude require --recursive.r#zListing files from remoteNTrQzListing files from remote (z files))include_patternsexclude_patternsc34K|]}|jywN)rF).0fs r3 zremove..s7A7s)r;zNo files to remove.z file(s) totaling  c3&K|] }d| yw)z N)rmr9s r3rozremove..sBtD6{BszRemove z from 'r`Frac3:K|]}dtd|yw)delete: r7N)r)rmr9rs r3rozremove..s#cPT- 1TFKcsz (dry run) z would be removed.)r\zRemoved ')r files_deletedrFzFile path cannot be empty.rtr7z"(dry run) 1 file would be removed.zRemove 'z' from 'z File removed)r9r)r r,r)r*r+rrstatusrV isinstancerappendupdater8donermatchesr9sumrtextjoinrbbatch_bucket_filesr.r)r5r=r]rdrerfr r1rRr0rw all_filesitemmatcherrn matched_files file_paths total_sizesize_str count_label file_pathrs @r3removerTszx (F I  F |I   k"DDM;b R  7I  !OPP 5 !C78&( (( >T)  UD $ +  & ;C N;K7ST U  1#i.1AIJ g#WwWG(1M1W__QVV5LQMMM%M&34aff4 4777 z$? HH* + Z));H:F 7 HHTYYBzBB C KK'+gi[C O  HHTYYcXbcc d HHz+.@A B  y< {m79+Q 7j/   $$%AB B  HHx yk9+F G HH9 :  hyk)B?SI y)= > YGYN5s L:LLmove)z/hf buckets move user/old-bucket user/new-bucketz/hf buckets move user/my-bucket my-org/my-bucketzIhf buckets move hf://buckets/user/old-bucket hf://buckets/user/new-bucketfrom_idzMSource bucket ID: namespace/bucket_name or hf://buckets/namespace/bucket_nameto_idzRDestination bucket ID: namespace/bucket_name or hf://buckets/namespace/bucket_namect|}|jr tjd|dtdt|}|jr tjd|dtdt |}|j |j|jtjd|j|jy)z2Move (rename) a bucket to a new name or namespace.z)Cannot specify a prefix for bucket move: r$r%r#)rrz Bucket movedN) r r)r*r+rr move_bucketr,rr.)rrr parsed_from parsed_tor0s r3rrs2$G,K  7yA--:O;Q S  "%(I  7w?--:O;Q S  5 !COOKNN),,O?JJ~{~~Y\\Jr4sync)z2hf buckets sync ./data hf://buckets/user/my-bucketz2hf buckets sync hf://buckets/user/my-bucket ./dataz;hf buckets sync ./data hf://buckets/user/my-bucket --deletez^hf buckets sync hf://buckets/user/my-bucket ./data --include "*.safetensors" --exclude "*.tmp"zIhf buckets sync ./data hf://buckets/user/my-bucket --plan sync-plan.jsonlz'hf buckets sync --apply sync-plan.jsonlzhf buckets cp my-config.json hf://buckets/user/my-bucket/logs/zKhf buckets cp my-config.json hf://buckets/user/my-bucket/remote-config.jsonz7hf buckets cp - hf://buckets/user/my-bucket/config.jsonzchf buckets cp hf://buckets/user/my-bucket/logs hf://buckets/user/archive-bucket/ # nests logs/ dirzihf buckets cp hf://buckets/user/my-bucket/logs/ hf://buckets/user/archive-bucket/ # copies contents onlyzehf buckets cp hf://datasets/user/my-dataset/processed/ hf://buckets/user/my-bucket/dataset/processed/srczJSource: local file, any hf:// URI (model, dataset, bucket), or - for stdindstz=Destination: local path, bucket hf://... URI, or - for stdoutcH t|}t|}|duxr t|}t|}|duxr t|}|dk(}|dk(} |r.|r, |j||tjd||y|s0|s.|s,|t j dt j d|r|st j d|rB|r@t|} | jd k(s|jd rt j d | r|st j d |s6|s4tjj|rt j d |rt|} | j | j} } | d k(s|jd rt j d| j#d dd}| rt%5t'5}tjj)||}|j+| | |fgt-|d5}|j/dx}r=t0j2j4j7||j/dx}r=dddddddddy||}nrtjj|s0|jtj8s|jd r!tjj)||}n|}tjj;|}|rtj<|d|j+| | |fgtjd||y|rvt|} t0j>j4j/}|jA| j || jfgtjdd|ytjjC|st j d|t|} | j | j} } | d k(r tjjE|}n9|jd r&| d ztjjE|z}n| }|jA| ||fgtjd|tF| d |y#t$r#} t j t| d} ~ wwxYw#1swYxYw#1swYxYw#1swYyxYw)zCopy files to or from buckets.r#N-Copied)rrz2Missing destination. Provide a bucket path as DST.z;One of SRC or DST must be a bucket path (hf://buckets/...).z+Stdin upload requires a bucket destination.rhr7zAStdin upload requires a full destination path including filename.z"Cannot pipe to stdout for uploads.zNSource must be a file, not a directory. Use `hf buckets sync` for directories.zJSource path must include a file name, not just a bucket or directory path.rrbiHT)r Downloaded)addUploadedstdinzSource file not found: )$rr r copy_files ValueErrorr*r+strrr.r r)endswithosr9isdirr,rsplitr r rdownload_bucket_filesopenreadsysstdoutbufferwritesepdirnamemakedirsrrisfilebasenamer)rrr r0 src_is_hf dst_is_hf src_is_bucket dst_is_bucket src_is_stdin dst_is_stdouter1rrRfilenametmp_dirtmp_pathrnchunk local_path parent_dirdata remote_paths r3rrs]4 5 !C#I42IcNI#C(MtO<(H--i68:L9MNh-;'(vvj'99e9JJ--33E:()vvj'99e9;; ; ;{% s#s||BFF';s||C?PWW\\#x8  4J J6  % %i6:2F1G H JJ|* = "3'yy$$& vyyf6I6I/J.KL :74ww~~c"$$'>se%DE E"3'"IIv':':6 R<''**3/K \\#  3,)9)9#)>>K K y[/A.BC :3}oi[+,WXo -$$SV, , -V;;;; ; ;sVS TAT AS>/T 7T S;S66S;>T T  T TT!)FNFN)NFFFNNrl)FFN)FFFNNN)NNFFFNNFNNNFFFN)NN)4__doc__rrtypingrr*huggingface_hubrhuggingface_hub._bucketsrrrrr huggingface_hub.utilsr r r hf_apir _cli_utilsrrrr _file_listingrr_outputrr get_logger__name__logger buckets_clicommandrArgumentboolOptionrr:rDrBrArWr\rUrrrrrrr4r3rsn >  #[Z! ;.   H %!EF    0   90R g  0R  +  0R t w   0R*  F  +0R8 90R: ;0R 0RfCD   0    E>  d W  >    7  > $  F  %> 2   C  3> B C> D E> F G> > B KTz K $J K K K  K : K  KFcccc c : c  c0  (5 " Y   "  "  " "   .  -.6 Y  .6    ,   .6  F  .6, -.6. /.6.6b   6    [H a  H   C  H$    ,   %H4  K  5HB S D d  CHN S D _  OHZ [H\ ]H HD  *"K  `  "K  e   "K "K "K"KT    &         yrP  d ^   rP  d c   rP   B   rP&  N  'rP4  O  5rPB  d  P   CrPN  d  6   OrPZ  H  [rPh S D I  irPt S D I  urP@ d  ;  ArPL  T  MrPZ  V  [rPh   8  irPx yrPz {rP rPt   , nY  ^U^^!m nn nY  d NENN(ghh  nY nY nY! nYr4