biw`$ddlZddlZddlZddlZddlZddlZddlZddlmZddl m Z m Z ddl m Z mZddlZddlZddlZddlmZmZddlmZddlmZ ddlZn #e$rYnwxYwejdkrddlZerdd lmZdd lm Z dd l!m"Z"d d dZ#e$ej%ej&zZ'e$ej%ej(zej&zdzZ)ddiddiddiddiddiddiddidZ*dZ+dd,e*-dZ.GddZ/Gdd Z0d!e1dzd"efd#Z2d$e3e1dzd"e4fd%Z5d0d&e1dzd"e3e4fd'Z6d(e1d)e3e4d*e4d"e1fd+Z7d"e1fd,Z8e9d-kre0d./dSdS)1N) AsyncIterator) AnnotatedAny)urljoinurlparse)AsyncInferenceClientChatCompletionStreamOutput)GenerationConfig)is_rich_availableWindows)Console)Live)Markdown localhosti@)hostnameportz .!\"#$%&'()*+,\-/:<=>?@[]^_`{|}~textz5There is a Llama in my lawn, how can I get rid of it?zyWrite a Python function that integrates any Python function f(x) numerically over an arbitrary interval [x_start, x_end].z4How many helicopters can a human eat in one sitting?z4Count to 10 but skip every number ending with an 'e'zWhy aren't birds real?z2Why is it important to eat socks after meditating?z$Which number is larger, 9.9 or 9.11?)llamacode helicopternumbersbirdssocksnumbers2a **TRANSFORMERS CHAT INTERFACE** Chat interface to try out a model. Besides chatting with the model, here are some basic commands: - **!help**: shows all available commands (set generation settings, save chat, etc.) - **!status**: shows the current status of the model and generation settings - **!clear**: clears the current conversation and starts a new one - **!exit**: closes the interface am **TRANSFORMERS CHAT INTERFACE HELP** Full command list: - **!help**: shows this help message - **!clear**: clears the current conversation and starts a new one - **!status**: shows the current status of the model and generation settings - **!example {NAME}**: loads example named `{NAME}` from the config and uses it as the user input. Available example names: `z`, `a#` - **!set {ARG_1}={VALUE_1} {ARG_2}={VALUE_2}** ...: changes the system prompt or generation settings (multiple settings are separated by a space). Accepts the same flags and format as the `generate_flags` CLI argument. If you're a new user, check this basic flag guide: https://huggingface.co/docs/transformers/llm_tutorial#common-options - **!save {SAVE_NAME} (optional)**: saves the current chat and settings to file by default to `./chat_history/{MODEL_ID}/chat_{DATETIME}.yaml` or `{SAVE_NAME}` if provided - **!exit**: closes the interface ceZdZdedefdZdeedeeeezdzffdZ defdZ d Z d efd Z d ed efd Z ddededefdZddefdZdefdZdS) RichInterfacemodel_iduser_idcHt|_||_||_dSN)r _consolerr)selfrrs P/root/projects/butler/venv/lib/python3.11/site-packages/transformers/cli/chat.py__init__zRichInterface.__init__gs     streamreturnNc(K|jd|jdt|jd5}d}d}|d{V23d{V}|jdjj}t|jdd|}|s=tj dd |}||z }g}| D]W}| || d r| d B| d Xtd|d } || d6 dddn #1swxYwY|j||fS)Nz [bold blue]:)consolerefresh_per_secondr finish_reasonz <(/*)(\w*)>z\<\1\2\>z``` z z github-dark) code_themeT)refresh)r!printrrchoicesdeltacontentgetattrresub splitlinesappend startswithrjoinstripupdate) r"r&liverr-tokenoutputslineslinemarkdowns r# stream_outputzRichInterface.stream_outputls <4=<<<=== $-A > > >( 4$D(,M%+||||||% 4% 4% 4% 4% 4% 4% 4e-*08 ' a(8/= Y Y &gFF OO----DLL&&&u--- T**** V,,,,#BGGENN$8$8$:$:}UUU Hd 3333K ,|( 4( 4( 4( 4( 4( 4( 4( 4( 4( 4( 4( 4( 4( 4( 4T ]""s E, EDE,,E03E0c~|jd|jd}|j|S)z!Gets user input from the console. [bold red]: )r!inputrr1)r"rGs r#rGzRichInterface.inputs> ##$D$,$D$D$DEE  r%c8|jdS)zClears the console.N)r!clear)r"s r#rIzRichInterface.clears r%rc|jd|jd||jdS)z%Prints a user message to the console.rFz>:[/ bold red] N)r!r1r)r"rs r#print_user_messagez RichInterface.print_user_messagesE N$,NNNNOOO r%colorcx|jd|d||jdS)z,Prints text in a given color to the console.z[bold ]N)r!r1)r"rrLs r# print_colorzRichInterface.print_colorsC 3U33T33444 r%Fmessagedefaultc|rdnd}|jd|d|d}|j|}|s|S|dvS)zFDisplays a yes/no prompt to the user, returning True for confirmation.zY/nzy/Nz [bold yellow]z (z): >yyes)r!rGr1r<lower)r"rPrQ default_hintresponses r#confirmzRichInterface.confirms '2uuU =&&'Sw'S'S,'S'S'STT >>##))++ N<''r%minimalc|jt|rtnt|jdS)z'Prints the help message to the console.N)r!r1rHELP_STRING_MINIMAL HELP_STRING)r"rYs r# print_helpzRichInterface.print_helpsE HG%T%8%8UUVVV r%configc|jd|jd|jd||jdS)zFPrints the status of the model and generation settings to the console.z[bold blue]Model: r.z [bold blue]N)r!r1r)r"r^s r# print_statuszRichInterface.print_statuss_ BBBBCCC 2&22333 r%)F)__name__ __module__ __qualname__strr$rr tuplerrDrGrIrKrOboolrXr]r r`r%r#rrfsIs .#-8R*S.#X]^acfilcloscs^sXt.#.#.#.#`s s C ( (s (T (d ( ( ( ($ #3r%rceZdZdZdeddedddddddfdeeejd fd eedzejd fd ee edzejd fdeedzej d fdeedzej d fdeeej d fdeedzej d fdeedzej d fddfdZ e dZ dededeeeeeffded e edee eeff d!Zd"ZdS)#Chat(Chat with a model from the command line.zhttp://r:rNz./chat_history/rz9ID of the model to use (e.g. 'HuggingFaceTB/SmolLM3-3B').)helpbase_urlz7Base url to connect to (e.g. http://localhost:8000/v1).generate_flagsaFlags to pass to `generate`, using a space as a separator between flags. Accepts booleans, numbers, and lists of integers, more advanced parameterization should be set through --generation-config. Example: `transformers chat max_new_tokens=100 do_sample=False eos_token_id=[1,2]`. If you're a new user, check this basic flag guide: https://huggingface.co/docs/transformers/llm_tutorial#common-optionsuserzKUsername to display in chat interface. Defaults to the current user's name. system_promptzSystem prompt. save_folderzFolder to save chat history. examples_pathz"Path to a yaml file with examples.generation_configzPath to a local generation config file or to a HuggingFace repo containing a `generation_config.json` file. Other generation settings passed as CLI arguments will be applied on top of this generation config.r'c ||_t|j} | jtdkr0| jtdkr||j||_||_||_t|} | dd| j d it|| |_ |||j d|_||n t|_|rAt#|5} t%j| |_dddn #1swxYwYn t*|_t-st/dt1j|dS) rjrrT) do_samplemax_new_tokens)rmrr^NzHYou need to install rich to use the chat interface. (`pip install rich`)rg)rmrrDEFAULT_HTTP_ENDPOINTr check_healthrrprqload_generation_configr=parse_generate_flagsr^to_dictsettings get_usernameroopenyaml safe_loadexamplesDEFAULT_EXAMPLESr ImportErrorasynciorun _inner_run) r"rrmrnrorprqrrrsparsedr^fs r#r$z Chat.__init__sF! $-(( ?3J? ? ?FKShioSpDpDp   dm , , ,  *&((9:: S 999 ==,^<<=== %-8t{ObObOdOdee !,DD,..   -m$$ 2 $q 1 1  2 2 2 2 2 2 2 2 2 2 2 2 2 2 2-DM!"" jhii i  DOO%%&&&&&sD..D25D2ct|dzd} tj|}|jdkrt d|d|jdn&#tj$rt d|dwxYwd S) N/healthzThe server running on z returned status code z on health check (/health).zNo server currently running on z. To run a local server, please run `transformers serve` in aseparate shell. Find more information here: https://huggingface.co/docs/transformers/servingT)rhttpxget status_code ValueError ConnectError)url health_urloutputs r#ryzChat.check_healthsS3Y11  Yz**F!S(( wSwwHZwww)!   p#ppp   ts :A#A3 user_input interfacerr^chatcd}|dkr*t|j}|n|dkr|n|drt |dkr|}t |dkr|dn@tj |j |j dtj dd }t|||j |d |d d n|drr|dd} | } | D]#} d| vr|d| ddn$|jd"it'| nB|drt |dkr|d} | |vr\|g}||| d|d|| ddnd| dt-|d} || dnN|dkr||n1d}|d |d!d||||fS)#z Handles all user commands except for `!exit`. May update the chat history (e.g. reset it) or the generation config (e.g. set a new flag). T!clear!help!savechat_%Y-%m-%d_%H-%M-%S.jsonfilenamerr}Chat saved to !greenrrL!setr)N=(Invalid flag format, missing `=` after `;`. Please use the format `arg_1=value_1 arg_2=value_2 ...`.red!examplerroroler4Example * not found in list of available examples: .!statusr^F'/' is not a valid command. Showing help message.rg)new_chat_historyrprIr]r:lensplitospathr;rqrtimestrftime save_chatr}rOr<r=r{rKr9listkeysr`) r"rrrr^r valid_command split_inputrnew_generate_flagsflag example_name example_errors r#handle_non_exit_user_commandsz"Chat.handle_non_exit_user_commands$sp  ! !#D$677D OO     7 " "  " " " "  " "7 + +1 #J4D4D4F4F0G0G!0K0K$**,,K{##q((AW\\$"2DMCt4=YlKmKmCtCtCtuu  xdT] K K K K  ! !'C'C'C'C7 ! S S S S  " "6 * *' #",ABB!5!5!7!7 !3!9!9!;!; * J Jd??))AtAAA$ *E# II 45G H HIII  " ": . . #3z7G7G7I7I3J3Ja3O3O%++--a0Lx''!!!,,Xl-CF-KLLL V8Nv8VWWXXXXp|ooW[\d\i\i\k\kWlWlooo%%=%FFFF 9 $ $  " "& " 1 1 1 1"M  ! !'f:'f'f'fns ! t t t  " " "]F**r%c Kt|j|j}|t |j}|d|j}t|j 4d{V}d} ||}d}| |n| }|dkrn|dkr)t |j}|i|dkr|| drt|d kr|}t|d kr|d n@tj|j|jd t'jd d }t+|||j|d|ddb| drr|dd} | } | D]#} d| vr|d| ddn$|jd,it5| | drt|d kr|d } | |jvrf|g}| |j| d|d|j| ddnd| dt;|jd} || dn||d kr||!| dr1|d"|d#d|U|d|d| |d|j|!|jd$%} |"| d{V\}}|d&|d|d'kr/|d(d)|#d*rd+}n#tH$rYnwxYw" dddd{VdS#1d{VswxYwYdS)-N)rrT)rY)rmz!exitrrrrrrrrrrrrrrr)rrrrrrrorrrrrrrr)rsmodel)r&r extra_body assistantlengthz2Generation stopped after reaching the token limit.yellowzContinue generating?u'Please continue. Do not repeat text.”rg)%rrrorIrrpr]r^rrmrKrGr:rrrrr;rqrrrr}rOr<r=r{rr9rrr`chat_completionto_json_stringrDrXKeyboardInterrupt)r"rrr^clientpending_user_inputrrrrrrrr& model_outputr-s r#rzChat._inner_runnsw!4=$)LLL  233 T***'???g g g g g g g 6-1 e d)5%7 -1*!44Z@@@@%.__%6%6 "W,,#x///0BCC!))) #w..!,,... #..w779MC @P@P@R@RR86R9R8;.R)R8*RR8CR!R8#BR(R8*D$RR8ARR8B9RR8 RR8RR88 SS)rarbrc__doc__rxrrdtyperArgumentrOptionr$ staticmethodryrdictr rerrrgr%r#riris22 Z+J7 Y Y:OPV:W Y Y  TXYjhl AD'D'C5p!q!q!qqrD' $J,efff f D' " I  EN[      D'& $J ELk l l l n 'D'.!t\U\?O-P-P-P!PQ/D'0sLEL6T$U$U$UUV1D'2!t\U\?c-d-d-d!de3D'6% $J ELg     7D'B CD'D'D'D'L\ H+H+!H+sDcN*+ H+ ! H+ 4j H+ tDz++ ,H+H+H+H+Tqqqqqr%rirsr'c|tSd|vrStj|}tj|}tj||Stj|S)Nr)r rrdirnamebasenamefrom_pretrained)rsrrs r#rzrzso !!!###'//"3447##$566/BBB/0ABBBr%rnc|t|dkriSd|D}d|D}d|D}dtdtfdfd |D}d d |D}d |zd z}|dd}|dd}|dd}|dd}|dd}|dd} t j|}n"#t j$rtdwxYw|S)zUParses the generate flags from the user input into a dictionary of `generate` kwargs.Nrci|]=}d|ddzdz|dd>S)"rrr)r).0rs r# z(parse_generate_flags..sClllUYcDJJsOOA$66Plllr%cni|]2\}}||dvr|n|3S))truefalse)rUrkvs r#rz(parse_generate_flags..sJBF!Q %666177999Ar%c*i|]\}}||dkrdn|S)Nonenullrgrs r#rz(parse_generate_flags..s*iii$!Qa1;;Aiiir%sr'c~|d}|dddS)N-rr,r) removeprefixreplaceisdigit)rs r# is_numberz'parse_generate_flags..is_numbers5 NN3  yyb!$$,,...r%c>i|]\}}||sd|dn|S)rrg)rrrrs r#rz(parse_generate_flags..s8pppAa1!DQ1pppr%z, c"g|] \}}|d| S)z: rgrs r# z(parse_generate_flags.. s&&^&^&^tq!!{{q{{&^&^&^r%{}z"null"rz"true"rz"false"rz"[[z]"rNrrkzFailed to convert `generate_flags` into a valid JSON object. `generate_flags` = {generate_flags} Converted JSON string = {generate_flags_string}) ritemsrdrfr;rjsonloadsJSONDecodeErrorr)rngenerate_flags_as_dictgenerate_flags_stringprocessed_generate_flagsrs @r#r{r{s^!4!4!9!9 ml]klllJ`JfJfJhJhjiJ`JfJfJhJhiii/S/T//// qpppQgQmQmQoQoppp !II&^&^?U?[?[?]?]&^&^&^__ "77#=299(FKK199(FKK199)WMM199$DD199$DD299#sCC #':.C#D#D       @    $#s E&&Frpc|rd|dgngS)z Returns a new chat conversation.systemrrg)rps r#rr$s=J RX- 8 8 9 9PRRr%rrr}c*tjtj|dt |d5}t j||d|ddddn #1swxYwYtj|S)z!Saves the chat history to a file.T)exist_okw)r} chat_historyr))indentN)rmakedirsrrrrdumpabspath)rrr}rs r#rr)sK))D9999 h  M x>>!LLLLMMMMMMMMMMMMMMM 7??8 $ $$sA++A/2A/ctjdkrtjSt jtjjS)z)Returns the username of the current user.r )platformrrgetloginpwdgetpwuidgetuidpw_namergr%r#r~r~1s:I%%{}}|BIKK((00r%__main__z meta-llama/Llama-3.2-3b-Instruct)rr ):rrrr r6stringrcollections.abcrtypingrr urllib.parserrrrrhuggingface_hubrr transformersr transformers.utilsr readlinerrr  rich.consoler rich.liver rich.markdownrrxset ascii_letters whitespaceALLOWED_KEY_CHARSdigitsALLOWED_VALUE_CHARSrr[r;rr\rrirdrzrrr{rrr~rargr%r#r#s  ))))))!!!!!!!!******** LLLLLLLL))))))000000 OOOO   D 8? !!JJJ'$$$$$$&&&&&&%0$??C,v/@@AAc 6=(6+<'>?? &________DWWWWWWWWt CcDj C=M C C C C3$cT)93$d3$3$3$3$lSSC$JS$t*SSSS %%4:%%#%%%%1c1111 zD4555555sAA! A!