Stream responses

Stream GPT-4，CHETGPT和GPT-3.5響應和部署簡化應用程序

該代碼是用GPT-4 API，CHATGPT API和DENSTICTGPT（GPT-3.5。）模型和STRAMLIT-APP來證明流媒體的用法。

先決條件：

該方法僅使用OpenAI和時間庫，並使用Print（End =''，flush = true）重新打印流：

!p ip install - - upgrade openai
import openai
import time
openai . api_key = user_secrets . get_secret ( "OPENAI_API_KEY" )
startime = time . time ()

免責聲明：生產用法流式傳輸的缺點是控制適當用法政策的控制：https：//beta.openai.com/docs/usage-guidelines，應提前對每個申請進行審查，因此我建議在此策略之前決定使用流媒體。

如何流式傳輸GPT-4 API模型（GPT-4或GPT-4-32K＆GPT-4-32K-0314）響應？

運行文件流。IPNYB第一部分。

 ### STREAM GPT-4 API RESPONSES
delay_time = 0.01 #  faster
max_response_length = 8000
answer = ''
# ASK QUESTION
prompt = input ( "Ask a question: " )
start_time = time . time ()

response = openai . ChatCompletion . create (
    # GPT-4 API REQQUEST
    model = 'gpt-4' ,
    messages = [
        { 'role' : 'user' , 'content' : f' { prompt } ' }
    ],
    max_tokens = max_response_length ,
    temperature = 0 ,
    stream = True ,  # this time, we set stream=True
)

for event in response : 
    # STREAM THE ANSWER
    print ( answer , end = '' , flush = True ) # Print the response    
    # RETRIEVE THE TEXT FROM THE RESPONSE
    event_time = time . time () - start_time  # CALCULATE TIME DELAY BY THE EVENT
    event_text = event [ 'choices' ][ 0 ][ 'delta' ] # EVENT DELTA RESPONSE
    answer = event_text . get ( 'content' , '' ) # RETRIEVE CONTENT
    time . sleep ( delay_time )

插入用戶輸入並按Enter後，您應該看到打印的輸出：

如何流式傳輸chatgpt API模型（GPT-3.5-Turbo）響應？

運行文件流。IPNYB第二部分。添加用戶輸入，您應該看到與以下相似的相似之處：

 ### STREAM CHATGPT API RESPONSES
delay_time = 0.01 #  faster
max_response_length = 200
answer = ''
# ASK QUESTION
prompt = input ( "Ask a question: " )
start_time = time . time ()

response = openai . ChatCompletion . create (
    # CHATPG GPT API REQQUEST
    model = 'gpt-3.5-turbo' ,
    messages = [
        { 'role' : 'user' , 'content' : f' { prompt } ' }
    ],
    max_tokens = max_response_length ,
    temperature = 0 ,
    stream = True ,  # this time, we set stream=True
)

for event in response : 
    # STREAM THE ANSWER
    print ( answer , end = '' , flush = True ) # Print the response    
    # RETRIEVE THE TEXT FROM THE RESPONSE
    event_time = time . time () - start_time  # CALCULATE TIME DELAY BY THE EVENT
    event_text = event [ 'choices' ][ 0 ][ 'delta' ] # EVENT DELTA RESPONSE
    answer = event_text . get ( 'content' , '' ) # RETRIEVE CONTENT
    time . sleep ( delay_time )

如何流式傳輸指令api模型（text-davinci-003）響應？

運行文件流。PNYB第三部分。添加用戶輸入，您應該看到與以下相似的相似之處：

 collected_events = []
completion_text = []
speed = 0.05 #smaller is faster
max_response_length = 200
start_time = time . time ()
prompt = input ( "Ask a question: " )
# Generate Answer
response = openai . Completion . create (
    model = 'text-davinci-003' ,
    prompt = prompt ,
    max_tokens = max_response_length ,
    temperature = 0 ,
    stream = True ,  # this time, we set stream=True
)

# Stream Answer
for event in response :
    event_time = time . time () - start_time  # calculate the time delay of the event
    collected_events . append ( event )  # save the event response
    event_text = event [ 'choices' ][ 0 ][ 'text' ]  # extract the text
    completion_text += event_text  # append the text
    time . sleep ( speed )
    print ( f" { event_text } " , end = "" , flush = True )

如何使用OpenAI API創建簡易應用？

我添加了一個有效的“ app_streamlit.py” -file，您可以用“ unignts.txt”將其分配到存儲庫中，然後在簡化中部署它。

在高級設置中，使用格式添加OpenAI_API_KEY-KEY-KEKEN-VAR-AREVAT：

 OPENAI_API_KEY = "INSERT HERE YOUR KEY" 

建議和改進

隨意叉，並按照許可證進一步改進代碼。例如，您可以進一步改進CHATML，以確保流程遵循所需的“系統”規則。我現在將這些空的東西留下來，使這個基本腳本非常通用。我建議您查看有關ChatGpt API的文章，以了解與流媒體，CHATML：帶有系統，助理和用戶角色以及ChatGpt API簡介教程的媒介中的流媒體響應。