llm_oracle

llm_oracle

LLM Oracle是一種用於預測未來事件的GPT-4動力工具。就像一個魔術8球能夠執行基礎研究，計算和推理。

演示網站

這是如何運作的？

提示

在高水平上，整個過程只是GPT-4的包裝紙。

1. 你問一個問題
2. 我們問gpt Is {question} a valid question for a prediction market? ，如果沒有錯誤
3. 我們問gpt

 Will the answer to "{question}" be yes?
Today is {date}

You can use these tools: Calculator, Wolfram Alpha, Google, etc.

Respond with arguments for and against, then with a final prediction.

4. GPT進入及時的鏈（React）以收集信息以最好地回答問題
5. 我們將最終預測（例如SOMEWHAT_LIKELY ）解析為0％至100％之間的概率。

限制

• 預計該模型的輸出不會進行校準（例如，它說80％的事件，只有50％最終才能正確）
• 該模型僅限於顯然可以用Google回答問題的方法。它無法執行詳細的研究，構建預測模型並運行高級多部分計算。
• LLM的典型局限性也像脆弱性一樣適用，以提示在問題和收集的互聯網文本，幻覺，偏見等中註射。

應用隱私

這是您使用演示應用程序時節省的內容的快速概述。如果發生違規，這也是可以獲得的。

存儲“永遠”

• 您提出的問題（任何具有預測ID的人或通常在設置中設置的“公共”可以查看的任何人）
• 與問題關聯的唯一匿名用戶ID（您可以使用新的瀏覽器，隱身等）
• GPT代理日誌可能包含有關該問題的公開獲取信息

未使用 /存儲

• 您共享的社交媒體帳戶

Python API

安裝

1. pip install git+https://github.com/sshh12/llm_oracle
2. 環境

 OPENAI_API_KEY=
SERPER_API_KEY= (required for tool agents)
SCRAPINGBEE_API_KEY= (required for tool agents)
WOLFRAM_ALPHA_APPID= (required for tool agents)
KALSHI_EMAIL= (required for kalshi API)
KALSHI_PASSWORD= (required for kalshi API)

DATABASE_URL= (required for demo app)

例子

 from llm_oracle . markets . kalshi import KalshiMarket
from llm_oracle . markets . custom import CustomMarket , CustomEvent
from llm_oracle . markets . manifold import ManifoldMarket
from llm_oracle . agents . agent_basic import BasicAgentv1 , BasicAgentv2 , BasicAgentv3
from llm_oracle . agents . agent_tools import ToolAgentv1 , ToolAgentv2 , ToolAgentv3
import datetime
import os

manifold_market = ManifoldMarket ()
kalshi_market = KalshiMarket ( email = os . environ [ "KALSHI_EMAIL" ], password = os . environ [ "KALSHI_PASSWORD" ])

kalshi_event_ids = [
    "GTEMP-23-P1.02" ,
    "NPPC-24DEC31" ,
    "BIDENVNEBRASKA-24DEC31" ,
    "TIKTOKBAN-23DEC31" ,
    "SFFA-COMPLETE" ,
    "COIN-23DEC31" ,
    "HURCTOTMAJ-23DEC01-T3" ,
    "SCOTUSN-23" ,
    "MOON-25" ,
]

manifold_event_ids = [
    "will-lex-fridman-interview-ai-by-20" ,
    "will-biden-be-the-2024-democratic-n" ,
    "will-a-nuclear-weapon-be-detonated-b71e74f6a8e4" ,
]

custom_market = CustomMarket (
    [
        CustomEvent (
            "Will a humanity be replaced by AI by 2050?" ,
            datetime . datetime ( 2050 , 1 , 1 ),
        ),
        CustomEvent (
            "Will a random number that I pull from a uniform distribution [0, 100] be greater or equal to 99?" ,
            datetime . datetime ( 2025 , 1 , 1 ),
        ),
    ]
)


EVENTS = (
    [ kalshi_market . get_event ( kid ) for kid in kalshi_event_ids ]
    + [ manifold_market . get_event ( mid ) for mid in manifold_event_ids ]
    + custom_market . events
)

AGENTS = {
    "basic_v1" : BasicAgentv1 (),
    "basic_v2" : BasicAgentv2 (),
    "basic_v3" : BasicAgentv3 (),
    "tool_v1" : ToolAgentv1 (),
    "tool_v2" : ToolAgentv2 (),
    "tool_v3" : ToolAgentv3 (),
}


for event in EVENTS :
    if not event . is_active ():
        continue
    title = event . get_title ()
    event_uid = event . get_universal_id ()
    market_p = event . get_market_probability ()
    for agent_name , agent in AGENTS . items ():
        p = agent . predict_event_probability ( event )
        with open ( "predictions.tsv" , "a" ) as f :
            f . write ( f" { event_uid } t { title } t { market_p } t { agent_name } t { p } n " )