openai elasticsearch rag

使用OpenAI和Elasticsearch作為矢量數據庫檢索增強發電（RAG）應用程序

用OpenAI和Elasticsearch在長格式文本上提供抹布（檢索效果的生成）應用程序。我們在這裡

• 使用Elasticsearch作為VectordB存儲索引數據並將其用於檢索。
• 從ES檢索相關的人後，使用OpenAI創建嵌入並為問題產生答案。

入門

先決條件：

1. Python 3.9或更高
2. pip
3. Virtualenv
4. Elasticsearch -7.14.0（https：//www.elastic.co/downloads/elasticsearch）
5. OpenAI API鍵（https://platform.openai.com/）

安裝步驟：

• 創建和激活虛擬環境

python -m venv venv

 source venv/bin/activate

• 安裝依賴項

pip install -r requirements.txt

• 下載用於NLP的Spacy模型，然後將文本分為句子。

python -m spacy download en_core_web_sm

• Elasticsearch設置（https://www.elastic.co/downloads/elasticsearch）
  1. 下載並解開彈性搜索
  2. 啟動Elasticsearch bin/elasticsearch （或Windows上的binelasticsearch.bat ）
  3. 通過運行curl -X GET "localhost:9200/"

運行應用程序

在運行應用程序之前，請使用.env-template創建.env文件，然後添加OpenAI API鍵和Elasticsearch憑據。

python run.py

應用程序將在端口8081上的localhost上開始。通過在終端運行的理智檢查。它應該返回Hello消息。

curl -X GET " localhost:8081 " 

索引和查詢數據

1。創建ES索引

curl --location --request PUT ' http://localhost:9200/first-index ' 
--header ' Content-Type: application/json ' 
--data-raw ' {
    "mappings": {
        "properties": {
            "text": {
                "type": "text"
            },
            "embedding": {
                "type": "dense_vector",
                "dims": 1536
            }
        }
    }
} '

如果要刪除用於乾淨啟動的索引，請使用以下命令。

curl --location --request DELETE ' http://localhost:9200/first-index '

2。索引數據

要索引數據，請使用以下curl命令。此命令將郵政請求發送到指定的端點，其中包含索引索引的文本和索引名稱的JSON有效載荷。

確保您的服務器在端口8081上的localhost上運行。

curl --location --request POST ' localhost:8081/api/index ' 
--header ' Content-Type: application/json ' 
--data-raw ' {
  "text": "Ajeet is an engineer turned product entrepreneur with experience in AI, SaaS, HealthTech and EdTech. He is a technology enthusiast and loves to work on new technologies. He was a founding member of leading health-tech startups HealthKart and TATA 1mg in India. He was the founder of Joe Hukum, a chatbot platform which was acquired by Freshworks. After Freshworks, he founded Seekho.ai to solve for the skill gap in Indian higher education. Currently, he is on a break and is exploring GenAI to solve for the next meaningful problem. He is passionate about solving zero to one problems and building products that can impact millions of lives.",
  "index_name": "first-index"
} '

3。查詢數據

示例1：

curl --location --request POST ' localhost:8081/api/query ' 
--header ' Content-Type: application/json ' 
--data-raw ' {
  "question": "Who is Ajeet?",
  "index_name": "first-index"
} '

示例2：

curl --location --request POST ' localhost:8081/api/query ' 
--header ' Content-Type: application/json ' 
--data-raw ' {
  "question": "Joe Hukum was acquired by which company?",
  "index_name": "first-index"
} ' 

由Ajeet與☕️和❤️建造