openai elasticsearch rag

使用OpenAI和Elasticsearch作为矢量数据库检索增强发电（RAG）应用程序

用OpenAI和Elasticsearch在长格式文本上提供抹布（检索效果的生成）应用程序。我们在这里

• 使用Elasticsearch作为VectordB存储索引数据并将其用于检索。
• 从ES检索相关的人后，使用OpenAI创建嵌入并为问题产生答案。

入门

先决条件：

1. Python 3.9或更高
2. pip
3. Virtualenv
4. Elasticsearch -7.14.0（https：//www.elastic.co/downloads/elasticsearch）
5. OpenAI API键（https://platform.openai.com/）

安装步骤：

• 创建和激活虚拟环境

python -m venv venv

 source venv/bin/activate

• 安装依赖项

pip install -r requirements.txt

• 下载用于NLP的Spacy模型，然后将文本分为句子。

python -m spacy download en_core_web_sm

• Elasticsearch设置（https://www.elastic.co/downloads/elasticsearch）
  1. 下载并解开弹性搜索
  2. 启动Elasticsearch bin/elasticsearch （或Windows上的binelasticsearch.bat ）
  3. 通过运行curl -X GET "localhost:9200/"

运行应用程序

在运行应用程序之前，请使用.env-template创建.env文件，然后添加OpenAI API键和Elasticsearch凭据。

python run.py

应用程序将在端口8081上的localhost上开始。通过在终端运行的理智检查。它应该返回Hello消息。

curl -X GET " localhost:8081 " 

索引和查询数据

1。创建ES索引

curl --location --request PUT ' http://localhost:9200/first-index ' 
--header ' Content-Type: application/json ' 
--data-raw ' {
    "mappings": {
        "properties": {
            "text": {
                "type": "text"
            },
            "embedding": {
                "type": "dense_vector",
                "dims": 1536
            }
        }
    }
} '

如果要删除用于干净启动的索引，请使用以下命令。

curl --location --request DELETE ' http://localhost:9200/first-index '

2。索引数据

要索引数据，请使用以下curl命令。此命令将邮政请求发送到指定的端点，其中包含索引索引的文本和索引名称的JSON有效载荷。

确保您的服务器在端口8081上的localhost上运行。

curl --location --request POST ' localhost:8081/api/index ' 
--header ' Content-Type: application/json ' 
--data-raw ' {
  "text": "Ajeet is an engineer turned product entrepreneur with experience in AI, SaaS, HealthTech and EdTech. He is a technology enthusiast and loves to work on new technologies. He was a founding member of leading health-tech startups HealthKart and TATA 1mg in India. He was the founder of Joe Hukum, a chatbot platform which was acquired by Freshworks. After Freshworks, he founded Seekho.ai to solve for the skill gap in Indian higher education. Currently, he is on a break and is exploring GenAI to solve for the next meaningful problem. He is passionate about solving zero to one problems and building products that can impact millions of lives.",
  "index_name": "first-index"
} '

3。查询数据

示例1：

curl --location --request POST ' localhost:8081/api/query ' 
--header ' Content-Type: application/json ' 
--data-raw ' {
  "question": "Who is Ajeet?",
  "index_name": "first-index"
} '

示例2：

curl --location --request POST ' localhost:8081/api/query ' 
--header ' Content-Type: application/json ' 
--data-raw ' {
  "question": "Joe Hukum was acquired by which company?",
  "index_name": "first-index"
} ' 

由Ajeet与☕️和❤️建造