knowledge gpt
1.0.0
InswardingGPT旨在从各种来源（包括Internet和本地数据）收集信息，这些信息可用于创建提示。然后，可以通过OpenAI的GPT-3模型来利用这些提示来生成随后存储在数据库中的答案，以供将来参考。
为了实现这一目标，首先使用开源模型或OpenAI模型将文本转换为固定尺寸的向量。提交查询时，文本也将转换为向量并将其与存储的知识嵌入进行比较。然后选择最相关的信息并用于生成及时上下文。
知识依据支持各种信息来源，包括网站，PDF，PowerPoint文件（PPTX）和文档（文档）。此外，它可以从YouTube字幕和音频（使用语音到文本技术）中提取文本，并将其用作信息来源。这允许收集和用于生成提示和答案的各种信息。
PYPI安装，以终端运行： pip install knowledgegpt
或者，您可以使用来自存储库的最新版本： pip install -r requirements.txt ，然后pip install .
下载用于解析的需要的语言模型： python3 -m spacy download en_core_web_sm
uvicorn server:app --reload
# Import the library
from knowledgegpt . extractors . web_scrape_extractor import WebScrapeExtractor
# Import OpenAI and Set the API Key
import openai
from example_config import SECRET_KEY
openai . api_key = SECRET_KEY
# Define target website
url = "https://en.wikipedia.org/wiki/Bombard_(weapon)"
# Initialize the WebScrapeExtractor
scrape_website = WebScrapeExtractor ( url = url , embedding_extractor = "hf" , model_lang = "en" )
# Prompt the OpenAI Model
answer , prompt , messages = scrape_website . extract ( query = "What is a bombard?" , max_tokens = 300 , to_save = True , mongo_client = db )
# See the answer
print ( answer )
# Output: 'A bombard is a type of large cannon used during the 14th to 15th centuries.'其他示例可以在示例文件夹中找到。但是，为了更好地了解如何使用库，这是一个简单的例子：
# Basic Usage
basic_extractor = BaseExtractor ( df )
answer , prompt , messages = basic_extractor . extract ( "What is the title of this PDF?" , max_tokens = 300 ) # PDF Extraction
pdf_extractor = PDFExtractor ( pdf_file_path , extraction_type = "page" , embedding_extractor = "hf" , model_lang = "en" )
answer , prompt , messages = pdf_extractor . extract ( query , max_tokens = 1500 ) # PPTX Extraction
ppt_extractor = PowerpointExtractor ( file_path = ppt_file_path , embedding_extractor = "hf" , model_lang = "en" )
answer , prompt , messages = ppt_extractor . extract ( query , max_tokens = 500 ) # DOCX Extraction
docs_extractor = DocsExtractor ( file_path = "../example.docx" , embedding_extractor = "hf" , model_lang = "en" , is_turbo = False )
answer , prompt , messages =
docs_extractor . extract ( query = "What is an object detection system?" , max_tokens = 300 ) # Extraction from Youtube video (audio)
scrape_yt_audio = YoutubeAudioExtractor ( video_id = url , model_lang = 'tr' , embedding_extractor = 'hf' )
answer , prompt , messages = scrape_yt_audio . extract ( query = query , max_tokens = 1200 )
# Extraction from Youtube video (transcript)
scrape_yt_subs = YTSubsExtractor ( video_id = url , embedding_extractor = 'hf' , model_lang = 'en' )
answer , prompt , messages = scrape_yt_subs . extract ( query = query , max_tokens = 1200 )docker build -t knowledgegptimage .
docker run -p 8888:8888 knowledgegptimage（要扩展...）
（用更好的图像更新）
