ContextDB

IA: Creating a context database for your prompts

I will show you how to develop a system to optimize the use of prompts in artificial intelligence applications using a context database. This project uses technologies such as Python , Duckdb, Langchain among others. It stands out for the use of DuckDB as a vector database, which helps to manage large data volumes efficiently, while the Langchain library is integrated to explore the capabilities of the ChatgPT-3.5 or higher language model.

Through this integration, the project incorporates advanced techniques such as Embeddings and Retieval-Augmeld Generation (RAG) , demonstrating a practical and sophisticated application of the covered theoretical concepts. This combination of tools and techniques not only raises the accuracy and efficiency of AI applications, but also exemplifies the practice of Few-Shot Prompt , in which a prompt is presented with some examples that contribute to a better understanding of what is being requested.

Video Presentation

Requirements

Application Deploy

Cloning the repository:

git clone https://github.com/Renatoelho/ContextDB.git contextDB

Preparing the environment:

• Accessing the cloned directory

 cd contextDB/app/

• Creating the virtual environment

python3 -m venv .venv

• Activating the virtual environment

 source .venv/bin/activate

• Installing the dependencies

pip install pip setuptools wheel && pip install -r requirements.txt

IMPORTANT: Update the .env file with OpenAi's access token, as well as having credits on your OpenAi developer platform https://platform.openai.com/.

• Testing the application

python3 ./app.py

Note : To test the application with other messages just change the text of the voucher atendimento in the app.py file.

References

CARRARO, Fabricio. Artificial Intelligence and ChatgPT: From the Revolution of Generative AI Models to Prompt Engineering. 1. Ed. Sao Paulo: Casa do Code Publisher, 2023.

Python DB API, Duckdb. Available at: https://duckdb.org/docs/api/python/dbapi. Accessed on: 22 Apr. 2024.

What is Rag?, AWS. Available at: https://aws.amazon.com/en/what-i/retieval-ugmented-generation/. Accessed on: 22 Apr. 2024.

Word Embedding: Making the computer understand the meaning of words - an introduction to very important NLP concepts: Embeddings and Word2vec, Medium. Available at: https://medium.com/turing-talks/word-empedding-fing-fo-computer-Nefing-da-das-lavras-92f2745057. Accessed on: 22 Apr. 2024.

langchain_community.vectors.duckdb.duckdb, API Python Langchain. Available at: https://api.python.langchain.com/en/latest/vectors/langchain_community.vectors.duckdb.html. Accessed on: 22 Apr. 2024.

langchain_community.embeddings.openai.openaieemBeddings, OpenIeeembeddings. Available at: https://api.python.langchain.com/en/latest/embeddings/langchain_community.embeddings.openai.openaieembedings.html#langchain-community-openai-oppedings. Accessed on: 22 Apr. 2024.