Vector_DB_with_LLM

vector_db_with_llm

vector_db_with_llm

FastApi是一个基于标准Python的现代化，快速（高性能），用于使用Python 3.8+构建API的Web框架。这是一个存储库，可以将记录传递给Prometheus-Export应用程序。

Langchain（https://github.com/langchain-ai/langchain）是开发由大语言模型（LLMS）提供支持的应用程序的框架。 Langchain是基于大语言模型（LLMS）构建应用程序的开源框架。 LLM是大量的深度学习模型，可以在大量数据上进行对用户查询的响应的大量数据。例如，回答问题或从基于文本的提示中创建图像。提示是人们用LLM寻求回复的疑问。

• 示例：https：//colab.research.google.com/github/i-am-shuan/learn-langchain/blob/main/main/langchain_rag_rag_example.ipynb， https://velog.io/
• 上
• ​https://hackernoon.com/lang/ko/ollama-python-%EB%B0%8FCHROMDB%EB%A5%BC- %B%EC%82%AC%EC%9A ％A9％ED％95％98％EC％97％AC-％EB％A1％9c％EC％AC％AC％EC％97％97％90％EC％84％9C-LLM％EC％9D％9D％84-％EC％84％A4％A4％EC％a 0％95％ED％95％98％EB％8A％94-％EB％8B％A8％EA％B3％B3％84％EB％B3％B3％84-％EA％B0％B0％80％EC％EC％9D％B4％B4％EB％93％9C％9C％9C％EB％a1％a1％9 C-％EB％82％98％EB％A7％8C％EC％9D％9D％98-％EB％9E％98％EA％B7％B7％B8-％EC％95％B1-％EB％A7％A7％A7％8C％EB％EB％93％A4％A4％A4％EA％EA％B8％B0
• 本地LLM：https：//github.com/firstpersoncode/local-rag?

ref=hackernoon.com vectordb（即，Milvus，Faiss（Facebook AI相似性搜索），Chroma，Qdrant，Pinecone，Pinecone，Pinecone）：vector Database（vectordb）是为vector database（vectordb）设计的，用于存储和管理Vector Depplication和Machine和Machine，Ai I In Machine和AI。向量数据是指对象的数值表示，可用于相似性搜索，聚类和其他任务。 （参考：https：//krishna-yogik.medium.com/vectordb-tutorial-a-beginners-guide-06dc333333fac2f）

• vectordb
  • 建议系统的用例系统：通过基于其矢量表示的类似用户或项目来为个性化建议提供动力。 （示例：电影，音乐或产品建议）
  • 图像和视频检索：通过搜索类似的视觉功能来启用基于内容的图像和视频检索。
  • 自然语言处理（NLP）：使用Bert或Word2Vec等模型的向量嵌入来查找语义上相似的文本，文档或查询。 （示例：文档聚类，情感分析，语义搜索）
  • 异常检测：识别高维数据中的异常模式或异常值。 （欺诈检测，网络入侵检测）

抹布（检索增强生成）是一种AI技术，允许公司将其最新和最相关的专有数据直接嵌入其LLM提示中。

• 我们不仅在谈论结构化数据，例如电子表格或关系数据库。
• 我们的意思是检索所有可用数据，包括非结构化数据：电子邮件，PDF，聊天日志，社交媒体帖子和其他类型的信息，这些信息可能会导致更好的AI输出。
• 简而言之，RAG可以帮助公司从各种内部来源检索并使用其数据，以获得更好的生成AI结果。
• 为了达到这种提高的精度，RAG与专门类型的数据库（称为矢量数据库）一起工作，以数字格式存储数据，该数据对AI有意义，并在提示时检索它。
• 如果没有矢量数据库来完成其工作，则RAG无法完成工作。
• 步骤（https://blog.naver.com/htk1019/223442628204）：文档加载程序⮕文本splitter⮕文本分离器⮕嵌入vector Store（保存向量）⮕逆转text restiver
• [1]文本块 - 将数据分成小块。
• [2]嵌入转换 - 使用嵌入模型将每个文本块转换为嵌入。
• [3]向量存储 - 将数据嵌入在矢量数据库中。
• [4]文本映射 - 保存文本块及其相应的嵌入，以备将来使用。
• [5]查询嵌入 - 使用前面使用的相同嵌入模型将用户的查询转换为嵌入。
• [6]数据库查询 - 使用查询嵌入从矢量数据库中检索相关数据嵌入。
• [7]相似性搜索 - 确定最相似的文本块，执行大约最近的邻居（ANN）搜索。
• [8]上下文映射 - 将检索的向量映射到相应的文本块。
• [9] LLM响应 - 将提示中检索到的上下文以及查询与LLM一起获取答案。这样可以确保答案仅依赖于给定的上下文。

Apache Flink：Apache Kafka和Apache Flink是大数据和流处理中的两个强大工具。尽管Kafka以其强大的消息传递系统而闻名，但Flink在实时流处理和分析方面很好。

• Apache Kafka：分布式流媒体平台，用于实时数据流的消息系统，Publish-Subscriber消息传递系统，主题/生产者/消费者
• Apache Flink：分布式流处理框架，流处理和分析，事件驱动/实时流程流程，数据播种/流程/流处理/流处理/窗口/窗口/窗口窗口/窗口窗口/窗口窗口/窗口（https://medium.com/@priyankbhandia_24919/apache-flink-for-for-data-enrichment-6118d48de04，https://yenaworldblog.wordpress.wordpress.com/2018/06/06/06/20/20/flink-install-ecc--install-cec------ulin-cec- ec- ec- ec- ec-umpis-umpis-umpin-98/d98/ https://medium.com/@aaron.cpsh/designing-streaming-data-pipeline-with-pipeline-with-apache-flink-and-kafka-a75c75c75d55918）apache
  • flink是一个框架和分布式处理引擎，用于对有限和没有绑定的数据流进行状态计算。可以将有界的数据流视为批处理处理，而无界的数据流可以被认为是流处理，因为常规讨论的
  • Pyflink是Apache Flink的Python API。它允许用户在Python中编写Flink程序，并在Flink群集上执行它们。
  • 可以从各种来源检索来源和接收器：
    • 文件系统（源/接收器）
    • apache kafka（source/sink）
    • apache cassandra（sink）
    • amazon kinesis streams（source/sink
    • ）elasticsearch（sink）
    • hadoop filesystem（sink
    ）
    • hadoop filesystem（
    • sink）rabbitmq（
    sink
    • ）rabbitmq
    • （source/sink sink）apache nifi（sink）twitter api
    • （sink
  • ） Flink : https://thingsolver.com/blog/streaming-analytics-in-banking-how-to-start-with-apache-flink-and-kafka-in-7-steps/, https://github.com/apache/flink/blob/master/flink-python/pyflink/examples/datastream/connectors/elasticsearch.py​​, https://github.com/apache/flink/blob/master/flink-python/pyflink/exampleas/datastream/datastream/datastream/connectors/kafka_json_jon_format.py

slealllit是一个开放式python python python forthon

• https://github.com/streamlit/llm-examples，https://pypi.org/project/project/streamlit-chat/
• shartlit install＆run

pip install streamlit pip install streamlit-chat streamlit run [streamlit-filenam.py] [--server.port 30001] streamlit run app.py

• App测试：https：//github.com/streamlit/llm--examples/blob/main/.github/workflows/app-testing.yml
• sumpllians unignts-dev.txt

black==23.3.0 mypy==1.4.1 pre-commit==3.3.3 watchdog pytest

Gradio（https://www.gradio.app/guides/quickstart）：Gradio是一个开放式Python软件包，可让您快速为机器学习模型，API或任何任意的Python功能构建演示或Web应用程序。然后，您可以在短短几秒钟内使用Gradio的内置共享功能共享指向演示或Web应用程序的链接。

• Run : ./gradio-start.sh
• Installation : We recommend installing Gradio using pip, which is included by default in Python ( pip install --upgrade gradio )

Elasticsearch Gen-AI : https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/generative-ai/chatbot.ipynb

• Codespaces （https://github.com/features/codespaces，https://velog.io/@profile_exe/github-codespaces）：github codespaces可以使您更快地使用GitHub的原始配置，安全的，安全的云开发环境。

• Pyautogui让您的Python脚本控制鼠标和键盘，以自动与其他应用程序进行交互。 API设计为简单。 Pyautogui在Windows，MacOS和Linux上工作，并在Python 2和3上运行。要与PIP安装，运行pip install pyautogui 。有关更多详细信息（https://pyautogui.readthedocs.io/en/latest/），请参见安装页面

import pyautogui import time while True: print(pyautogui.position ()) pyautogui.moveTo(100,200) pyautogui.click(100, 200) # pyautogui.moveTo(200,200, duration=0.5) time.sleep(10)

• ./jupyter-notebook.sh
  • ​
  • ​
  • -L可用主题：Chesterish 3级GruvBoxd gruvboxl monokai oceans16 oneDork solarizedd solarizeD（.venv）

python v3.9安装

• gunicorn/fastapi：https：//chaechae..life/blife/blog/bastapi/fastapi-deployment-deployment-deployment-gnoicorn
• #google_voogle_vignette p.1111111111111111111111111 （https://datamoney.tistory.com/378）sudo

sudo yum install gcc openssl-devel bzip2-devel libffi-devel zlib-devel git wget https://www.python.org/ftp/python/3.9.0/Python-3.9.0.tgz tar –zxvf Python-3.9.0.tgz or tar -xvf Python-3.9.0.tgz cd Python-3.9.0 ./configure --libdir=/usr/lib64 sudo make sudo make altinstall # python3 -m venv .venv --without-pip sudo yum install python3-pip sudo ln -s /usr/lib64/python3.9/lib-dynload/ /usr/local/lib/python3.9/lib-dynload # -- From Python ^3.10 , It need to be installed openssl # openssl cd /usr/local/src wget https://www.openssl.org/source/openssl-1.1.1t.tar.gz tar xvf openssl-1.1.1t.tar.gz cd openssl-1.1.1t/ ./config --prefix=/usr/local/ssl --openssldir=/usr/local/ssl shared zlib make sudo make install export LDFLAGS= " -L/usr/local/ssl/lib " export CPPFLAGS= " -I/usr/local/ssl/include " # openssl확인 /usr/local/ssl/bin/openssl version export LD_LIBRARY_PATH=/usr/local/ssl/lib: $LD_LIBRARY_PATH echo $LD_LIBRARY_PATH sudo yum install gcc openssl-devel bzip2-devel libffi-devel zlib-devel git wget https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tgz tar –zxvf Python-3.11.0.tgz or tar -xvf Python-3.11.0.tgz cd Python-3.11.0 # --with-openssl-rpath=auto 옵션을 추가하여 파이썬이 자동으로 올바른 OpenSSL 라이브러리 경로를 찾도록 함 # ./configure --libdir=/usr/lib64 --with-openssl=/usr/local/ssl --with-openssl-rpath=auto ./configure --libdir=/usr/lib64 --with-openssl=/usr/bin/ssl --with-openssl-rpath=auto sudo make sudo make altinstall # -- Error occurs when installing packages via pip like below (.venv) -bash-4.2$ pip install elasticsearch==7.13 WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available. WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by ' SSLError("Can ' t connect to HTTPS URL because the SSL module is not available. " )': /simple/elasticsearch/ WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError( " Can ' t connect to HTTPS URL because the SSL module is not available.") ' : /simple/elasticsearch/ ERROR: Operation cancelled by user # -- python -m venv .venv source .venv/bin/activate # -- Swagger pip install poetry poetry add fastapi poetry add uvicorn poetry add gunicorn poetry add pytz poetry add httpx poetry add pytest poetry add pytest-cov poetry add requests poetry add pyyaml poetry add elasticsearch==7.13 poetry add python-dotenv poetry add jupyter # -- # -- Vector poetry config virtualenvs.in-project true pip install poetry poetry init poetry add openai langchain langchainhub tiktoken chromadb langchain-community bs4 python-dotenv poetry add sentence-transformers poetry add pypdf poetry add docx2txt poetry add faiss-cpu poetry add requests pip install --q openai langchain langchainhub tiktoken chromadb langchain-community bs4 # when error occur like this # ImportError: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'OpenSSL 1.0.2k-fips 26 Jan 2017'. See: https://github.com/urllib3/urllib3/issues/2168 pip install urllib3==1.26.18 pip install pytz pip install requests==2.27.1

API Run

• Start service using ./service-start.sh
• FYI) Install library with setup.py like pip install git+https://github.com/jm/git_pip_install.git or git+https://oss.navercorp.com/nsml/nsml_notebook.git@branch_name （https://liferecorde.tistory.com/49，https://newsight.tistory.com/296）
• 使用langchain.document_loaders提取带有tika（ ./Langchain/workflow/ workflow/）

curl -X ' POST '  ' http://localhost:7001/vector/uploadfile '  -H ' accept: application/json '  -H ' Content-Type: multipart/form-data '  -F ' [email protected];type=application/msword ' { " filename " : [ { " index " : { " _index " : " test_context " , " _type " : " search " } }, { " ES_UPLOADED " : " JSON_FORMAT " , " CONTENT " : " This is a test Word document for the TemplatePackage example site. " } ] }

• 使用python ./Langchain/workflow/text_loader.py

*** type : < class ' list ' > / len : 1 data : [Document(metadata={'source': 'C:\Users\euiyoung.hwang\Git_Workspace\Vector_DB_with_LLM/Data/Sample.hwp'}, page_content='KTX 노선도ﺎĀ Ā※ KTX 소요 시간과 운임은 철도청의 사정에 따라 변동할 수 있습니다.Ā서울용산광명Ā천안아산대전서대전동대구익산논산김제밀양정읍구포장성광주부산광주송정나주목포ĀྠĀ 경부선 ྠĀ 호남선ĀĀKTX 소요시간호남선 (서울~천안아산 경부선과 동일)서울서대전논산익산김제정읍장성광주광주송정나주목포시:분6:226:517:15-7:38--8:06-8:38경부선서울용산광명천안아산대전동대구밀양구포부산시:분5:305:456:246:307:187:498:148:27KTX 운임안내최저운임8,100원 (월~목 요금제 기준)호남선 (단위:원)행신8,1008,10015,10024,50026,40028,80031,20032,80035,00037,10039,00040,40044,80040,000용산8,10013,30022,70024,70027,20029,60031,20033,40036,30037,50038,90043,30038,400광명11,30020,70022,60025,10028,60029,40031,60034,50036,40037,10041,50036,700천안아산9,40011,40014,00017,70019,10021,70024,70026,60027,70032,20027,300서대전8,1008,1008,30010,00012,70015,90018,10019,70024,30019,300계룡8,1008,1008,10010,70014,00016,10017,70022,50017,400논산8,1008,1008,10011,40013,50015,10020,20014,800익산8,1008,1008,1009,80011,40016,50011,100김제8,1008,1008,1009,70014,8009,300정읍8,1008,1008,10012,1008,100장성8,1008,1008,9008,100광주송정8,1008,100-나주8,100-목포-광주경부선 (단위:원)행신8,1008,10014,10022,80039,70044,20047,80049,100서울8,10012,70021,40038,40043,00046,60047,900광명10,50019,20036,50041,00044,60046,000천안아산8,70025,60029,40032,90034,200대전16,90022,10025,30026,700동대구8,1009,30010,800밀양8,1008,100구포8,100부산')] page_content : KTX 노선도ﺎĀ Ā※ KTX 소요 시간과 운임은 철도청의 사정에 따라 변동할 수 있습니다.Ā서울용산광명Ā천안아산대전서대전동대구익산논산김제밀양정읍구포장성광주부산광주송정나주목포ĀྠĀ 경부선 ྠĀ 호남선ĀĀKTX 소요시간호남선 (서울~천안아산 경부선과 동일)서울서대전논산익산김제정읍장성광주광주송정나주목포시:분6:226:517:15-7:38--8:06-8:38경부선서울용산광명천안아산대전동대구밀양구포부산시:분5:305:456:246:307:187:498:148:27KTX 운임안내최저운임8,100원 (월~목 요금제 기준)호남선 (단위:원)행신8,1008,10015,10024,50026,40028,80031,20032,80035,00037,10039,00040,40044,80040,000용산8,10013,30022,70024,70027,20029,60031,20033,40036,30037,50038,90043,30038,400광명11,30020,70022,60025,10028,60029,40031,60034,50036,40037,10041,50036,700천안아산9,40011,40014,00017,70019,10021,70024,70026,60027,70032,20027,300서대전8,1008,1008,30010,00012,70015,90018,10019,70024,30019,300계룡8,1008,1008,10010,70014,00016,10017,70022,50017,400논산8,1008,1008,10011,40013,50015,10020,20014,800익산8,1008,1008,1009,80011,40016,50011,100김제8,1008,1008,1009,70014,8009,300정읍8,1008,1008,10012,1008,100장성8,1008,1008,9008,100광주송정8,1008,100-나주8,100-목포-광주경부선 (단위:원)행신8,1008,10014,10022,80039,70044,20047,80049,100서울8,10012,70021,40038,40043,00046,60047,900광명10,50019,20036,50041,00044,60046,000천안아산8,70025,60029,40032,90034,200대전16,90022,10025,30026,700동대구8,1009,30010,800밀양8,1008,100구포8,100부산 *** [ { " index " : { " _index " : " test_context " , " _type " : " search " } }, { " ES_UPLOADED " : " JSON_FORMAT " , "CONTENT": "KTX 노선도ﺎĀ Ā※ KTX 소요 시간과 운임은 철도청의 사정에 따라 변동할 수 있습니다.Ā서울용산광명Ā천안아산대전서대전동대구익산논산김제밀양정읍구포장성광주부산광주송정나주목포ĀྠĀ 경부선 ྠĀ 호남선ĀĀKTX 소요시간호남선 (서울~천안아산 경부선과 동일)서울서대전논산익산김제정읍장성광주광주송정나주목포시:분6:226:517:15-7:38--8:06-8:38경부선서울용산광명천안아산대전동대구밀양구포부산시:분5:305:456:246:307:187:498:148:27KTX 운임안내최저운임8,100원 (월~목 요금제 기준)호남선 (단위:원)행신8,1008,10015,10024,50026,40028,80031,20032,80035,00037,10039,00040,40044,80040,000용산8,10013,30022,70024,70027,20029,60031,20033,40036,30037,50038,90043,30038,400광명11,30020,70022,60025,10028,60029,40031,60034,50036,40037,10041,50036,700천안아산9,40011,40014,00017,70019,10021,70024,70026,60027,70032,20027,300서대전8,1008,1008,30010,00012,70015,90018,10019,70024,30019,300계룡8,1008,1008,10010,70014,00016,10017,70022,50017,400논산8,1008,1008,10011,40013,50015,10020,20014,800익산8,1008,1008,1009,80011,40016,50011,100김제8,1008,1008,1009,70014,8009,300정읍8,1008,1008,10012,1008,100장성8,1008,1008,9008,100광주송정8,1008,100-나주8,100-목포-광주경부선 (단위:원)행신8,1008,10014,10022,80039,70044,20047,80049,100서울8,10012,70021,40038,40043,00046,60047,900광명10,50019,20036,50041,00044,60046,000천안아산8,70025,60029,40032,90034,200대전16,90022,10025,30026,700동대구8,1009,30010,800밀양8,1008,100구포8,100부산" } ]

启动并提取文本。

*** type : < class ' list ' > / len : 1 data : [Document(metadata={'source': 'C:\Users\euiyoung.hwang\Git_Workspace\Vector_DB_with_LLM/Data/Sample.hwp'}, page_content='KTX 노선도ﺎĀ Ā※ KTX 소요 시간과 운임은 철도청의 사정에 따라 변동할 수 있습니다.Ā서울용산광명Ā천안아산대전서대전동대구익산논산김제밀양정읍구포장성광주부산광주송정나주목포ĀྠĀ 경부선 ྠĀ 호남선ĀĀKTX 소요시간호남선 (서울~천안아산 경부선과 동일)서울서대전논산익산김제정읍장성광주광주송정나주목포시:분6:226:517:15-7:38--8:06-8:38경부선서울용산광명천안아산대전동대구밀양구포부산시:분5:305:456:246:307:187:498:148:27KTX 운임안내최저운임8,100원 (월~목 요금제 기준)호남선 (단위:원)행신8,1008,10015,10024,50026,40028,80031,20032,80035,00037,10039,00040,40044,80040,000용산8,10013,30022,70024,70027,20029,60031,20033,40036,30037,50038,90043,30038,400광명11,30020,70022,60025,10028,60029,40031,60034,50036,40037,10041,50036,700천안아산9,40011,40014,00017,70019,10021,70024,70026,60027,70032,20027,300서대전8,1008,1008,30010,00012,70015,90018,10019,70024,30019,300계룡8,1008,1008,10010,70014,00016,10017,70022,50017,400논산8,1008,1008,10011,40013,50015,10020,20014,800익산8,1008,1008,1009,80011,40016,50011,100김제8,1008,1008,1009,70014,8009,300정읍8,1008,1008,10012,1008,100장성8,1008,1008,9008,100광주송정8,1008,100-나주8,100-목포-광주경부선 (단위:원)행신8,1008,10014,10022,80039,70044,20047,80049,100서울8,10012,70021,40038,40043,00046,60047,900광명10,50019,20036,50041,00044,60046,000천안아산8,70025,60029,40032,90034,200대전16,90022,10025,30026,700동대구8,1009,30010,800밀양8,1008,100구포8,100부산')] page_content : KTX 노선도ﺎĀ Ā※ KTX 소요 시간과 운임은 철도청의 사정에 따라 변동할 수 있습니다.Ā서울용산광명Ā천안아산대전서대전동대구익산논산김제밀양정읍구포장성광주부산광주송정나주목포ĀྠĀ 경부선 ྠĀ 호남선ĀĀKTX 소요시간호남선 (서울~천안아산 경부선과 동일)서울서대전논산익산김제정읍장성광주광주송정나주목포시:분6:226:517:15-7:38--8:06-8:38경부선서울용산광명천안아산대전동대구밀양구포부산시:분5:305:456:246:307:187:498:148:27KTX 운임안내최저운임8,100원 (월~목 요금제 기준)호남선 (단위:원)행신8,1008,10015,10024,50026,40028,80031,20032,80035,00037,10039,00040,40044,80040,000용산8,10013,30022,70024,70027,20029,60031,20033,40036,30037,50038,90043,30038,400광명11,30020,70022,60025,10028,60029,40031,60034,50036,40037,10041,50036,700천안아산9,40011,40014,00017,70019,10021,70024,70026,60027,70032,20027,300서대전8,1008,1008,30010,00012,70015,90018,10019,70024,30019,300계룡8,1008,1008,10010,70014,00016,10017,70022,50017,400논산8,1008,1008,10011,40013,50015,10020,20014,800익산8,1008,1008,1009,80011,40016,50011,100김제8,1008,1008,1009,70014,8009,300정읍8,1008,1008,10012,1008,100장성8,1008,1008,9008,100광주송정8,1008,100-나주8,100-목포-광주경부선 (단위:원)행신8,1008,10014,10022,80039,70044,20047,80049,100서울8,10012,70021,40038,40043,00046,60047,900광명10,50019,20036,50041,00044,60046,000천안아산8,70025,60029,40032,90034,200대전16,90022,10025,30026,700동대구8,1009,30010,800밀양8,1008,100구포8,100부산 *** [ { " index " : { " _index " : " test_context " , " _type " : " search " } }, { " ES_UPLOADED " : " JSON_FORMAT " , "CONTENT": "KTX 노선도ﺎĀ Ā※ KTX 소요 시간과 운임은 철도청의 사정에 따라 변동할 수 있습니다.Ā서울용산광명Ā천안아산대전서대전동대구익산논산김제밀양정읍구포장성광주부산광주송정나주목포ĀྠĀ 경부선 ྠĀ 호남선ĀĀKTX 소요시간호남선 (서울~천안아산 경부선과 동일)서울서대전논산익산김제정읍장성광주광주송정나주목포시:분6:226:517:15-7:38--8:06-8:38경부선서울용산광명천안아산대전동대구밀양구포부산시:분5:305:456:246:307:187:498:148:27KTX 운임안내최저운임8,100원 (월~목 요금제 기준)호남선 (단위:원)행신8,1008,10015,10024,50026,40028,80031,20032,80035,00037,10039,00040,40044,80040,000용산8,10013,30022,70024,70027,20029,60031,20033,40036,30037,50038,90043,30038,400광명11,30020,70022,60025,10028,60029,40031,60034,50036,40037,10041,50036,700천안아산9,40011,40014,00017,70019,10021,70024,70026,60027,70032,20027,300서대전8,1008,1008,30010,00012,70015,90018,10019,70024,30019,300계룡8,1008,1008,10010,70014,00016,10017,70022,50017,400논산8,1008,1008,10011,40013,50015,10020,20014,800익산8,1008,1008,1009,80011,40016,50011,100김제8,1008,1008,1009,70014,8009,300정읍8,1008,1008,10012,1008,100장성8,1008,1008,9008,100광주송정8,1008,100-나주8,100-목포-광주경부선 (단위:원)행신8,1008,10014,10022,80039,70044,20047,80049,100서울8,10012,70021,40038,40043,00046,60047,900광명10,50019,20036,50041,00044,60046,000천안아산8,70025,60029,40032,90034,200대전16,90022,10025,30026,700동대구8,1009,30010,800밀양8,1008,100구포8,100부산" } ]

• REST API URL： http://localhost:7001/docs

Pytest

• Go to virtual enviroment using source .venv/bin/activate
• Run this command manually: poetry run py.test -v --junitxml=test-reports/junit/pytest.xml --cov-report html --cov tests/ or ./pytest.sh

$ ./pytest.sh tests t est_api.py::test_skip SKIPPED (no way of currently testing this) [ 50%] tests t est_api.py::test_api PASSED [100%] ---------- coverage: platform win32, python 3.11.7-final-0 ----------- Name Stmts Miss Cover Missing ---------------------------------------------------------------- config c onfig.py 8 4 50% 16-20, 33 config l og_config.py 32 1 97% 42 controller _ _init__.py 0 0 100% controller c luster_controller.py 14 0 100% injector.py 25 0 100% main.py 23 9 61% 38-57, 69 service _ _init__.py 0 0 100% service e s_search_handler.py 139 103 26% 32-88, 131-132, 139-155, 160-173, 178-191, 196-213, 219-254, 259-271, 276-287, 292-304, 309-315 service e s_util.py 21 16 24% 5-11, 16-19, 24-35, 40-41 service q uery_builder.py 42 27 36% 21-40, 48-51, 64-83, 89-98, 102-137 service s tatus_handler.py 13 2 85% 12, 19 tests _ _init__.py 0 0 100% tests c onftest.py 8 0 100% tests t est_api.py 9 1 89% 7 ---------------------------------------------------------------- TOTAL 334 163 51% $

CI/CD环境

• Circleci（ ./circleci/config.yml config.yml）：CircleCi是一个连续的集成和连续的交付平台，可帮助软件团队更加聪明，更快地工作。借助CircleCi，每个提交都会在我们的平台中启动一项新工作，并且代码将被构建，测试和部署。
• github操作（ ./.github/workflows/build-and-test.yml Action是一个连续的集成和连续交付（CI/CD）平台（CI/CD）平台，使您可以自动化构建，测试和部署管道。您可以创建工作流程，以构建和测试向存储库的每个拉请请求，或将合并后的拉请请求部署到生产中。

注册服务

• sudo服务vector_interface_api status/stop/start/start/restart

# -- /etc/systemd/system/vector_interface_api.service [Unit] Description=Swagger ES Service [Service] User=devuser Group=devuser Type=simple ExecStart=/bin/bash /home/devuser/Git_Repo/service-start.sh ExecStop= /usr/bin/killall vector_interface_api [Install] WantedBy=default.target # Service command sudo systemctl daemon-reload sudo systemctl start vector_interface_api.service sudo systemctl status vector_interface_api.service sudo systemctl stop vector_interface_api.service sudo service vector_interface_api status/stop/start