MLOps

MLOps

Table of Contents

• AI as a Service: simple AI-based service with serverless (AWS Lambda)

• AWS

  • Lambda

    • invoke sagemaker with aws lambda

  • SageMaker

    • SageMaker HuggingFace toolkits
    • SageMaker serverless inference

• Data Management

  • DVC

• Kubernetes

  • Basic examples
  • Troubleshootings
    • minikube troubleshooting

• LLM

  • large language models
    • document oriented agent with openai api
    • function callings - make LLM to use tools
  • RAG
  • peft
  • fsdp_qlora
  • llama_index
  • TensorLLM
  • prompt engineering
    • leaked-system-prompts: leaked system prompts of actual LLM-based services
    • source codes and notebooks for learning prompt engineering with openai

• ML serving

  • bento-ml
  • build ML pipelines with Nvidia Triton Server
  • serving ML models with fastapi
  • serving ML models with flask
  • ray
    • Serve GPT-2 pipeline with Ray-serve and FastAPI
    • Serve GPT-J-6B model with Ray-AIR

GPU Recommendations