mlc llm

MLC LLM

帶有ML彙編的通用LLM部署引擎

入門|文檔|博客

關於

MLC LLM是用於大型語言模型的機器學習編譯器和高性能部署引擎。該項目的任務是使每個人都可以在每個人的平台上開發，優化和部署AI模型。

	AMD GPU	NVIDIA GPU	蘋果GPU	英特爾GPU
Linux / Win	✅VULKAN，ROCM	✅VULKAN，CUDA	N/A。	✅VULKAN
macos	✅金屬（DGPU）	N/A。	✅金屬	✅金屬（IGPU）
Web瀏覽器	✅webgpu和wasm
iOS / iPados	✅蘋果A系列GPU上的金屬
安卓	✅在Adreno GPU上的OPENCL		Mali GPU上的OPENCL

MLC LLM在MLCengine上編譯並運行代碼 - 上述平台上的統一高性能LLM推理引擎。 MLCENGINE提供通過REST服務器，Python，JavaScript，iOS和Android的OpenAI兼容API，所有這些都得到了我們不斷改進社區的同一引擎和編譯器的支持。

開始

請訪問我們的文檔以開始使用MLC LLM。

• 安裝
• 快速開始
• 介紹

引用

如果您覺得有用，請考慮引用我們的項目：

 @software { mlc-llm ,
    author = { {MLC team} } ,
    title = { {MLC-LLM} } ,
    url = { https://github.com/mlc-ai/mlc-llm } ,
    year = { 2023-2024 }
}

MLC LLM的基礎技術包括：

參考（單擊以展開）

 @inproceedings { tensorir ,
    author = { Feng, Siyuan and Hou, Bohan and Jin, Hongyi and Lin, Wuwei and Shao, Junru and Lai, Ruihang and Ye, Zihao and Zheng, Lianmin and Yu, Cody Hao and Yu, Yong and Chen, Tianqi } ,
    title = { TensorIR: An Abstraction for Automatic Tensorized Program Optimization } ,
    year = { 2023 } ,
    isbn = { 9781450399166 } ,
    publisher = { Association for Computing Machinery } ,
    address = { New York, NY, USA } ,
    url = { https://doi.org/10.1145/3575693.3576933 } ,
    doi = { 10.1145/3575693.3576933 } ,
    booktitle = { Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 } ,
    pages = { 804–817 } ,
    numpages = { 14 } ,
    keywords = { Tensor Computation, Machine Learning Compiler, Deep Neural Network } ,
    location = { Vancouver, BC, Canada } ,
    series = { ASPLOS 2023 }
}

@inproceedings { metaschedule ,
    author = { Shao, Junru and Zhou, Xiyou and Feng, Siyuan and Hou, Bohan and Lai, Ruihang and Jin, Hongyi and Lin, Wuwei and Masuda, Masahiro and Yu, Cody Hao and Chen, Tianqi } ,
    booktitle = { Advances in Neural Information Processing Systems } ,
    editor = { S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh } ,
    pages = { 35783--35796 } ,
    publisher = { Curran Associates, Inc. } ,
    title = { Tensor Program Optimization with Probabilistic Programs } ,
    url = { https://proceedings.neurips.cc/paper_files/paper/2022/file/e894eafae43e68b4c8dfdacf742bcbf3-Paper-Conference.pdf } ,
    volume = { 35 } ,
    year = { 2022 }
}

@inproceedings { tvm ,
    author = { Tianqi Chen and Thierry Moreau and Ziheng Jiang and Lianmin Zheng and Eddie Yan and Haichen Shen and Meghan Cowan and Leyuan Wang and Yuwei Hu and Luis Ceze and Carlos Guestrin and Arvind Krishnamurthy } ,
    title = { {TVM}: An Automated {End-to-End} Optimizing Compiler for Deep Learning } ,
    booktitle = { 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18) } ,
    year = { 2018 } ,
    isbn = { 978-1-939133-08-3 } ,
    address = { Carlsbad, CA } ,
    pages = { 578--594 } ,
    url = { https://www.usenix.org/conference/osdi18/presentation/chen } ,
    publisher = { USENIX Association } ,
    month = oct,
}