torchrec下載torchrec源代碼下載

torchrec

Python

v1.0.0

下載

Torchrec

Torchrec是一個構建的Pytorch域庫，旨在提供大規模推薦系統（RECSYS）所需的常見稀疏性和並行性原始圖。 Torchrec可以訓練和推斷模型，這些模型在許多GPU上都碎片，並為Meta的許多生產Recsys模型提供動力。

外部存在

Torchrec已用於加速推薦系統的進步，一些例子：

Meta的DLRM（深度學習推薦模型）的最新版本是使用Torchrec構建的
分解多型：有效的大規模推薦紙的拓撲感知建模技術
Twitter的算法ML
培訓推薦模型使用數據助理
使用嵌入卸載紙的100TB型號

介紹

要開始學習Torchrec，請查看：

我們完整的Torchrec教程
Torchrec文檔概述了Torchrec和API參考

Torchrec功能

使用混合數據並行性/模型並行性的大型性能多設備/多節點模型，可以輕鬆地創作大型，性能的多設備/多節點模型。
碎片到碎片嵌入表具有不同策略的片段，包括數據並行，桌面，行，桌面，列，列和桌子柱的碎片。
可以自動為模型生成優化的碎片計劃的計劃者。
管道訓練重疊的數據加載設備傳輸（副本到GPU），設備間通信（Input_dist）和計算（向後，向後）以提高性能。
由FBGEMM提供動力的Recsys的優化內核。
量化對降低精度訓練和推斷的支持，並優化了用於C ++推斷的Torchrec模型。
Recsys的常見模塊。
recsys數據集（Criteo單擊日誌和Movielens）
端到端培訓的示例此類DLRM事件預測模型在Criteo單擊日誌數據集上進行了訓練。

安裝

查看文檔中的“入門”部分，以獲取推薦的設置Torchrec的方法。

來自來源

通常，無需從源頭構建。對於大多數用例，請按照上面的部分設置Torchrec。但是，要從源頭構建並獲得最新更改，請執行以下操作：

安裝Pytorch。請參閱Pytorch文檔。

 CUDA 12.4

pip install torch --index-url https://download.pytorch.org/whl/nightly/cu124

CUDA 12.1

pip install torch --index-url https://download.pytorch.org/whl/nightly/cu121

CUDA 11.8

pip install torch --index-url https://download.pytorch.org/whl/nightly/cu118

CPU

pip install torch --index-url https://download.pytorch.org/whl/nightly/cpu

克隆Torchrec。

 git clone --recursive https://github.com/pytorch/torchrec
cd torchrec

安裝FBGEMM。

 CUDA 12.4

pip install fbgemm-gpu --index-url https://download.pytorch.org/whl/nightly/cu124

CUDA 12.1

pip install fbgemm-gpu --index-url https://download.pytorch.org/whl/nightly/cu121

CUDA 11.8

pip install fbgemm-gpu --index-url https://download.pytorch.org/whl/nightly/cu118

CPU

pip install fbgemm-gpu --index-url https://download.pytorch.org/whl/nightly/cpu

安裝其他要求。
```
 pip install -r requirements.txt
```
安裝Torchrec。
```
 python setup.py install develop
```

測試安裝（使用torchx-oftly用於3.11;對於3.12，torchx當前不起作用）。

 GPU mode

torchx run -s local_cwd dist.ddp -j 1x2 --gpu 2 --script test_installation.py

CPU Mode

torchx run -s local_cwd dist.ddp -j 1x2 --script test_installation.py -- --cpu_only

有關啟動分佈式和遠程作業的更多信息，請參見Torchx。

如果您想運行一個更複雜的示例，請查看Torchrec DLRM示例。

貢獻

有關為Torchrec貢獻的詳細信息，請參見貢獻。

引用

如果您使用的是Torchrec，請參閱Bibtex條目以引用此工作：

 @inproceedings{10.1145/3523227.3547387,
author = {Ivchenko, Dmytro and Van Der Staay, Dennis and Taylor, Colin and Liu, Xing and Feng, Will and Kindi, Rahul and Sudarshan, Anirudh and Sefati, Shahin},
title = {TorchRec: a PyTorch Domain Library for Recommendation Systems},
year = {2022},
isbn = {9781450392785},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3523227.3547387},
doi = {10.1145/3523227.3547387},
abstract = {Recommendation Systems (RecSys) comprise a large footprint of production-deployed AI today. The neural network-based recommender systems differ from deep learning models in other domains in using high-cardinality categorical sparse features that require large embedding tables to be trained. In this talk we introduce TorchRec, a PyTorch domain library for Recommendation Systems. This new library provides common sparsity and parallelism primitives, enabling researchers to build state-of-the-art personalization models and deploy them in production. In this talk we cover the building blocks of the TorchRec library including modeling primitives such as embedding bags and jagged tensors, optimized recommender system kernels powered by FBGEMM, a flexible sharder that supports a veriety of strategies for partitioning embedding tables, a planner that automatically generates optimized and performant sharding plans, support for GPU inference and common modeling modules for building recommender system models. TorchRec library is currently used to train large-scale recommender models at Meta. We will present how TorchRec helped Meta’s recommender system platform to transition from CPU asynchronous training to accelerator-based full-sync training.},
booktitle = {Proceedings of the 16th ACM Conference on Recommender Systems},
pages = {482–483},
numpages = {2},
keywords = {information retrieval, recommender systems},
location = {Seattle, WA, USA},
series = {RecSys '22}
}