torchrec下载torchrec源代码下载

torchrec

Python

v1.0.0

下载

Torchrec

Torchrec是一个构建的Pytorch域库，旨在提供大规模推荐系统（RECSYS）所需的常见稀疏性和并行性原始图。 Torchrec可以训练和推断模型，这些模型在许多GPU上都碎片，并为Meta的许多生产Recsys模型提供动力。

外部存在

Torchrec已用于加速推荐系统的进步，一些例子：

Meta的DLRM（深度学习推荐模型）的最新版本是使用Torchrec构建的
分解多型：有效的大规模推荐纸的拓扑感知建模技术
Twitter的算法ML
培训推荐模型使用数据助理
使用嵌入卸载纸的100TB型号

介绍

要开始学习Torchrec，请查看：

我们完整的Torchrec教程
Torchrec文档概述了Torchrec和API参考

Torchrec功能

使用混合数据并行性/模型并行性的大型性能多设备/多节点模型，可以轻松地创作大型，性能的多设备/多节点模型。
碎片到碎片嵌入表具有不同策略的片段，包括数据并行，桌面，行，桌面，列，列和桌子柱的碎片。
可以自动为模型生成优化的碎片计划的计划者。
管道训练重叠的数据加载设备传输（副本到GPU），设备间通信（Input_dist）和计算（向后，向后）以提高性能。
由FBGEMM提供动力的Recsys的优化内核。
量化对降低精度训练和推断的支持，并优化了用于C ++推断的Torchrec模型。
Recsys的常见模块。
recsys数据集（Criteo单击日志和Movielens）
端到端培训的示例此类DLRM事件预测模型在Criteo单击日志数据集上进行了训练。

安装

查看文档中的“入门”部分，以获取推荐的设置Torchrec的方法。

来自来源

通常，无需从源头构建。对于大多数用例，请按照上面的部分设置Torchrec。但是，要从源头构建并获得最新更改，请执行以下操作：

安装Pytorch。请参阅Pytorch文档。

 CUDA 12.4

pip install torch --index-url https://download.pytorch.org/whl/nightly/cu124

CUDA 12.1

pip install torch --index-url https://download.pytorch.org/whl/nightly/cu121

CUDA 11.8

pip install torch --index-url https://download.pytorch.org/whl/nightly/cu118

CPU

pip install torch --index-url https://download.pytorch.org/whl/nightly/cpu

克隆Torchrec。

 git clone --recursive https://github.com/pytorch/torchrec
cd torchrec

安装FBGEMM。

 CUDA 12.4

pip install fbgemm-gpu --index-url https://download.pytorch.org/whl/nightly/cu124

CUDA 12.1

pip install fbgemm-gpu --index-url https://download.pytorch.org/whl/nightly/cu121

CUDA 11.8

pip install fbgemm-gpu --index-url https://download.pytorch.org/whl/nightly/cu118

CPU

pip install fbgemm-gpu --index-url https://download.pytorch.org/whl/nightly/cpu

安装其他要求。
```
 pip install -r requirements.txt
```
安装Torchrec。
```
 python setup.py install develop
```

测试安装（使用torchx-oftly用于3.11;对于3.12，torchx当前不起作用）。

 GPU mode

torchx run -s local_cwd dist.ddp -j 1x2 --gpu 2 --script test_installation.py

CPU Mode

torchx run -s local_cwd dist.ddp -j 1x2 --script test_installation.py -- --cpu_only

有关启动分布式和远程作业的更多信息，请参见Torchx。

如果您想运行一个更复杂的示例，请查看Torchrec DLRM示例。

贡献

有关为Torchrec贡献的详细信息，请参见贡献。

引用

如果您使用的是Torchrec，请参阅Bibtex条目以引用此工作：

 @inproceedings{10.1145/3523227.3547387,
author = {Ivchenko, Dmytro and Van Der Staay, Dennis and Taylor, Colin and Liu, Xing and Feng, Will and Kindi, Rahul and Sudarshan, Anirudh and Sefati, Shahin},
title = {TorchRec: a PyTorch Domain Library for Recommendation Systems},
year = {2022},
isbn = {9781450392785},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3523227.3547387},
doi = {10.1145/3523227.3547387},
abstract = {Recommendation Systems (RecSys) comprise a large footprint of production-deployed AI today. The neural network-based recommender systems differ from deep learning models in other domains in using high-cardinality categorical sparse features that require large embedding tables to be trained. In this talk we introduce TorchRec, a PyTorch domain library for Recommendation Systems. This new library provides common sparsity and parallelism primitives, enabling researchers to build state-of-the-art personalization models and deploy them in production. In this talk we cover the building blocks of the TorchRec library including modeling primitives such as embedding bags and jagged tensors, optimized recommender system kernels powered by FBGEMM, a flexible sharder that supports a veriety of strategies for partitioning embedding tables, a planner that automatically generates optimized and performant sharding plans, support for GPU inference and common modeling modules for building recommender system models. TorchRec library is currently used to train large-scale recommender models at Meta. We will present how TorchRec helped Meta’s recommender system platform to transition from CPU asynchronous training to accelerator-based full-sync training.},
booktitle = {Proceedings of the 16th ACM Conference on Recommender Systems},
pages = {482–483},
numpages = {2},
keywords = {information retrieval, recommender systems},
location = {Seattle, WA, USA},
series = {RecSys '22}
}