BigDL 2.x
BigDL release 2.5.0b1
重要的
bigdl-llm現在已成為ipex-llm ，我們的未來開發將轉向IPEX-LLM項目。
BigDL無縫將數據分析和AI應用程序從筆記本電腦到雲擴展，並附有以下庫：
LLM （棄用 - 請改用IPEX -LLM） ：優化的大型語言模型庫，用於Intel CPU和GPU
ORCA：Spark和Ray上的分佈式大數據和AI（TF＆Pytorch）管道
Nano：Intel CPU/GPU上TensorFlow和Pytorch程序的透明加速度
Dllib：“相當於Spark Mllib”，以進行深度學習
計時：使用Automl的可擴展時間序列分析
弗里斯安：端到端推薦系統
PPML：安全大數據和AI（使用SGX/TDX硬件安全性）
有關更多信息，您可以閱讀文檔。
流程圖TD;
功能1 {{{hw固定大數據＆ai？ }}};
功能1-->> feature2 {{{python vs. scala/java？ }}};
功能1--“是” - > codeppml（[[<em> <strong> ppml </strong> </em>]）;
功能2-- python-> feature3 {{{哪種類型的應用程序？ }};
功能2-- scala/java-> referdllib（[<em> <strong> dllib </strong> </em>]）;
功能3--“大語言模型” - > referllm（[[<em> <strong> llm </strong> </em>]）;
功能3--“大數據 + AI（tf/pytorch）” - > referororca（[[<em> <strong> orca </strong> </em>]）;
功能3-加速張量/pytorch-> refernano（[[<em> <strong> nano </strong> </em>]）;
功能3--用於spark mllib-> referdllib2（[<em> <strong> dllib </strong> </em>]））;
功能3--高級應用框架 - > feature4 {{domain？ }};
功能4--時間序列 - > referChronos（[[<em> <strong> chronos </strong> </em>]）;
功能4--推薦系統 - > Referfriesian（[<em> <strong> friesian </strong> </em>]）;
單擊reflellm“ https://github.com/intel-analytics/ipex-llm”
單擊Refernano“ https://github.com/intel-analytics/bigdl-2.x#nano”
單擊Referorca“ https://github.com/intel-analytics/bigdl-2.x#orca”
單擊Refuddllib“ https://github.com/intel-analytics/bigdl-2.x#dllib”
單擊refledlib2“ https://github.com/intel-analytics/bigdl-2.x#dllib”
單擊ReforeChronos“ https://github.com/intel-analytics/bigdl-2.x#chronos”
單擊Referfriesian“ https://github.com/intel-analytics/bigdl-2.x#friesian”
單擊Refleppml“ https://github.com/intel-analytics/bigdl-2.x#ppml”
classDef referstyle1填充：＃5099ce，中風：＃5099ce;
classDef功能填充：＃fff，中風：＃08409c，stroke寬度：1px;
類Referllm，Refernano，Reveororca，Referdllib，Referdllib2，referdrib2，referfriesian，referppml refersTyle1;
類功能1，feature2，feature3，feature4，feature4，feature5，feature6，feature7 feature;
要安裝BIGDL，我們建議使用Conda環境：
conda create -n my_env
conda activate my_env
pip install bigdl要安裝最新的夜間構建，請使用pip install --pre --upgrade bigdl ；有關更多詳細信息，請參見Python和Scala用戶指南。
要安裝每個單獨的庫，例如Chronos，請使用pip install bigdl-chronos ；有關更多詳細信息，請參見文檔網站。
ORCA庫無縫地縮放您的單個節點TensorFlow ， Pytorch或OpenVino程序，跨大型群集（以處理分佈式大數據）。
您可以在4個簡單的步驟中使用ORCA構建端到端，分佈式數據處理和AI程序：
# 1. Initilize Orca Context (to run your program on K8s, YARN or local laptop)
from bigdl . orca import init_orca_context , OrcaContext
sc = init_orca_context ( cluster_mode = "k8s" , cores = 4 , memory = "10g" , num_nodes = 2 )
# 2. Perform distribtued data processing (supporting Spark DataFrames,
# TensorFlow Dataset, PyTorch DataLoader, Ray Dataset, Pandas, Pillow, etc.)
spark = OrcaContext . get_spark_session ()
df = spark . read . parquet ( file_path )
df = df . withColumn ( 'label' , df . label - 1 )
...
# 3. Build deep learning models using standard framework APIs
# (supporting TensorFlow, PyTorch, Keras, OpenVino, etc.)
from tensorflow import keras
...
model = keras . models . Model ( inputs = [ user , item ], outputs = predictions )
model . compile (...)
# 4. Use Orca Estimator for distributed training/inference
from bigdl . orca . learn . tf . estimator import Estimator
est = Estimator . from_keras ( keras_model = model )
est . fit ( data = df ,
feature_cols = [ 'user' , 'item' ],
label_cols = [ 'label' ],
...)有關更多詳細信息，請參見Orca用戶指南以及TensorFlow和Pytorch QuickStarts。
此外，您還可以使用orca中的RayonsPark在Spark群集上運行標準射線程序。
您不僅可以在Spark群集上運行射線程序，還可以使用SPARK CODE（以處理內存Spark RDD或DataFrames）使用RayonsPark在ORCA中編寫射線代碼。
# 1. Initilize Orca Context (to run your program on K8s, YARN or local laptop)
from bigdl . orca import init_orca_context , OrcaContext
sc = init_orca_context ( cluster_mode = "yarn" , cores = 4 , memory = "10g" , num_nodes = 2 , init_ray_on_spark = True )
# 2. Distribtued data processing using Spark
spark = OrcaContext . get_spark_session ()
df = spark . read . parquet ( file_path ). withColumn (...)
# 3. Convert Spark DataFrame to Ray Dataset
from bigdl . orca . data import spark_df_to_ray_dataset
dataset = spark_df_to_ray_dataset ( df )
# 4. Use Ray to operate on Ray Datasets
import ray
@ ray . remote
def consume ( data ) -> int :
num_batches = 0
for batch in data . iter_batches ( batch_size = 10 ):
num_batches += 1
return num_batches
print ( ray . get ( consume . remote ( dataset )))有關更多詳細信息，請參見RayonsPark用戶指南和快速入門。
您可以使用Nano在筆記本電腦或服務器上透明地加速張量或Pytorch程序。隨著最小代碼更改， Nano會自動將現代CPU優化（例如SIMD，多處理，低精度等）應用於標準Tensorflow和Pytorch代碼，並具有最大的速度。
您可以自動優化訓練有素的Pytorch模型，用於使用Nano ：用於推理或部署：
model = ResNet18 (). load_state_dict (...)
train_dataloader = ...
val_dataloader = ...
def accuracy ( pred , target ):
...
from bigdl . nano . pytorch import InferenceOptimizer
optimizer = InferenceOptimizer ()
optimizer . optimize ( model ,
training_data = train_dataloader ,
validation_data = val_dataloader ,
metric = accuracy )
new_model , config = optimizer . get_best_model ()
optimizer . summary () optimizer.summary()將是：
-------------------------------- ---------------------- -------------- ----------------------
| method | status | latency(ms) | metric value |
-------------------------------- ---------------------- -------------- ----------------------
| original | successful | 45.145 | 0.975 |
| bf16 | successful | 27.549 | 0.975 |
| static_int8 | successful | 11.339 | 0.975 |
| jit_fp32_ipex | successful | 40.618 | 0.975* |
| jit_fp32_ipex_channels_last | successful | 19.247 | 0.975* |
| jit_bf16_ipex | successful | 10.149 | 0.975 |
| jit_bf16_ipex_channels_last | successful | 9.782 | 0.975 |
| openvino_fp32 | successful | 22.721 | 0.975* |
| openvino_int8 | successful | 5.846 | 0.962 |
| onnxruntime_fp32 | successful | 20.838 | 0.975* |
| onnxruntime_int8_qlinear | successful | 7.123 | 0.981 |
-------------------------------- ---------------------- -------------- ----------------------
* means we assume the metric value of the traced model does not change, so we don't recompute metric value to save time.
Optimization cost 60.8s in total.
model = ResNet18 ()
optimizer = torch . optim . SGD (...)
train_loader = ...
val_loader = ...
from bigdl . nano . pytorch import TorchNano
# Define your training loop inside `TorchNano.train`
class Trainer ( TorchNano ):
def train ( self ):
# call `setup` to prepare for model, optimizer(s) and dataloader(s) for accelerated training
model , optimizer , ( train_loader , val_loader ) = self . setup ( model , optimizer ,
train_loader , val_loader )
for epoch in range ( num_epochs ):
model . train ()
for data , target in train_loader :
optimizer . zero_grad ()
output = model ( data )
# replace the loss.backward() with self.backward(loss)
loss = loss_fuc ( output , target )
self . backward ( loss )
optimizer . step ()
# Accelerated training (IPEX, BF16 and Multi-Instance Training)
Trainer ( use_ipex = True , precision = 'bf16' , num_processes = 2 ). train ()有關更多詳細信息，請參見Nano用戶指南和Tutotial。
使用DLLIB ，您可以使用相同的Spark DataFrames和ML Pipeline API將分佈式深度學習應用程序作為標準（ Scala或Python ）Spark程序編寫。
您可以在3個簡單的步驟中使用dllib scala apis構建Spark的分佈式深度學習應用程序：
// 1. Call `initNNContext` at the beginning of the code:
import com . intel . analytics . bigdl . dllib . NNContext
val sc = NNContext .initNNContext()
// 2. Define the deep learning model using Keras-style API in DLlib:
import com . intel . analytics . bigdl . dllib . keras . layers . _
import com . intel . analytics . bigdl . dllib . keras . Model
val input = Input [ Float ](inputShape = Shape ( 10 ))
val dense = Dense [ Float ]( 12 ).inputs(input)
val output = Activation [ Float ]( " softmax " ).inputs(dense)
val model = Model (input, output)
// 3. Use `NNEstimator` to train/predict/evaluate the model using Spark DataFrame and ML pipeline APIs
import org . apache . spark . sql . SparkSession
import org . apache . spark . ml . feature . MinMaxScaler
import org . apache . spark . ml . Pipeline
import com . intel . analytics . bigdl . dllib . nnframes . NNEstimator
import com . intel . analytics . bigdl . dllib . nn . CrossEntropyCriterion
import com . intel . analytics . bigdl . dllib . optim . Adam
val spark = SparkSession .builder().getOrCreate()
val trainDF = spark.read.parquet( " train_data " )
val validationDF = spark.read.parquet( " val_data " )
val scaler = new MinMaxScaler ().setInputCol( " in " ).setOutputCol( " value " )
val estimator = NNEstimator (model, CrossEntropyCriterion ())
.setBatchSize( 128 ).setOptimMethod( new Adam ()).setMaxEpoch( 5 )
val pipeline = new Pipeline ().setStages( Array (scaler, estimator))
val pipelineModel = pipeline.fit(trainDF)
val predictions = pipelineModel.transform(validationDF)您可以使用Dllib Python API在3個簡單步驟中構建Spark的分佈式深度學習應用程序：
# 1. Call `init_nncontext` at the beginning of the code:
from bigdl . dllib . nncontext import init_nncontext
sc = init_nncontext ()
# 2. Define the deep learning model using Keras-style API in DLlib:
from bigdl . dllib . keras . layers import Input , Dense , Activation
from bigdl . dllib . keras . models import Model
input = Input ( shape = ( 10 ,))
dense = Dense ( 12 )( input )
output = Activation ( "softmax" )( dense )
model = Model ( input , output )
# 3. Use `NNEstimator` to train/predict/evaluate the model using Spark DataFrame and ML pipeline APIs
from pyspark . sql import SparkSession
from pyspark . ml . feature import MinMaxScaler
from pyspark . ml import Pipeline
from bigdl . dllib . nnframes import NNEstimator
from bigdl . dllib . nn . criterion import CrossEntropyCriterion
from bigdl . dllib . optim . optimizer import Adam
spark = SparkSession . builder . getOrCreate ()
train_df = spark . read . parquet ( "train_data" )
validation_df = spark . read . parquet ( "val_data" )
scaler = MinMaxScaler (). setInputCol ( "in" ). setOutputCol ( "value" )
estimator = NNEstimator ( model , CrossEntropyCriterion ())
. setBatchSize ( 128 )
. setOptimMethod ( Adam ())
. setMaxEpoch ( 5 )
pipeline = Pipeline ( stages = [ scaler , estimator ])
pipelineModel = pipeline . fit ( train_df )
predictions = pipelineModel . transform ( validation_df )有關更多詳細信息，請參見DLLIB NNFRAMES和KERAS API用戶指南。
Chronos庫使快速，準確和可擴展的時間序列分析應用程序（使用AUTOML）易於構建。
您可以在3個簡單的步驟中使用計時訓練時間序列預報器：
from bigdl . chronos . forecaster import TCNForecaster
from bigdl . chronos . data . repo_dataset import get_public_dataset
# 1. Process time series data using `TSDataset`
tsdata_train , tsdata_val , tsdata_test = get_public_dataset ( name = 'nyc_taxi' )
for tsdata in [ tsdata_train , tsdata_val , tsdata_test ]:
data . roll ( lookback = 100 , horizon = 1 )
# 2. Create a `TCNForecaster` (automatically configured based on train_data)
forecaster = TCNForecaster . from_tsdataset ( train_data )
# 3. Train the forecaster for prediction
forecaster . fit ( train_data )
pred = forecaster . predict ( test_data )要應用Automl，請使用AutoTSEstimator代替普通預測器。
# Create and fit an `AutoTSEstimator`
from bigdl . chronos . autots import AutoTSEstimator
autotsest = AutoTSEstimator ( model = "tcn" , future_seq_len = 10 )
tsppl = autotsest . fit ( data = tsdata_train , validation_data = tsdata_val )
pred = tsppl . predict ( tsdata_test )有關更多詳細信息，請參見Chronos用戶指南和快速啟動。
弗里斯圖書館（ Friesian Library）使建造端到端的大規模推薦系統（包括脫機功能轉換和轉移，近線功能和模型更新以及在線服務管道）變得易於構建。
有關更多詳細信息，請參見Freisian Readme。
BIGDL PPML提供了一個硬件（Intel SGX）受保護的受信任集群環境，用於運行分佈式大數據和AI應用程序（以私有或公共雲上的安全方式）。
有關更多詳細信息，請參見PPML用戶指南和教程。
如果您發現BIGDL對您的項目有用，則可以引用我們的論文如下：
@INPROCEEDINGS{9880257,
title={BigDL 2.0: Seamless Scaling of AI Pipelines from Laptops to Distributed Cluster},
author={Dai, Jason Jinquan and Ding, Ding and Shi, Dongjie and Huang, Shengsheng and Wang, Jiao and Qiu, Xin and Huang, Kai and Song, Guoqiong and Wang, Yang and Gong, Qiyuan and Song, Jiaming and Yu, Shan and Zheng, Le and Chen, Yina and Deng, Junwei and Song, Ge},
booktitle={2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2022},
pages={21407-21414},
doi={10.1109/CVPR52688.2022.02076}
}
@INPROCEEDINGS{10.1145/3357223.3362707,
title = {BigDL: A Distributed Deep Learning Framework for Big Data},
author = {Dai, Jason Jinquan and Wang, Yiheng and Qiu, Xin and Ding, Ding and Zhang, Yao and Wang, Yanzhang and Jia, Xianyan and Zhang, Cherry Li and Wan, Yan and Li, Zhichao and Wang, Jiao and Huang, Shengsheng and Wu, Zhongyuan and Wang, Yang and Yang, Yuhao and She, Bowen and Shi, Dongjie and Lu, Qi and Huang, Kai and Song, Guoqiong},
booktitle = {Proceedings of the ACM Symposium on Cloud Computing (SoCC)},
year = {2019},
pages = {50–60},
doi = {10.1145/3357223.3362707}
}
