evotorch下載 - evotorch源代碼下載

evotorch

Python

0.5.1

下載

歡迎來到Evotorch項目！ Evotorch是在Nnaisense開發的開源進化計算庫，建於Pytorch之上。有關使用Evotorch的深入指導，請參見文檔，並加入我們的Slack進行討論。

通過安裝evotorch開始：

 pip install evotorch

對於Evotorch，無論它們是否可區分（即允許梯度下降），都可以解決各種優化問題。在可解決的問題類型中，Evotorch是：

黑盒優化問題（連續或離散）
強化學習任務
監督學習任務

各種進化計算算法在Evotorch中提供：

基於分佈的搜索算法：
- PGPE：具有基於參數的探索的政策梯度。
- XNES：指數自然進化策略。
- CMA-ES：協方差矩陣適應進化策略。
- SNES：可分離的自然進化策略。
- CEM：交叉熵方法。
基於人群的搜索算法：
- GenityGorithm：一種遺傳算法實施。還支持多個目標，在這種情況下，它的行為類似於NSGA-II 。
- Cosyne：合作突觸神經進化。
- Mapelites：表型精英的多維檔案

由於所有這些算法都是在Pytorch中實現的，因此它們受益於在GPU上使用矢量化和並行化，因此在可用GPU時會大大加快優化。使用射線，通過將工作量跨越：

多個CPU
多個GPU
射線集群中的多台計算機

例子

以下是一些代碼示例，這些示例證明了Evotorch的API。

黑盒優化示例

與Pytorch一起使用的任何目標函數都可以直接與Evotorch一起使用。非矢量化的目標函數只需接收解決方案作為1維型火炬張量，然後返回適應性作為標量。矢量化的目標函數接收一批溶液作為二維火炬張量，並返回適應性的一維張量。以下示例演示瞭如何定義和解決經典的rastrigin問題。

 from evotorch import Problem
from evotorch . algorithms import SNES
from evotorch . logging import StdOutLogger , PandasLogger
import math
import matplotlib . pyplot as plt
import torch


# Declare the objective function
def rastrigin ( x : torch . Tensor ) -> torch . Tensor :
    A = 10
    ( _ , n ) = x . shape
    return A * n + torch . sum (( x ** 2 ) - A * torch . cos ( 2 * math . pi * x ), 1 )


# Declare the problem
problem = Problem (
    "min" ,
    rastrigin ,
    initial_bounds = ( - 5.12 , 5.12 ),
    solution_length = 100 ,
    vectorized = True ,
    # device="cuda:0"  # enable this line if you wish to use GPU
)

# Initialize the SNES algorithm to solve the problem
searcher = SNES ( problem , popsize = 1000 , stdev_init = 10.0 )

# Initialize a standard output logger, and a pandas logger
_ = StdOutLogger ( searcher , interval = 10 )
pandas_logger = PandasLogger ( searcher )

# Run SNES for the specified amount of generations
searcher . run ( 2000 )

# Get the progress of the evolution into a DataFrame with the
# help of the PandasLogger, and then plot the progress.
pandas_frame = pandas_logger . to_dataframe ()
pandas_frame [ "best_eval" ]. plot ()
plt . show ()

強化學習示例

以下示例演示瞭如何解決通過健身房圖書館可用的加強學習任務。

 from evotorch . algorithms import PGPE
from evotorch . logging import StdOutLogger , PicklingLogger
from evotorch . neuroevolution import GymNE

# Declare the problem to solve
problem = GymNE (
    env = "Humanoid-v4" ,  # Solve the Humanoid-v4 task
    network = "Linear(obs_length, act_length)" ,  # Linear policy
    observation_normalization = True ,  # Normalize the policy inputs
    decrease_rewards_by = 5.0 ,  # Decrease each reward by 5.0
    num_actors = "max" ,  # Use all available CPUs
    # num_actors=4,    # Explicit setting. Use 4 actors.
)

# Instantiate a PGPE algorithm to solve the problem
searcher = PGPE (
    problem ,
    # Base population size
    popsize = 200 ,
    # For each generation, sample more solutions until the
    # number of simulator interactions reaches this threshold
    num_interactions = int ( 200 * 1000 * 0.75 ),
    # Stop re-sampling solutions if the current population size
    # reaches or exceeds this number.
    popsize_max = 3200 ,
    # Learning rates
    center_learning_rate = 0.0075 ,
    stdev_learning_rate = 0.1 ,
    # Radius of the initial search distribution
    radius_init = 0.27 ,
    # Use the ClipUp optimizer with the specified maximum speed
    optimizer = "clipup" ,
    optimizer_config = { "max_speed" : 0.15 },
)

# Instantiate a standard output logger
_ = StdOutLogger ( searcher )

# Optional: Instantiate a logger to pickle and save the results periodically.
# In this example, among the saved results will be the center of the search
# distribution, since we are using PGPE which is distribution-based.
_ = PicklingLogger ( searcher , interval = 10 )

# Run the algorithm for the specified amount of generations
searcher . run ( 500 )

# Get the center point of the search distribution,
# obtain a policy out of that point, and visualize the
# agent using that policy.
center_solution = searcher . status [ "center" ]
trained_policy = problem . make_net ( center_solution )
problem . visualize ( trained_policy )

可以在此處找到更多示例。

如何引用

如果您在研究中使用Evotorch，請考慮引用我們的論文。

 @article { evotorch2023arxiv ,
  title = { {EvoTorch}: Scalable Evolutionary Computation in {Python} } ,
  author = { Toklu, Nihat Engin and Atkinson, Timothy and Micka, Vojtv{e}ch and Liskowski, Pawel{} and Srivastava, Rupesh Kumar } ,
  journal = { arXiv preprint } ,
  year = { 2023 } ,
  note = { https://arxiv.org/abs/2302.12600 }
}