evotorch下载 - evotorch源代码下载

evotorch

Python

0.5.1

下载

欢迎来到Evotorch项目！ Evotorch是在Nnaisense开发的开源进化计算库，建于Pytorch之上。有关使用Evotorch的深入指导，请参见文档，并加入我们的Slack进行讨论。

通过安装evotorch开始：

 pip install evotorch

对于Evotorch，无论它们是否可区分（即允许梯度下降），都可以解决各种优化问题。在可解决的问题类型中，Evotorch是：

黑盒优化问题（连续或离散）
强化学习任务
监督学习任务

各种进化计算算法在Evotorch中提供：

基于分布的搜索算法：
- PGPE：具有基于参数的探索的政策梯度。
- XNES：指数自然进化策略。
- CMA-ES：协方差矩阵适应进化策略。
- SNES：可分离的自然进化策略。
- CEM：交叉熵方法。
基于人群的搜索算法：
- GenityGorithm：一种遗传算法实施。还支持多个目标，在这种情况下，它的行为类似于NSGA-II 。
- Cosyne：合作突触神经进化。
- Mapelites：表型精英的多维档案

由于所有这些算法都是在Pytorch中实现的，因此它们受益于在GPU上使用矢量化和并行化，因此在可用GPU时会大大加快优化。使用射线，通过将工作量跨越：

多个CPU
多个GPU
射线集群中的多台计算机

例子

以下是一些代码示例，这些示例证明了Evotorch的API。

黑盒优化示例

与Pytorch一起使用的任何目标函数都可以直接与Evotorch一起使用。非矢量化的目标函数只需接收解决方案作为1维型火炬张量，然后返回适应性作为标量。矢量化的目标函数接收一批溶液作为二维火炬张量，并返回适应性的一维张量。以下示例演示了如何定义和解决经典的rastrigin问题。

 from evotorch import Problem
from evotorch . algorithms import SNES
from evotorch . logging import StdOutLogger , PandasLogger
import math
import matplotlib . pyplot as plt
import torch


# Declare the objective function
def rastrigin ( x : torch . Tensor ) -> torch . Tensor :
    A = 10
    ( _ , n ) = x . shape
    return A * n + torch . sum (( x ** 2 ) - A * torch . cos ( 2 * math . pi * x ), 1 )


# Declare the problem
problem = Problem (
    "min" ,
    rastrigin ,
    initial_bounds = ( - 5.12 , 5.12 ),
    solution_length = 100 ,
    vectorized = True ,
    # device="cuda:0"  # enable this line if you wish to use GPU
)

# Initialize the SNES algorithm to solve the problem
searcher = SNES ( problem , popsize = 1000 , stdev_init = 10.0 )

# Initialize a standard output logger, and a pandas logger
_ = StdOutLogger ( searcher , interval = 10 )
pandas_logger = PandasLogger ( searcher )

# Run SNES for the specified amount of generations
searcher . run ( 2000 )

# Get the progress of the evolution into a DataFrame with the
# help of the PandasLogger, and then plot the progress.
pandas_frame = pandas_logger . to_dataframe ()
pandas_frame [ "best_eval" ]. plot ()
plt . show ()

强化学习示例

以下示例演示了如何解决通过健身房图书馆可用的加强学习任务。

 from evotorch . algorithms import PGPE
from evotorch . logging import StdOutLogger , PicklingLogger
from evotorch . neuroevolution import GymNE

# Declare the problem to solve
problem = GymNE (
    env = "Humanoid-v4" ,  # Solve the Humanoid-v4 task
    network = "Linear(obs_length, act_length)" ,  # Linear policy
    observation_normalization = True ,  # Normalize the policy inputs
    decrease_rewards_by = 5.0 ,  # Decrease each reward by 5.0
    num_actors = "max" ,  # Use all available CPUs
    # num_actors=4,    # Explicit setting. Use 4 actors.
)

# Instantiate a PGPE algorithm to solve the problem
searcher = PGPE (
    problem ,
    # Base population size
    popsize = 200 ,
    # For each generation, sample more solutions until the
    # number of simulator interactions reaches this threshold
    num_interactions = int ( 200 * 1000 * 0.75 ),
    # Stop re-sampling solutions if the current population size
    # reaches or exceeds this number.
    popsize_max = 3200 ,
    # Learning rates
    center_learning_rate = 0.0075 ,
    stdev_learning_rate = 0.1 ,
    # Radius of the initial search distribution
    radius_init = 0.27 ,
    # Use the ClipUp optimizer with the specified maximum speed
    optimizer = "clipup" ,
    optimizer_config = { "max_speed" : 0.15 },
)

# Instantiate a standard output logger
_ = StdOutLogger ( searcher )

# Optional: Instantiate a logger to pickle and save the results periodically.
# In this example, among the saved results will be the center of the search
# distribution, since we are using PGPE which is distribution-based.
_ = PicklingLogger ( searcher , interval = 10 )

# Run the algorithm for the specified amount of generations
searcher . run ( 500 )

# Get the center point of the search distribution,
# obtain a policy out of that point, and visualize the
# agent using that policy.
center_solution = searcher . status [ "center" ]
trained_policy = problem . make_net ( center_solution )
problem . visualize ( trained_policy )

可以在此处找到更多示例。

如何引用

如果您在研究中使用Evotorch，请考虑引用我们的论文。

 @article { evotorch2023arxiv ,
  title = { {EvoTorch}: Scalable Evolutionary Computation in {Python} } ,
  author = { Toklu, Nihat Engin and Atkinson, Timothy and Micka, Vojtv{e}ch and Liskowski, Pawel{} and Srivastava, Rupesh Kumar } ,
  journal = { arXiv preprint } ,
  year = { 2023 } ,
  note = { https://arxiv.org/abs/2302.12600 }
}