ECEMasterProject
Initial Release of Master Project
在自然灾害/现象的情况下，可能会发生停电。为了对抗这一点，电动汽车（电动汽车）驾驶员可以将其车辆用作客户/客户的电池。为了模拟这一挑战的困难，即在停电期结束之前，找到尽可能多的客户，创建了RL环境。该环境具有基于节点网络大小，房屋数量（不能大于节点大小），停电数据和住房模型的随机场景，以模拟高水平的响应。
动作中的随机代理的示例。
负载代表目前从节点所需的功率。
当前费用代表代理（或汽车）具有的功率。
边缘的值代表了边缘成本的里程。
蓝色节点代表缓冲区。绿节点代表房屋。灰色节点表示停电节点。
可以在Action＆State Space部分中找到更多详细信息。
克隆项目
git clone https://github.com/Natemen96/ECEMasterProject.git转到项目目录
cd ECEMasterProject/RL python main.py警告：这是一个较大的安装（〜2GB），根据互联网速度，大约需要30分钟。根据您的设置，您可能会发现自己更容易自己安装包装。该列表可以在conda_env文件夹中找到。有关最新的包装列表，请参见'conda_env/rl_window.yml'。在Windows上最有效。
要轻松安装，首先使用CONDA确保已安装Conda。
安装conda后，请执行以下操作以设置环境
conda env create -f conda_env/rl_window.yml --name rl_env # or any name you prefer
conda activate rl_env # verify it work 确保在Conda Env中运行该程序。
可以从山核桃街/dataport下载住房数据
从“家”（节点）前往每个位置以换取电力（EV的主要资源），卸载电力以换取奖励，回到家中以换取电力。代理商可以恢复“家”的电力，并根据其恢复的力量获得更多的奖励。它自己恢复的力量越多，奖励就越多。
将电源恢复到具有电源充电站的节点中，代理也可以在那里恢复能量。有些节点将取决于参数。
graph_env: Randomly generated a fully connected graph n nodes
Args:
n (int, optional): [number of nodes]. Defaults to 5.
numofhome (int, optional): [number of homes for agent starting
point]. Defaults to 1.
numofblackout (int, optional): [number of houses with blackout].
Defaults to 1.
numofblackoutws (int, optional): [number of blackout houses with
solarpower]. Defaults to 0.
numofchargingstation (int, optional): [number of charging
station]. Defaults to 0.
max_actions (int, optional): [max number of action can be done
times 7. By default 3 (7) = 21]. Defaults to 3.
blackout_str (str, optional): [Reads Blackout data from npy
file. Blackout data from the 2021 Texas Power Crisis, a file for
each country]. Defaults to 'Brazoria'.
agents (list, optional): [Agent objects]. Defaults to [None].
nonsolarhouse_data_paths (list, optional): [Path for npy file
with nonsolar data info. Added to a node per path.]. Defaults to
[None].
solarhouse_data_paths (list, optional): [Path for npy file with
solarhouse data info. Added to a node per path.]. Defaults to
[None].
data_sample (int, optional): [how many data sample it take from
data in paths, samples taken evenly throughout data. By default
(0/6. 1/6 ... 6/6 0/6 ...) sample]. Defaults to 6.BasicAgent: Basic Agent that act as skeleton for other agent. Won't work by itself.
Args:
car ([dict]): [car information in the form of a python
dictionary. Required dict information: model, make, year,
avg_energy_consumption(kWh/mile), battery_capacity(kWh)]SmartQLAgent: Smart Agent that uses Qtable to make decisions
Args:
car ([dict]): [car information in the form of a python
dictionary. Required dict information: model, make, year,
avg_energy_consumption(kWh/mile), battery_capacity(kWh)]
sample_rate (int, optional): [decides how often a sample of the
qtable is taken. If left to default it's every 1/100 of total ep]
. Defaults to 100.
qtabletraining (bool, optional): [Flag for turning qtable
training on]. Defaults to True.
quiet (bool, optional): [Flag for turning data collection on].
Defaults to True.RandomDataAgent: Used for data collection of random agent. Doesn't
use Observation.
Args:
car ([dict]): [car information in the form of a python
dictionary. Required dict information: model, make, year,
avg_energy_consumption(kWh/mile), battery_capacity(kWh)]给予卸载，充电给予奖励。电力恢复的多样性将带来更多的回报。奖励因能量用尽而失去。当电动汽车出现能源或网络完全动力时，情节已经结束。
Each Agent will have 24 actions per ep by default.
e+2 Actions:
e - Move (at cost) to Node if possible
1 - Unload (at cost)
1 - Recharge (no cost, expect an action)
2 * n + 6 * n + 300 * n + c States:
n - Statuses of Each Node
Key - Assignment
0 - Buffer (Blue)
1 - Blackout (Darkgrey)
2 - Blackout with Solar (Lightgrey)
3 - Powered (Blue)
4 - Charging Station (Yellow)
5 - Home (Green)
n - Cost of each edge
n - Is the path reachable or not?
0 - No
1 - Yes
c - Charge State (400 possible values by default)
where n is the number of nodes states, e is the number of edges, and c number of is Charge states 每个节点均设计为基于房屋数据集。与非极性动力房屋相比，一些太阳能的房屋需要更少的电动汽车功率。房屋节点的成本可以从住房数据中得出。
基于德克萨斯停电数据以及每天更改的停电百分比。
至少使用Q学习来找到最佳解决方案。
可以缩放到多个EV（需要MARL）和更大的网络。 MARL尚未经过全面测试，可能很大。
在50K EPS的测试运行期间，与随机代理相比（随机操作）相比，QTABLES支持的智能代理提高了263.13％。
scripts/Data/TexasBlackout.csv
-csv file of data collected from https://poweroutage.us/area/state/texas during the 2021 Texas Power Crisis with the following
- Metrics: ID,County,Customers Tracked,Customers Out,Outage %,DateTime
ECEMasterProject/RL/Data/
Avg_House_0/
- npy files of average daily power per every 15 mins of various houses provide by Dataport
Avg_House_1/
- npy files of next day regression of Avg_House_0 per house
Avg_House_n/
- npy files of concatenation of Avg_House_0 adn Avg_House_1 per houses
County/
- npy files of Texas County that were effect the most during the 2021 Texas Power Crisis
RL/agent/Sim_Data
- csv file result from various RL training, only stored reward.
RL/agent/agent_jsons/
-various json file of EV information
-Metrics: model, make, year, avg_energy_consumption(kWh/mile), battery_capacity(kWh), source
models/
KM/
-Classifier models trained using Time Series K Means stored in h5
LSTM/
-Old LSTM regression model trained on one house stored in h5
LSTM_nonsolar/
-LSTM regression model trained on non solar data stored in h5
LSTM_solar/
-LSTM regression model trained on solar data stored in h5
Qtables/
-Qtables from RL training stored in h5
notebooks/
analysis_c0.ipynb
-Analysis of classifier and regression model on solar houses data
analysis_c1.ipynb
-Analysis of classifier and regression model on nonsolar houses data
cluster.ipynb
-Creation and Analysis of classifier that can distinguish between solar and nonsolar data
compare_qtable_analyis.ipynb
-Comparing Qtable Smart Agent with Random Agent Baseline
D_LSTM_HousingData.ipynb
-Creation and analysis of regression on avg housing data where the original data sample occurs every 15 mins -> 98 sample per day
Dp1_analysis.ipynb
-Creation and analysis of regression on one house
EveryNyHouseAvg.ipynb
-Creation and analysis of regression on all house samples with model trained on one house (house 1)
LSTM_cluster0.ipynb
-Creation and analysis of regression on one house using most of the solar housing data
LSTM_cluster1.ipynb
-Creation and analysis of regression on one house using most of the nonsolar housing data
qtable_analyis_4_19.ipynb
-Analysis of Smart Agents performance
qtable_analyis_4_22.ipynb
-Analysis of Smart Agents performance
qtable_analyis.ipynb
-Analysis of Smart Agents performance
random_agent_analyis.ipynb
-Analysis of Random Agents performance
Texas_Blackout.ipynb
-Analysis of recorded Blackout Data随着天的过去，一些边缘可以阻止并恢复（模拟自然灾害）
添加测试
DRL和经典优化技术
调试马尔
转换为OpenAI环境
了解更多有关制作DL/ML模型的信息。
从探索数据和DL/ML培训结果中了解了有关数据分析的更多信息。
了解了网络X及其与项目相关的图理论的功能。
了解更多有关增强学习环境和代理的知识。
在2021年德克萨斯州电力危机期间需要刮擦停电数据，了解了有关数据刮擦的更多信息
学会了如何使用Python中的图像制作GIF。
感谢我母校普渡大学的秦教授的惊人支持，并为该项目分享了他的专业知识。
感谢Dataport和Pecan Street提供的住房数据以构建模型。
感谢棋盘游戏的创建者Matt Leacock启发了这种强化学习环境。
