c++ - Training NN to calculate atan2(y, x)
问题描述
I've been working on a Q reinforcement learning implementation, where Q(π, a) is is approximated with a neural network. During trouble-shooting, I reduced the problem down to a very simple first step: train a NN to calculate atan2(y, x).
I'm using FANN for this problem, but the library is largely irrelevant as this question is more about the appropriate technique to use.
I have been struggling to teach the NN, given input = {x, y}, to calculate output = atan2(y, x).
Here is the naïve approach I have been using. It's extremely simplistic, but I'm trying to keep this simple to work up.
#include "fann.h"
#include <cstdio>
#include <random>
#include <cmath>
int main()
{
// creates a 3 layered, densely connected neural network, 2-3-1
fann *ann = fann_create_standard(3, 2, 3, 1);
// set the activation functions for the layers
fann_set_activation_function_hidden(ann, FANN_SIGMOID_SYMMETRIC);
fann_set_activation_function_output(ann, FANN_SIGMOID_SYMMETRIC);
fann_type input[2];
fann_type expOut[1];
fann_type *calcOut;
std::default_random_engine rng;
std::uniform_real_distribution<double> unif(0.0, 1.0);
for (int i = 0; i < 100000000; ++i) {
input[0] = unif(rng);
input[1] = unif(rng);
expOut[0] = atan2(input[1], input[0]);
// does a single incremental training round
fann_train(ann, input, expOut);
}
input[0] = unif(rng);
input[1] = unif(rng);
expOut[0] = atan2(input[1], input[0]);
calcOut = fann_run(ann, input);
printf("Testing atan2(%f, %f) = %f -> %f\n", input[1], input[0], expOut[0], calcOut[0]);
fann_destroy(ann);
return 0;
}
Super simple, right? However, even after 100,000,000 iterations this neural network fails:
Testing atan2(0.949040, 0.756997) = 0.897493 -> 0.987712
I also tried using a linear activation function on the output layer (FANN_LINEAR
). No luck. In fact, the results are much worse. After 100,000,000 iterations, we get:
Testing atan2(0.949040, 0.756997) = 0.897493 -> 7.648625
Which is even worse than when the weights were randomly initialized. How could a NN get worse after training?
I found this issue with FANN_LINEAR
to be consistent with other tests. When linear output is needed (e.g. in the calculation of the Q value, which corresponds to arbitrarily large or small rewards), this approach fails miserably and error actually appears to increase with training.
So what is going on? Is using a fully-connected 2-3-1 NN inappropriate for this situation? Is a symmetric sigmoid activation function in the hidden layer inappropriate? I fail to see what else could possibly account for this error.
解决方案
The problem you are facing is normal, and the quality of your predictor won't improve by augmenting the number of iterations, you should augment the size of your NN, either by adding some layers or by augemnting the size of the hidden layer. Instead of having 2-3-1 you can try 2-256-128-1 for example. Normally that will work better. If you want have a look on this simple code I wrote in python to do the same task, and it is wroking well
import numpy as np
from numpy import arctan2
from keras.models import Sequential
from keras.layers import Dense, InputLayer
nn_atan2 = Sequential()
nn_atan2.add(Dense(256, activation="sigmoid", input_shape=(2,)))
nn_atan2.add(Dense(128, activation="sigmoid"))
nn_atan2.add(Dense(1, activation='tanh'))
nn_atan2.compile(optimizer="adam", loss="mse")
nn_atan2.summary()
N = 100000
X = np.random.uniform(size=(N,2) )
y = arctan2(X[:,0], X[:,1])/(np.pi*0.5)
nn_atan2.fit(X,y, epochs=10, batch_size=128)
def predict(x, y):
return float(nn_atan2.predict(np.array([[x, y]]))*(np.pi*0.5))
Runnin this code will give
Epoch 1/10
100000/100000 [==============================] - 3s 26us/step - loss: 0.0289
Epoch 2/10
100000/100000 [==============================] - 2s 24us/step - loss: 0.0104
Epoch 3/10
100000/100000 [==============================] - 2s 24us/step - loss: 0.0102
Epoch 4/10
100000/100000 [==============================] - 2s 24us/step - loss: 0.0096
Epoch 5/10
100000/100000 [==============================] - 2s 24us/step - loss: 0.0082
Epoch 6/10
100000/100000 [==============================] - 2s 23us/step - loss: 0.0051
Epoch 7/10
100000/100000 [==============================] - 2s 23us/step - loss: 0.0027
Epoch 8/10
100000/100000 [==============================] - 2s 23us/step - loss: 0.0019
Epoch 9/10
100000/100000 [==============================] - 2s 23us/step - loss: 0.0014
Epoch 10/10
100000/100000 [==============================] - 2s 23us/step - loss: 0.0010
推荐阅读
- teiid - 从虚拟存储过程调用 Oracle 和 MySQL 存储过程
- r - 近似字符串匹配的逻辑是什么?
- reactjs - 如何修复“确切”路线?
- visual-studio - “NPM 审计”命令报告某个软件包版本的问题“NPM 显示版本”说我没有
- grails - Grails 2.5,beforeDelete 无法访问一对多关系
- python - 将 3D 数组另存为 3D tif 图像
- angular - Angular 在 prod 和 dev 构建中都不能在 IE 11 中工作
- visual-studio-code - 如何在 Visual Studio Code 中自定义搜索结果颜色
- android - Firebase根据Android中的特定值删除所有节点
- sql - 如何将日期从十进制转换为日期