今日は昼過ぎからpythonを睨んでた．
6時間くらいかけてようやくLeNet-5を使った学習（MNISTデータセットの手書き文字認識）が上手くできた．
Epoch 1 loss(train) = variable(230.29222106933594), accuracy(train) = 0.1032, accuracy(test) = 0.101 Epoch 2 loss(train) = variable(219.98751831054688), accuracy(train) = 0.3816, accuracy(test) = 0.3776 Epoch 3 loss(train) = variable(129.0580291748047), accuracy(train) = 0.6319, accuracy(test) = 0.6383 Epoch 4 loss(train) = variable(82.83393096923828), accuracy(train) = 0.7655, accuracy(test) = 0.7681 Epoch 5 loss(train) = variable(61.799598693847656), accuracy(train) = 0.8188, accuracy(test) = 0.8188 ：
： Epoch 18 loss(train) = variable(14.283223152160645), accuracy(train) = 0.9605, accuracy(test) = 0.9506 Epoch 19 loss(train) = variable(13.457979202270508), accuracy(train) = 0.9619, accuracy(test) = 0.9533 Epoch 20 loss(train) = variable(12.705121994018555), accuracy(train) = 0.9629, accuracy(test) = 0.9547
ただ，SGDでパラメータを最適化した場合だと，下のように精度が11％で止まってしまう．
Adamだと上のように上手くいく．
謎．
Epoch 1 loss(train) = variable(230.16461181640625), accuracy(train) = 0.1127, accuracy(test) = 0.1135 Epoch 2 loss(train) = variable(230.06752014160156), accuracy(train) = 0.1127, accuracy(test) = 0.1135 Epoch 3 loss(train) = variable(230.06521606445312), accuracy(train) = 0.1127, accuracy(test) = 0.1135 Epoch 4 loss(train) = variable(230.06492614746094), accuracy(train) = 0.1127, accuracy(test) = 0.1135 Epoch 5 loss(train) = variable(230.06471252441406), accuracy(train) = 0.1127, accuracy(test) = 0.1135 ：
： Epoch 18 loss(train) = variable(230.0627899169922), accuracy(train) = 0.1127, accuracy(test) = 0.1135 Epoch 19 loss(train) = variable(230.06265258789062), accuracy(train) = 0.1127, accuracy(test) = 0.1135 Epoch 20 loss(train) = variable(230.06248474121094), accuracy(train) = 0.1127, accuracy(test) = 0.1135
明日はResNetと格闘する．
