Contenu connexe


2016 dg2

  1. Digital Garage & Naviplus 主催 第2回Pythonで実践する深層学習 浅川伸一
  2. 10/06/2016 2/24 Notice 次回以降で取り上げる話題,データ,プロジェクト,質疑応答のために slack のチームを作成しました。今回参加されなかった方でも自由に参 加できます。チーム名は です。ご参加ください。 参加ご希望の方は までメールをお 願いします。 #DLwPY
  3. 10/06/2016 3/24 本日のお品書き ● Basic knowledge of neurons ● Cross entropy ● Feedforwards and feedbacks of neural networks ● Try googleplayground ● Convolutional neural networks ● Try keras
  4. 10/06/2016 4/24 前回の補足 最速で理解するには 1. 線形回帰 linear regression 2. ロジスティック回帰 logistic regression 3. 正則化 regularization 4. 多層パーセプトロン multi-layered perceptrons 5. 畳み込みニューラルネットワーク convolutional neural networks 6. リカレントニューラルネットワーク recurrent neural networks 7. 強化学習 reinforcement learning
  5. 10/06/2016 5/24 Warning: Stealing Machine Learning Models via Prediction APIs Machine learning (ML) models may be deemed confidential due to their sensitive training data, commercial value, or use in security applications. Increasingly often, confidential ML models are being deployed with publicly accessible query interfaces. ML-as-a-service ("predictive analytics") systems are an example: Some allow users to train models on potentially sensitive data and charge others for access on a pay-per-query basis. The tension between model confidentiality and public access motivates our investigation of model extraction attacks. In such attacks, an adversary with black-box access, but no prior knowledge of an ML model's parameters or training data, aims to duplicate the functionality of (i.e., "steal") the model. Unlike in classical learning theory settings, ML-as-a-service offerings may accept partial feature vectors as inputs and include confidence values with predictions. Given these practices, we show simple, efficient attacks that extract target ML models with near-perfect fidelity for popular model classes including logistic regression, neural networks, and decision trees. We demonstrate these attacks against the online services of BigML and Amazon Machine Learning. We further show that the natural countermeasure of omitting confidence values from model outputs still admits potentially harmful model extraction attacks. Our results highlight the need for careful ML model deployment and new model extraction countermeasures.
  6. 10/06/2016 6/24 Prerequisites ● Basics about Neural Networks – How the brain actually works? – How parallel computation works adapting parameters inspired by neurons? – How the brain implements learning algorithms? ● (ペ)What is it a good idea to try to emulate the brain when solving a recognition task?
  7. 10/06/2016 7/24 A schematic neuron There are many neurotransimtters, but we deal with those as positive/negative weights and also positive negative inputs. (ペ) why?
  8. 10/06/2016 8/24 Why computer vision is so difficult?
  9. 10/06/2016 9/24 Cross entropy
  10. 10/06/2016 10/24
  11. 10/06/2016 11/24 Convnet.js
  12. 10/06/2016 12/24 TensorFlow Tips ● Computation graph (see , その翻訳記事 は ) ● Ways of installations (c.f. Download and Setup ) – Pip – Virutalenv – Anaconda – Docker ● Let’s try ● You can also check it out, Karpthy’s convnetjs ● Keras is another choice to consider
  13. 10/06/2016 13/24 A computation graph
  14. 10/06/2016 14/24 Sample code of TensorFlow import tensorflow as tf W = tf.get_variable(shape=[], name='W') b = tf.get_variable(shape=[], name='b') x = tf.placeholder(shape=[None], dtype=tf.float32, name='x') y = tf.matmul(W, x) + b with tf.Session() as sess: print(, feed_dict={x: x_in}))
  15. 10/06/2016 15/24 The difference between placeholder and variable Since Tensor computations compose graphs then it's better to interpret the two in terms of graphs. When you launch the graph, variables have to be explicitly initialized before you can run Ops that use their value. Then during the process of the an operation variables should be constant. import tensorflow as tf # Create a variable. # w = tf.Variable(<initial-value>, name=<optional-name>) w = tf.Variable(tf.truncated_normal([10, 40])) v = tf.Variable(tf.truncated_normal([40, 20])) # Use the variable in the graph like any Tensor. # The variable should be initialized before this operation! y = tf.matmul(w, v) # Assign a new value to the variable with `assign()` or a related method. w.assign(w + 1.0) w.assign_add(1.0) tf.Variableはオペレーション実行前に初期化される
  16. 10/06/2016 16/24 The difference between placeholder and variable A placeholder is a handle of a value in the operation and it can be not initialized before the execution of the graph (launching the graph in the session which does its computation relaying on a highly efficient C+ + backend). x = tf.placeholder(tf.float32, shape=(1024, 1024)) # You don't need to initialize it to calculate y, it's different from # the variable above, the placeholder is a "variable"(not intialized) # in this operation. y = tf.matmul(x, x) with tf.Session() as sess: # However you should initialize x to execute y for the execution of the graph. print( # ERROR: will fail because x was not fed. rand_array = np.random.rand(1024, 1024) print(, feed_dict={x: rand_array})) # Will succeed. プレースホルダーは初期化されない
  17. 10/06/2016 17/24 Convolutional neural networks: LeNet5 LeCun1998 より
  18. 10/06/2016 18/24 AlexNet Krizensky+2012 より
  19. 10/06/2016 19/24 GoogLeNet
  20. 10/06/2016 20/24 GoogLeNet Inception module
  21. 10/06/2016 21/24 畳み込み演算
  22. 10/06/2016 22/24 kernels
  23. 10/06/2016 23/24 To understand convolution ● 畳み込みの理解には
  24. 10/06/2016 24/24 What is convolution? ● Convolution is an operation from signal processing ● Filters, or kernels in machine learning,