文档章节

用TensorFlow实现AND和XOR

propagator
 propagator
发布于 2017/03/11 11:18
字数 1463
阅读 524
收藏 1

用TensorFlow实现简单的AND和XOR函数,是一个很好的多层神经网络的例子。因为XOR函数无法用单层神经网络实现。

 

先上代码(此段代码计算XOR函数,AND函数只需修改OUTPUT_TRAIN的值即可)

#!/usr/bin/env python

import tensorflow as tf
import math
import numpy as np

INPUT_COUNT = 2
OUTPUT_COUNT = 2
HIDDEN_COUNT = 2
LEARNING_RATE = 0.1
MAX_STEPS = 5000

# For every training loop we are going to provide the same input and expected output data
INPUT_TRAIN = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
OUTPUT_TRAIN = np.array([[1, 0], [0, 1], [0, 1], [1, 0]])

# Nodes are created in Tensorflow using placeholders. Placeholders are values that we will input when we ask Tensorflow to run a computation.
# Create inputs x consisting of a 2d tensor of floating point numbers
inputs_placeholder = tf.placeholder("float",
shape=[None, INPUT_COUNT])
labels_placeholder = tf.placeholder("float",
shape=[None, OUTPUT_COUNT])

# We need to create a python dictionary object with placeholders as keys and feed tensors as values
feed_dict = {
inputs_placeholder: INPUT_TRAIN,
labels_placeholder: OUTPUT_TRAIN,
}

# Define weights and biases from input layer to hidden layer
WEIGHT_HIDDEN = tf.Variable(tf.truncated_normal([INPUT_COUNT, HIDDEN_COUNT]))
BIAS_HIDDEN = tf.Variable(tf.zeros([HIDDEN_COUNT]))

# Define an activation function for the hidden layer. Here we are using the Sigmoid function, but you can use other activation functions offered by Tensorflow.
AF_HIDDEN = tf.nn.sigmoid(tf.matmul(inputs_placeholder, WEIGHT_HIDDEN) + BIAS_HIDDEN)

#  Define weights and biases from hidden layer to output layer. The biases are initialized with tf.zeros to make sure they start with zero values.
WEIGHT_OUTPUT = tf.Variable(tf.truncated_normal([HIDDEN_COUNT, OUTPUT_COUNT]))
BIAS_OUTPUT = tf.Variable(tf.zeros([OUTPUT_COUNT]))

# With one line of code we can calculate the logits tensor that will contain the output that is returned
logits = tf.matmul(AF_HIDDEN, WEIGHT_OUTPUT) + BIAS_OUTPUT
# We then compute the softmax probabilities that are assigned to each class
y = tf.nn.softmax(logits)

# The tf.nn.softmax_cross_entropy_with_logits op is added to compare the output logits to expected output
#cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits, y)
cross_entropy = -tf.reduce_sum(labels_placeholder * tf.log(y))
# It then uses tf.reduce_mean to average the cross entropy values across the batch dimension as the total loss
loss = tf.reduce_mean(cross_entropy)

# Next, we instantiate a tf.train.GradientDescentOptimizer that applies gradients with the requested learning rate. Since Tensorflow has access to the entire computation graph, it can find the gradients of the cost of all the variables.
train_step = tf.train.GradientDescentOptimizer(LEARNING_RATE).minimize(loss)

# Next we create a tf.Session () to run the graph
init = tf.global_variables_initializer()
with tf.Session() as sess:
	# Then we run the session
	sess.run(init)
	
	# The following code fetch two values [train_step, loss] in its run call. Because there are two values to fetch, sess.run() returns a tuple with two items. We also print the loss and outputs every 100 steps.
	for step in range(MAX_STEPS):
		loss_val = sess.run([train_step, loss], feed_dict)
		if step % 100 == 0:
			print ("Step:", step, "loss: ", loss_val)
			for input_value in INPUT_TRAIN:
				print (input_value, sess.run(y, 
				feed_dict={inputs_placeholder: [input_value]}))

 

运行结果如下

 

虽然英文注释已经写得很清楚,不过接下来还是逐条说明一下。

 

import tensorflow as tf
import math
import numpy as np

INPUT_COUNT = 2
OUTPUT_COUNT = 2
HIDDEN_COUNT = 2
LEARNING_RATE = 0.1
MAX_STEPS = 5000

# For every training loop we are going to provide the same input and expected output data
INPUT_TRAIN = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
OUTPUT_TRAIN = np.array([[1, 0], [0, 1], [0, 1], [1, 0]])

这段代码导入了必要的库,并定义了基本的常数,比如输入变量的个数INPUT_COUNT,输出变量的个数OUTPUT_COUNT,隐藏层神经元个数HIDDEN_COUNT,学习速率LEARNING_RATE,以及最多训练的次数MAX_STEPS。值得一提的是训练集,对于XOR函数,有 XOR(0, 0) = 0, XOR(0, 1) = 1, XOR(1, 0) = 1, XOR(1, 1) = 0。那么训练集应该是输入集[[0,0], [0,1], [1,0], [1,1]],输出集[0, 1, 1, 0],为何此处定义看起来很奇怪?原因是tensorflow的输出默认使用了one hot encoding(独热码),类似前面MNIST例子中用到的那样,对每个状态做了区分。此处输出包含两个状态0和1,因此分别使用[1,0]和[0,1]代表(也可以反过来用[0,1]代表0,[1,0]代表1,只要一开始规定好就行)。关于one hot encoding的基本信息,可参阅如下网址或自行百度

http://baike.baidu.com/link?url=7Qw-sLrFRegdtYgJPpPXR687n-PwbpczZ7lmPwskwHi0YtUDHE4PLwNfTftnBT0ZIsq-glmxkqp01VDK4NTU6qU5Kwe_4p99wtjWwdLGa_A4EysH219Sh6oZjXDkNQ3s

对于AND函数,可以将输出训练集改为OUTPUT_TRAIN=np.array([[1, 0], [1, 0], [1, 0], [0, 1])。(或者OUTPUT_TRAIN=np.array([[0, 1], [0, 1], [0, 1], [1, 0]]),取决于如何定义,每次计算都可以独立定义。)

 

# Nodes are created in Tensorflow using placeholders. Placeholders are values that we will input when we ask Tensorflow to run a computation.
# Create inputs x consisting of a 2d tensor of floating point numbers
inputs_placeholder = tf.placeholder("float",
shape=[None, INPUT_COUNT])
labels_placeholder = tf.placeholder("float",
shape=[None, OUTPUT_COUNT])

# We need to create a python dictionary object with placeholders as keys and feed tensors as values
feed_dict = {
inputs_placeholder: INPUT_TRAIN,
labels_placeholder: OUTPUT_TRAIN,
}

接下来这几句定义了输入和输出的占位符,并将这个占位符和前面定义的输入输出变量矩阵关联起来。feed_dict的定义纯粹是为了方便,也可以类似MNIST例子中那样直接放在session.run()中。

 

# Define weights and biases from input layer to hidden layer
WEIGHT_HIDDEN = tf.Variable(tf.truncated_normal([INPUT_COUNT, HIDDEN_COUNT]))
BIAS_HIDDEN = tf.Variable(tf.zeros([HIDDEN_COUNT]))

# Define an activation function for the hidden layer. Here we are using the Sigmoid function, but you can use other activation functions offered by Tensorflow.
AF_HIDDEN = tf.nn.sigmoid(tf.matmul(inputs_placeholder, WEIGHT_HIDDEN) + BIAS_HIDDEN)

#  Define weights and biases from hidden layer to output layer. The biases are initialized with tf.zeros to make sure they start with zero values.
WEIGHT_OUTPUT = tf.Variable(tf.truncated_normal([HIDDEN_COUNT, OUTPUT_COUNT]))
BIAS_OUTPUT = tf.Variable(tf.zeros([OUTPUT_COUNT]))

接下来定义了两层网络的权重WEIGHT_HIDDEN,WEIGHT_OUTPUT和偏置BIAS_HIDDEN,BIAS_OUTPUT。AF_HIDDEN是第一层的输出,用到了经典的sigmoid函数。

 

# With one line of code we can calculate the logits tensor that will contain the output that is returned
logits = tf.matmul(AF_HIDDEN, WEIGHT_OUTPUT) + BIAS_OUTPUT
# We then compute the softmax probabilities that are assigned to each class
y = tf.nn.softmax(logits)

# The tf.nn.softmax_cross_entropy_with_logits op is added to compare the output logits to expected output
#cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits, y)
cross_entropy = -tf.reduce_sum(labels_placeholder * tf.log(y))
# It then uses tf.reduce_mean to average the cross entropy values across the batch dimension as the total loss
loss = tf.reduce_mean(cross_entropy)

与MNIST的例子类似,上面的几句定义了交叉熵(符号使用略有差异),之所以不使用tf.nn.softmax_cross_entropy_with_logits函数(注释掉的cross_entropy那句),是因为在此例中,和MNIST中不同,这个方法收敛性差。

 

# Next, we instantiate a tf.train.GradientDescentOptimizer that applies gradients with the requested learning rate. Since Tensorflow has access to the entire computation graph, it can find the gradients of the cost of all the variables.
train_step = tf.train.GradientDescentOptimizer(LEARNING_RATE).minimize(loss)

# Next we create a tf.Session () to run the graph
init = tf.global_variables_initializer()
with tf.Session() as sess:
	# Then we run the session
	sess.run(init)
	
	# The following code fetch two values [train_step, loss] in its run call. Because there are two values to fetch, sess.run() returns a tuple with two items. We also print the loss and outputs every 100 steps.
	for step in range(MAX_STEPS):
		loss_val = sess.run([train_step, loss], feed_dict)
		if step % 100 == 0:
			print ("Step:", step, "loss: ", loss_val)
			for input_value in INPUT_TRAIN:
				print (input_value, sess.run(y, 
				feed_dict={inputs_placeholder: [input_value]}))

最后定义了优化算法,然后初始化session,开始运行,并且每100次训练就给出一个训练结果。

 

MNIST的例子可参照如下地址:

https://my.oschina.net/propagator/blog/851912

© 著作权归作者所有

propagator
粉丝 8
博文 40
码字总数 45924
作品 0
昆明
私信 提问
史上最全TensorFlow学习资源汇总

来源 悦动智能(公众号ID:aibbtcom) 本篇文章将为大家总结TensorFlow纯干货学习资源,非常适合新手学习,建议大家收藏。 ▌一 、TensorFlow教程资源 1)适合初学者的TensorFlow教程和代码示...

悦动智能
2018/04/12
424
0
【干货】史上最全的Tensorflow学习资源汇总,速藏!

一 、Tensorflow教程资源: 1)适合初学者的Tensorflow教程和代码示例:(https://github.com/aymericdamien/TensorFlow-Examples)该教程不光提供了一些经典的数据集,更是从实现最简单的“Hel...

技术小能手
2018/04/16
0
0
TensorFlow 携手 NVIDIA,使用 TensorRT 优化 TensorFlow Serving 性能

雷锋网 AI 科技评论按:日前,TensorFlow 团队与 NVIDIA 携手合作,将 NVIDIA 用来实现高性能深度学习推理的平台——TensorRT 与 TensorFlow Serving 打通结合,使用户可以轻松地实现最佳性能...

丛末
03/07
0
0
成为 Master,TensorFlow 新增对 Windows 支持

现在,我们开始在 Windows 上实现对 TensorFlow 的初步支持。 在 Windows 上原生支持 TensorFlow 是我们在开放 TensorFlow 源代码后收到的第一批请求之一。虽然有些 Windows 用户已经设法在 ...

两味真火
2017/01/05
6.1K
10
图解TensorFlow架构与设计

本文转载自人工智能头条 (ID:AI_Thinker) TensorFlow是什么? TensorFlow基于数据流图,用于大规模分布式数值计算的开源框架。节点表示某种抽象的计算,边表示节点之间相互联系的张量。 ...

人工智能架构
07/30
0
0

没有更多内容

加载失败,请刷新页面

加载更多

java通过ServerSocket与Socket实现通信

首先说一下ServerSocket与Socket. 1.ServerSocket ServerSocket是用来监听客户端Socket连接的类,如果没有连接会一直处于等待状态. ServetSocket有三个构造方法: (1) ServerSocket(int port);...

Blueeeeeee
10分钟前
1
0
用 Sphinx 搭建博客时,如何自定义插件?

之前有不少同学看过我的个人博客(http://python-online.cn),也根据我写的教程完成了自己个人站点的搭建。 点此:使用 Python 30分钟 教你快速搭建一个博客 为防有的同学不清楚 Sphinx ,这...

王炳明
昨天
3
0
黑客之道-40本书籍助你快速入门黑客技术免费下载

场景 黑客是一个中文词语,皆源自英文hacker,随着灰鸽子的出现,灰鸽子成为了很多假借黑客名义控制他人电脑的黑客技术,于是出现了“骇客”与"黑客"分家。2012年电影频道节目中心出品的电影...

badaoliumang
昨天
12
0
很遗憾,没有一篇文章能讲清楚线程的生命周期!

(手机横屏看源码更方便) 注:java源码分析部分如无特殊说明均基于 java8 版本。 简介 大家都知道线程是有生命周期,但是彤哥可以认真负责地告诉你网上几乎没有一篇文章讲得是完全正确的。 ...

彤哥读源码
昨天
13
0
jquery--DOM操作基础

本文转载于:专业的前端网站➭jquery--DOM操作基础 元素的访问 元素属性操作 获取:attr(name);$("#my").attr("src"); 设置:attr(name,value);$("#myImg").attr("src","images/1.jpg"); ......

前端老手
昨天
6
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部