文档章节

Deeplearning4j 手写体数字识别

孟飞阳
 孟飞阳
发布于 2017/08/12 23:40
字数 2859
阅读 371
收藏 5

最近这几年,深度学习很火,包括自己在内的很多对机器学习还是一知半解的小白也开始用深度学习做些应用。由于小白的等级不高,算法自己写不出来,所以就用了开源库。Deep Learning的开源库有多,如果以语言来划分的话,就有Python系列的tensowflow,theano,keras,C/C++系列的Caffe,还有Lua系列的torch等等。但咱们公司是用Java为主,大部分项目最终也是做成一个Java Web的服务,所以我最终选择了Deeplearning4j。

    Deeplearning4j是国外创业公司Skymind的产品。目前最新的版本更新到了0.7.2。源码全部公开并托管在github上(https://github.com/deeplearning4j/deeplearning4j)。从这个库的名字上可以看出,它就是转为Java程序员写的Deep Learning库。其实这个库吸引人的地方不仅仅在于它支持Java,更为重要的是它可以支持Spark。由于Deep Learning模型的训练需要大量的内存,而且原始数据的存储有时候也需要很大的外存空间,所以如果可以利用集群来处理便是最好不过了。当然,除了Deeplearning4j以外,还有一些Deep Learning的库可以支持Spark,比如yahoo/CaffeOnSpark,AMPLab/SparkNet以及Intel最近开源的BigDL。这些库我自己都没怎么用过,所以就不多说了,这里重点说说Deeplearning4j的使用。

    一般开始使用别人的代码库,都会先跑一些demo,或者说Hello World的例子,就好像学习一门编程语言一样,第一行代码都是打印Hello World。Deep Learning的Hello World的例子一般是两个,一个是Mnist数据集的分类,另一个就是Word2Vec找相似词。由于Word2Vec并不是严格意义上的深度神经网络,因此这里就用Lenet网络处理Mnist数据集来作为Deep Learning的Hello World。Mnist是开源的28x28的黑白手写体数字图片集(http://yann.lecun.com/exdb/mnist/),其中包含6W张训练图片和1W张测试图片。至于Lenet的相关结构描述,可以参考这个链接:http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf。下面就详细讲述下,利用Deeplearning4j如何进行建模、训练和预测评估。

    首先,我们建立一个maven项目。然后在pom文件里加入Deeplearning4j的一些相关依赖。最主要的有三个:deeplearning4j-core,datavec,nd4j。deeplearning4j-core是神经网络结构实现的代码,nd4j是用于做张量运算的库,通过JavaCPP来调用编译好的C++库(可选:ATAL, MKL, 和OpenBLAS),datavec则主要负责数据的ETL。具体可见代码:

<properties>  
  <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>  
  <nd4j.version>0.7.1</nd4j.version>  
  <dl4j.version>0.7.1</dl4j.version>  
  <datavec.version>0.7.1</datavec.version>  
  <scala.binary.version>2.10</scala.binary.version>  
</properties>  
<dependencies>  
<dependency>  
    <groupId>org.nd4j</groupId>  
    <artifactId>nd4j-native</artifactId>   
    <version>${nd4j.version}</version>  
</dependency>  
<dependency>  
    <groupId>org.deeplearning4j</groupId>  
    <artifactId>dl4j-spark_2.11</artifactId>  
    <version>${dl4j.version}</version>  
</dependency>  
     <dependency>  
          <groupId>org.datavec</groupId>  
          <artifactId>datavec-spark_${scala.binary.version}</artifactId>  
          <version>${datavec.version}</version>  
    </dependency>  
      <dependency>  
   <groupId>org.deeplearning4j</groupId>  
   <artifactId>deeplearning4j-core</artifactId>  
   <version>${dl4j.version}</version>  
</dependency>  
</dependencies>  
  1.     这些依赖里面有和Spark相关的,主要是跑Spark要用到。不过没有关系,先引进来即可。

    接着,我们解释下面的代码。我们先要定义一些具体的参数,比如分类的个数(outputNum),mini-batch的数量(batchSize)等等,具体在图中已经做了注释。需要说明的是MnistDataSetIterator这个迭代器类。这个类其实是一个读取二进制Mnist数据集的high-level的封装。通过debug我们可以发现,其中包括从网络中下载Mnist数据集,读取数据和标注,再构建迭代器的过程。在源码中,默认将下载的文件放在系统的user.home目录下,具体每个人不同会有所不同。由于我自己所处的环境网络不咋的,所以很有可能在利用这种high-level的接口的时候,因为下载Mnist数据失败而抛出异常,最终无法训练。所以,大家可以先自行下载好这些数据,然后按照源码的要求,放到相应的目录下并根据源码正确命名文件,那这样就依然可以利用这种high-level的接口。具体需要参考的是MnistDataFetcher类中相关代码。


        int nChannels = 1;      //black & white picture, 3 if color image
        int outputNum = 10;     //number of classification
        int batchSize = 64;     //mini batch size for sgd
        int nEpochs = 10;       //total rounds of training
        int iterations = 1;     //number of iteration in each traning round
        int seed = 123;         //random seed for initialize weights

        log.info("Load data....");
        DataSetIterator mnistTrain = null;
        DataSetIterator mnistTest = null;

        mnistTrain = new MnistDataSetIterator(batchSize, true, 12345);
        mnistTest = new MnistDataSetIterator(batchSize, false, 12345);

 

当我们正确读取数据后,我们需要定义具体的神经网络结构,这里我用的是Lenet,Deeplearning4j的实现参考了官网(https://github.com/deeplearning4j/dl4j-examples)的例子。具体代码如下:

        MultiLayerConfiguration.Builder builder = new NeuralNetConfiguration.Builder()
                .seed(seed)
                .iterations(iterations)
                .regularization(true).l2(0.0005)
                .learningRate(0.01)//.biasLearningRate(0.02)
                //.learningRateDecayPolicy(LearningRatePolicy.Inverse).lrPolicyDecayRate(0.001).lrPolicyPower(0.75)
                .weightInit(WeightInit.XAVIER)
                .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
                .updater(Updater.NESTEROVS).momentum(0.9)
                .list()
                .layer(0, new ConvolutionLayer.Builder(5, 5)
                        //nIn and nOut specify depth. nIn here is the nChannels and nOut is the number of filters to be applied
                        .nIn(nChannels)
                        .stride(1, 1)
                        .nOut(20)
                        .activation("identity")
                        .build())
                .layer(1, new SubsamplingLayer.Builder(SubsamplingLayer.PoolingType.MAX)
                        .kernelSize(2,2)
                        .stride(2,2)
                        .build())
                .layer(2, new ConvolutionLayer.Builder(5, 5)
                        //Note that nIn need not be specified in later layers
                        .stride(1, 1)
                        .nOut(50)
                        .activation("identity")
                        .build())
                .layer(3, new SubsamplingLayer.Builder(SubsamplingLayer.PoolingType.MAX)
                        .kernelSize(2,2)
                        .stride(2,2)
                        .build())
                .layer(4, new DenseLayer.Builder().activation("relu")
                        .nOut(500).build())
                .layer(5, new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                        .nOut(outputNum)
                        .activation("softmax")
                        .build())
                .backprop(true).pretrain(false)
                .cnnInputSize(28, 28, 1);
        // The builder needs the dimensions of the image along with the number of channels. these are 28x28 images in one channel
        //new ConvolutionLayerSetup(builder,28,28,1);

        MultiLayerConfiguration conf = builder.build();
        MultiLayerNetwork model = new MultiLayerNetwork(conf);
        model.init();        
        model.setListeners(new ScoreIterationListener(1));         // a listener which can print loss function score after each iteration

可以发现,神经网络需要定义很多的超参数,学习率、正则化系数、卷积核的大小、激励函数等都是需要人为设定的。不同的超参数,对结果的影响很大,其实后来发现,很多时间都花在数据处理和调参方面。毕竟自己设计网络的能力有限,一般都是参考大牛的论文,然后自己照葫芦画瓢地实现。这里实现的Lenet的结构是:卷积-->下采样-->卷积-->下采样-->全连接。和原论文的结构基本一致。卷积核的大小也是参考的原论文。具体细节可参考之前发的论文链接。这里我们设置了一个Score的监听事件,主要是可以在训练的时候获取每一次权重更新后损失函数的收敛情况。后面一会有截图。

定义完网络结构之后,我们就可以对之前读取的数据进行训练和分类准确性评估。先看下代码:

for( int i = 0; i < nEpochs; ++i ) {  
    model.fit(mnistTrain);  
    log.info("*** Completed epoch " + i + "***");  
  
    log.info("Evaluate model....");  
    Evaluation eval = new Evaluation(outputNum);  
    while(mnistTest.hasNext()){  
        DataSet ds = mnistTest.next();            
        INDArray output = model.output(ds.getFeatureMatrix(), false);  
        eval.eval(ds.getLabels(), output);  
    }  
    log.info(eval.stats());  
    mnistTest.reset();  
} 

    相信这部分是比较容易理解的。每训练完一轮后,我们会对测试集合进行评估,然后打印出类似下面的结果。图中的上半部分是具体分类的统计,包括分对的和分错的图片数量都可以看得到。然后,我们耐心等待一段时间,可以看到经过10轮训练的Lenet对于Mnist数据集的分类准确率达到99%如下:

Examples labeled as 0 classified by model as 0: 974 times
Examples labeled as 0 classified by model as 6: 2 times
Examples labeled as 0 classified by model as 7: 2 times
Examples labeled as 0 classified by model as 8: 1 times
Examples labeled as 0 classified by model as 9: 1 times
Examples labeled as 1 classified by model as 0: 1 times
Examples labeled as 1 classified by model as 1: 1128 times
Examples labeled as 1 classified by model as 2: 1 times
Examples labeled as 1 classified by model as 3: 2 times
Examples labeled as 1 classified by model as 5: 1 times
Examples labeled as 1 classified by model as 6: 2 times
Examples labeled as 2 classified by model as 2: 1026 times
Examples labeled as 2 classified by model as 4: 1 times
Examples labeled as 2 classified by model as 6: 1 times
Examples labeled as 2 classified by model as 7: 3 times
Examples labeled as 2 classified by model as 8: 1 times
Examples labeled as 3 classified by model as 0: 1 times
Examples labeled as 3 classified by model as 1: 1 times
Examples labeled as 3 classified by model as 2: 1 times
Examples labeled as 3 classified by model as 3: 998 times
Examples labeled as 3 classified by model as 5: 3 times
Examples labeled as 3 classified by model as 7: 1 times
Examples labeled as 3 classified by model as 8: 4 times
Examples labeled as 3 classified by model as 9: 1 times
Examples labeled as 4 classified by model as 2: 1 times
Examples labeled as 4 classified by model as 4: 973 times
Examples labeled as 4 classified by model as 6: 2 times
Examples labeled as 4 classified by model as 7: 1 times
Examples labeled as 4 classified by model as 9: 5 times
Examples labeled as 5 classified by model as 0: 2 times
Examples labeled as 5 classified by model as 3: 4 times
Examples labeled as 5 classified by model as 5: 882 times
Examples labeled as 5 classified by model as 6: 1 times
Examples labeled as 5 classified by model as 7: 1 times
Examples labeled as 5 classified by model as 8: 2 times
Examples labeled as 6 classified by model as 0: 4 times
Examples labeled as 6 classified by model as 1: 2 times
Examples labeled as 6 classified by model as 4: 1 times
Examples labeled as 6 classified by model as 5: 4 times
Examples labeled as 6 classified by model as 6: 945 times
Examples labeled as 6 classified by model as 8: 2 times
Examples labeled as 7 classified by model as 1: 5 times
Examples labeled as 7 classified by model as 2: 3 times
Examples labeled as 7 classified by model as 3: 1 times
Examples labeled as 7 classified by model as 7: 1016 times
Examples labeled as 7 classified by model as 8: 1 times
Examples labeled as 7 classified by model as 9: 2 times
Examples labeled as 8 classified by model as 0: 1 times
Examples labeled as 8 classified by model as 3: 1 times
Examples labeled as 8 classified by model as 5: 2 times
Examples labeled as 8 classified by model as 7: 2 times
Examples labeled as 8 classified by model as 8: 966 times
Examples labeled as 8 classified by model as 9: 2 times
Examples labeled as 9 classified by model as 3: 1 times
Examples labeled as 9 classified by model as 4: 2 times
Examples labeled as 9 classified by model as 5: 4 times
Examples labeled as 9 classified by model as 6: 1 times
Examples labeled as 9 classified by model as 7: 5 times
Examples labeled as 9 classified by model as 8: 3 times
Examples labeled as 9 classified by model as 9: 993 times


==========================Scores========================================
 Accuracy:        0.9901
 Precision:       0.99
 Recall:          0.99
 F1 Score:        0.99
========================================================================
[main] INFO cv.LenetMnistExample - ****************Example finished********************

    因为图传不上去,我就直接粘帖了结果。从中我们看到最终的一个准确率,还有就是哪些图片是分类正确的,哪些是分类错误的。当然我们可以通过增加训练的轮次还有调超参数来进一步优化,不过实际上这样的结果已经可以拿到生产上去用了。

    总结一下。其实包括我自己在内的很多人都对深度学习不了解,记得当时看csdn上写的有关深度学习的博客的时候,都觉得自己不可能达到那种水平。但其实,我们都忽略了一点,深度学习自身再复杂,它也是一个算法模型,也是一种机器学习。虽然它比感知机、逻辑回归等模型复杂很多(其实逻辑回归可看作神经网络中的一个神经元,充当的是激励函数的作用,类似的激励函数很多,如tanh,relu等),但终究用它的目的依然是完成回归、分类、压缩数据等任务。所以第一步尝试还是挺重要的。当然,我们不可能从复杂的模型开始,一开始就跟上当下最流行的模型,所以就从Mnist识别的例子开始,找找感觉。以后会写一些用Deeplearning4j在Spark的案例,也还是从Mnist开始。分享的同时自己也复习一下。。。

本文转载自:http://blog.csdn.net/wangongxi/article/details/54576594

共有 人打赏支持
孟飞阳
粉丝 206
博文 964
码字总数 543203
作品 5
朝阳
个人站长
私信 提问
使用DeepLearning4j训练和保存模型

一、前言 最近一直在研究深度学习,联想起之前所学,感叹数学是一门朴素而神奇的科学。F=Gm1m2/r²万有引力描述了宇宙星河运转的规律,E=mc²描述了恒星发光的奥秘,V=H*d哈勃定律描述了宇宙...

孟飞阳
06/28
0
0
10.机器学习sklearn-------手写数字识别实例

1.概念介绍: 图像识别(Image Recognition)是指利用计算机对图像进行处理、分析和理解,以识别各种不同模式的目标和对像的技术。 图像识别的发展经历了三个阶段:文字识别、数字图像处理与...

bxg1065283526
04/24
0
0
python机器学习库keras——AutoEncoder自编码、特征压缩

全栈工程师开发手册 (作者:栾鹏) python教程全解 keras使用深度网络实现自编码,也就是说对每个样本的n维特征,使用k为特征来表示,实现编码压缩的功能。也实现了特征选择的功能。比如手写...

luanpeng825485697
05/01
0
0
EasyOCR 1.2.0 新增 MNIST 数据集图片和宽幅数字识别

本软件是一个手写体数字识别软件,采用BP神经网络,基于colt数学库,有完整源码,可以保存训练结果,基于开源例程neuralnetwork-sample,原作可以在GitHub中找到。主要改进了训练结果保存,并...

火箭船
09/23
0
0
入门项目数字手写体识别:使用Keras完成CNN模型搭建

对于图像分类任务而言,卷积神经网络(CNN)是目前最优的网络结构,没有之一。在面部识别、自动驾驶、物体检测等领域,CNN被广泛使用,并都取得了最优性能。对于绝大多数深度学习新手而言,数...

【方向】
05/25
0
0

没有更多内容

加载失败,请刷新页面

加载更多

nuc970 uboot nand-boot,kernel, filesystem 烧录位置

一 烧写到Nand Flash **1.1 **相关文件说明 l BSP版本:nuc970bsp-release-20150519.zip l NuWriter版本:2015/04/28-V01,nuvoTon Nu-Writer V1.0 l 烧写文件: u-boot-spl.bin:负责将u-b......

CookieDemo
35分钟前
1
0
python中sort和sorted函数小结

L.sort(cmp=None, key=None, reverse=False) sorted(iterable, cmp=None, key=None, reverse=False) 这样看,sorted函数只比sort函数多一个iterable参数,其余没什么不同,iterable是一个迭代......

上官夏洛特
今天
3
0
thinkphp 常用SQL执行语句总结

第一条:Db::tablera('vr_panomas')->where(['delete_time'=>0,'id'=>['in',$pids]])->field(['id'=>'id','post_thumb'=>'thumb','post_title'=>'title','post_tags'=>'tags','post_price'=>......

koothon
今天
3
0
支付宝返回状态resultStatus意思

上一篇集成支付宝的时候,会有一些支付宝返回的resultStatus,具体意思是: 9000 订单支付成功 8000 正在处理中 4000 订单支付失败 6001 用户中途取消 6002 网络连接出错 还有memo,意思就是...

RainOrz
今天
3
0
electron webview 页面加载事件顺序

1.did-start-loading 页面开始加载 2.load-commit 主页面文档加载 3.page-title-updated title 4.dom-ready 主页面 dom 加载完成 5.load-commit frame文档加载 6.did-frame-finish-load fram......

dubox
今天
3
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部