文档章节

Watson Natural Language Understanding

q
 qwerttaa
发布于 2017/11/29 11:17
字数 1053
阅读 668
收藏 0

 

沃森机器人自然语言理解服务的开发教程,学习使用 IBM Bluemix Watson NLU服务 完成自然语言的文本内容分析。

导入 Watson Python SDK

可能需要手动安装Python SDK,可以通过命令:pip install --upgrade watson-developer-cloud 进行安装与更新。

In [1]:

from watson_developer_cloud import NaturalLanguageUnderstandingV1
from watson_developer_cloud.natural_language_understanding_v1 import (
    Features, EntitiesOptions, KeywordsOptions)

通过用户名和密码连接服务

用户名和密码都可根据实际情况在服务凭证信息中找到

In [2]:

username = '9461ca60-a53a-48d4-83da-89e52eeb2688'
password = 'qq4GhRKYyVAM'

In [3]:

nlu = NaturalLanguageUnderstandingV1(version='2017-12-01', username=username, password=password)

In [4]:

features = Features(entities=EntitiesOptions(), keywords=KeywordsOptions())

准备一些文本资料开始测试

需要注意的是该服务的中文版本仍在开发状态,所以建议大家用英文进行测试

In [5]:

texts = [
    '''Welcome to the official documentation of Godot Engine,
    the free and open source community-driven 2D and 3D game engine!
    If you are new to this documentation,
    we recommend that you read the introduction page to get
    an overview of what this documentation has to offer.''',
    
   '''Godot Engine is an open source project developed by a community of volunteers.
    It means that the documentation team can always use your feedback and help to
    improve the tutorials and class reference. If you do not manage to understand something,
    or cannot find what you are looking for in the docs,
    help us make the documentation better by letting us know!''' 
]

分析文本列表0的内容

In [6]:

nlu.analyze(features=features, text=texts[0])

Out[6]:

{'entities': [{'count': 1,
   'relevance': 0.992572,
   'text': 'Godot Engine',
   'type': 'Company'},
  {'count': 1, 'relevance': 0.338173, 'text': 'official', 'type': 'JobTitle'}],
 'keywords': [{'relevance': 0.908999, 'text': 'Godot Engine'},
  {'relevance': 0.741794, 'text': 'open source'},
  {'relevance': 0.702832, 'text': 'introduction page'},
  {'relevance': 0.674454, 'text': 'official documentation'},
  {'relevance': 0.660084, 'text': 'game engine'},
  {'relevance': 0.230546, 'text': 'overview'}],
 'language': 'en',
 'usage': {'features': 2, 'text_characters': 282, 'text_units': 1}}

我们可以看到服务为我们识别出来两个主题,即类型为“Company”的“Godot Engine”;和类型为“JobTitle”的“official”。除此之外还得到了一些关键词,如:Godot Engine,open source,introduction page,official documentation……,以及统计出了字符数量和文本单元数量。

封装方法并测试文本列表1的内容

In [7]:

NLUA = lambda text: nlu.analyze(features=features, text=text)

In [8]:

NLUA(texts[1])

Out[8]:

{'entities': [],
 'keywords': [{'relevance': 0.973304, 'text': 'open source project'},
  {'relevance': 0.838422, 'text': 'Godot Engine'},
  {'relevance': 0.624781, 'text': 'class reference'},
  {'relevance': 0.60158, 'text': 'documentation team'},
  {'relevance': 0.369586, 'text': 'docs'},
  {'relevance': 0.32839, 'text': 'feedback'},
  {'relevance': 0.28266, 'text': 'community'},
  {'relevance': 0.282402, 'text': 'volunteers'},
  {'relevance': 0.274328, 'text': 'tutorials'}],
 'language': 'en',
 'usage': {'features': 2, 'text_characters': 372, 'text_units': 1}}

测试更大的文本内容并将关键字保存为CSV文档

通过 Python 的 Pandas 数据分析框架将关键字信息保存在CSV文档中,便于使用Excel之类的工具进行浏览。

In [9]:

import pandas as pd

In [10]:

text = '''
Introduction

The Blender Game Engine (BGE) is Blender’s tool for real time projects,
from architectural visualizations and simulations to games.

A word of warning, before you start any big or serious project with the Blender Game Engine,
you should note that it is currently not very supported and that there are plans for its retargeting and
refactoring that, in the very least, will break compatibility. For further information,
you should get in touch with the developers via mailing list or IRC and read the development roadmap.

Use Cases and Sample Games

Blender has its own built in Game Engine that allows you to create interactive 3D applications or simulations.
The major difference between Game Engine and the conventional Blender system is in the rendering process.
In the normal Blender engine, images and animations are built off-line – once rendered they cannot be modified.
Conversely, the Blender Game Engine renders scenes continuously in real-time,
and incorporates facilities for user interaction during the rendering process.'''

In [11]:

response = NLUA(text)
response

Out[11]:

{'entities': [{'count': 1,
   'relevance': 0.325141,
   'text': 'Blender',
   'type': 'Company'},
  {'count': 1, 'relevance': 0.109868, 'text': 'BGE', 'type': 'Organization'}],
 'keywords': [{'relevance': 0.968481, 'text': 'Blender Game Engine'},
  {'relevance': 0.672281, 'text': 'normal Blender engine'},
  {'relevance': 0.642034, 'text': 'Blender’s tool'},
  {'relevance': 0.585455, 'text': 'conventional Blender'},
  {'relevance': 0.544693, 'text': 'real time projects'},
  {'relevance': 0.516305, 'text': 'Engine renders scenes'},
  {'relevance': 0.50586, 'text': 'interactive 3D applications'},
  {'relevance': 0.503355, 'text': 'rendering process'},
  {'relevance': 0.440065, 'text': 'architectural visualizations'},
  {'relevance': 0.42788, 'text': 'development roadmap'},
  {'relevance': 0.415892, 'text': 'mailing list'},
  {'relevance': 0.415685, 'text': 'major difference'},
  {'relevance': 0.411704, 'text': 'Sample Games'},
  {'relevance': 0.411643, 'text': 'user interaction'},
  {'relevance': 0.358522, 'text': 'simulations'},
  {'relevance': 0.348923, 'text': 'retargeting'},
  {'relevance': 0.326577, 'text': 'compatibility'},
  {'relevance': 0.325289, 'text': 'warning'},
  {'relevance': 0.324025, 'text': 'Introduction'},
  {'relevance': 0.322093, 'text': 'BGE'},
  {'relevance': 0.321985, 'text': 'IRC'},
  {'relevance': 0.320995, 'text': 'plans'},
  {'relevance': 0.320874, 'text': 'touch'},
  {'relevance': 0.320867, 'text': 'information'},
  {'relevance': 0.320457, 'text': 'word'},
  {'relevance': 0.319156, 'text': 'developers'},
  {'relevance': 0.319064, 'text': 'Cases'}],
 'language': 'en',
 'usage': {'features': 2, 'text_characters': 1050, 'text_units': 1}}

In [12]:

keywords = pd.DataFrame(response['keywords'])
keywords

Out[12]:

  relevance text
0 0.968481 Blender Game Engine
1 0.672281 normal Blender engine
2 0.642034 Blender’s tool
3 0.585455 conventional Blender
4 0.544693 real time projects
5 0.516305 Engine renders scenes
6 0.505860 interactive 3D applications
7 0.503355 rendering process
8 0.440065 architectural visualizations
9 0.427880 development roadmap
10 0.415892 mailing list
11 0.415685 major difference
12 0.411704 Sample Games
13 0.411643 user interaction
14 0.358522 simulations
15 0.348923 retargeting
16 0.326577 compatibility
17 0.325289 warning
18 0.324025 Introduction
19 0.322093 BGE
20 0.321985 IRC
21 0.320995 plans
22 0.320874 touch
23 0.320867 information
24 0.320457 word
25 0.319156 developers
26 0.319064 Cases

In [13]:

keywords.to_csv('keyowrds.csv')

© 著作权归作者所有

共有 人打赏支持
q
粉丝 12
博文 21
码字总数 16694
作品 0
私信 提问
创建认知型行业聊天机器人

通过 Watson Natural Language Understanding 来识别实体并使用 Watson Tone Analyzer 来检测客户情绪,从而创建改进对话流程的机器人。IBM Cloud,即刻免费体验!...

qwerttaa
05/07
0
0
Watson Natural Language Understanding

沃森机器人自然语言理解服务的开发教程,学习使用 IBM Bluemix Watson NLU服务 完成自然语言的文本内容分析。

qwerttaa
2017/12/04
0
0
从人脸识别到机器翻译:52个有用的机器学习和预测API

选自KDnuggets 机器之心编译参与:吴攀 人工智能正在成为新一代技术变革的基础技术,但从头开始为自己的应用和业务开发人工智能程序既成本高昂,且往往很难达到自己想要的性能表现,但好在我...

tkkzc3e6s4ou4
05/12
0
0
52个有用的机器学习和预测API(各个方向资源)

人工智能正在成为新一代技术变革的基础技术,但从头开始为自己的应用和业务开发人工智能程序既成本高昂,且往往很难达到自己想要的性能表现,但好在我们有大量现成可用的 API 可以使用。开发...

t7sfokzord1jaymsfk4
05/12
0
0
feiwang/Awesome-Chatbot

Awesome Chatbot Projects Chatbot ParlAI A framework for training and evaluating AI models on a variety of openly available dialog datasets. https://github.com/facebookresearch/P......

feiwang
2017/09/07
0
0

没有更多内容

加载失败,请刷新页面

加载更多

进程管理利器Supervisor--centos7下安装与卸载

目录 概述 环境准备 检查python环境 在线安装 配置Supervisor 启动并验证 概述 Supervisor的安装可以有在线安装和离线安装两种方式。安装方式取决于服务器是否联网,联网的话可采用在线安装,...

java_龙
15分钟前
2
0
spring系列汇总

spring spring-基于可扩展Schema的特性自定义标签 springmvc-同一个访问资源,多种视图解析方式 spring cache 配置使用aspectj模式织入 基于注解的spring缓存,轻松无侵入解决cache问题 spri...

细肉云吞
16分钟前
2
0
vsftpd搭建ftp

12月10日任务 14.4 exportfs命令 14.5 NFS客户端问题 15.1 FTP介绍 15.2/15.3 使用vsftpd搭建ftp exportfs命令 在nfs运行后的一段时间内,如果需要添加共享目录,这时需要修改exports文件,并...

robertt15
18分钟前
2
0
numpy常用操作

水平合并数组 import numpy as npa = [1,2,3]b = [4,5,6]np.hstack((a,b))# array([1, 2, 3, 4, 5, 6])c = [a,['a','b','c']]d = [b,['d','e','f']]np.hstack((c,d))#array([['1'......

datadev_sh
35分钟前
5
0
四种检测异常值的常用技术简述

摘要: 本文介绍了异常值检测的常见四种方法,分别为Numeric Outlier、Z-Score、DBSCAN以及Isolation Forest 在训练机器学习算法或应用统计技术时,错误值或异常值可能是一个严重的问题,它们...

阿里云官方博客
38分钟前
2
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部