ElasticSearch 插件安装及其搜索略解

2015/12/17 22:21
阅读数 146

(1) marvel 远程安装方式: bin/plugin -i elasticsearch/marvel/latest

本地安装方式: wget https://download.elasticsearch.org/elasticsearch/marvel/marvel-latest.zip bin/plugin -i marvel -u file:/home/elasticsearch-1.5.1/marvel-latest.zip 在启动后,可以通过以下方式查看elasticsearch运行情况 http://xxx.xxx.xxx.xxx:8765/_plugin/marvel/

(2) elasticsearch service [非常喜欢] https://github.com/elastic/elasticsearch-servicewrapper 将service文件放置在elasticsearch bin 目录下 mv elasticsearch-servicewrapper-master/service/ bin/ 配置bin/service/elasticsearch.conf vim bin/service/elasticsearch.conf 按需作如下修改 set.default.ES_HOME=/home/elasticsearch-1.5.1 #替换为实际的elasticsearch路径 wrapper.java.command=/usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java #替换为实际的java二进制文件路径

(3). ElasticHQ [非常喜欢] http://www.elastichq.org/ bin/plugin -i royrusso/elasticsearch-HQ -u file:/home/elasticsearch-1.5.1/royrusso-elasticsearch-HQ-603ae9e.zip

在启动后,可以通过以下方式查看elasticsearch运行情况 http://xxx.xxx.xxx.xxx:8765/_plugin/HQ/

(4) elasticsearch-head [比较喜欢] https://github.com/mobz/elasticsearch-head bin/plugin -i mobz/elasticsearch-head -u file:/home/elasticsarch-1.5.1/elasticsearch-head-master.zip

在启动后,可以通过以下方式查看elasticsearch运行情况 http://xxx.xxx.xxx.xxxx:8765/_plugin/head

第四步:启动elasticsearch bin/service/elasticsearch start|stop|console|install|remove

start 在后台运行elasticsearch stop 停止elasticsearch console 在前台运行elasticsearch install elasticsearch自启动 remove elasticsearch取消自启动

二、基本操作

首先我们批量导入示例数据——莎士比亚全集 (参照http://kibana.logstash.es/content/v3/10-minute-walk-through.html kibana 3指南10分钟入门 wget http://www.elasticsearch.org/guide/en/kibana/3.0/snippets/shakespeare.json

curl -XPUT http://localhost:8765/_bulk --data-binary @shakespeare.json more shakespeare.json 察看存储内容 {"index":{"_index":"shakespeare","_type":"act","_id":0}} {"line_id":1,"play_name":"Henry IV","speech_number":"","line_number":"","speaker":"","text_entry":"ACT I"} {"index":{"_index":"shakespeare","_type":"scene","_id":1}}

接下来我们来通过与熟悉的关系数据库来对比elasticsearch的数据组成 (1)数据组成:元数据+实际数据 相当于察看数据库的模式定义 http localhost:8765/shakespeare/ 返回 { "shakespeare": { "mappings": { "act": { "properties": { "line_id": { "type": "long" }, "line_number": { "type": "string" }, "play_name": { "type": "string" }, "speaker": { "type": "string" }, "speech_number": { "type": "long" }, "text_entry": { "type": "string" } } }, "line": { "properties": { "line_id": { "type": "long" }, "line_number": { "type": "string" }, "play_name": { "type": "string" }, "speaker": { "type": "string" }, "speech_number": { "type": "long" }, "text_entry": { "type": "string" } } }, "scene": { "properties": { "line_id": { "type": "long" }, "line_number": { "type": "string" }, "play_name": { "type": "string" }, "speaker": { "type": "string" }, "speech_number": { "type": "long" }, "text_entry": { "type": "string" } } } }, "settings": { "index": { "creation_date": "1429691321987", "number_of_replicas": "1", "number_of_shards": "5", "uuid": "rrCmsKKcSDyLSpLFVnQnbg", "version": { "created": "1040299" } } } } }

我们用熟悉的关系数据库来进行对比,映射关系如下 elasticsearch RDBS indices 索引 databases数据库 types 类型 tables表 documents文档 rows行 fields 字段 columns列

示例中,索引名为shakespeare(等同于数据库名为shakespeare) 类型有3个:act, line, scene (等同于表名为act, line, scene) 字段组成(等同于表的结构) 字段名 字段类型 line_id long line_number string play_name string speaker string speech_number long text_entry string

(2)简单检索 示例1:通过index+type+文档_id来察看内容 格式:host:port/index_name/type_name/_id http localhost:8108/shakespeare/line/2

结果如下: { "_id": "2", "_index": "shakespeare", "_source": { "line_id": 3, "line_number": "", "play_name": "Henry IV", "speaker": "", "speech_number": "", "text_entry": "Enter KING HENRY, LORD JOHN OF LANCASTER, the EARL of WESTMORELAND, SIR WALTER BLUNT, and others" }, "_type": "line", "_version": 1, "found": true } elasticsearch的数据由两部分组成:文档元数据(例如_id)与文档数据 名字 说明 _index 类似RDBS的“数据库”概念 _type 类似RDBS的“表”概念 _id 文档的唯一编号 _source 字段里的内容为文档数据(真实存储的数据),我们可以使用如下方法只读取实际数据 http localhost:8108/shakespeare/line/2/_source 结果如下:

{ "line_id": 3, "line_number": "", "play_name": "Henry IV", "speaker": "", "speech_number": "", "text_entry": "Enter KING HENRY, LORD JOHN OF LANCASTER, the EARL of WESTMORELAND, SIR WALTER BLUNT, and others" }

示例2:指定字段field进行搜索,例如搜索play_name字段为Romeo and Juliet http localhost:8108/shakespeare/_search?q=play_name:"Romeo and Juliet" 结果如下(截取部分): { "_shards": { "failed": 0, "successful": 5, "total": 5 }, "hits": { "hits": [ { "_id": "86748", "_index": "shakespeare", "_score": 3.3792284, "_source": { "line_id": 86749, "line_number": "", "play_name": "Romeo and Juliet", "speaker": "JULIET", "speech_number": 19, "text_entry": "Exeunt" }, "_type": "line" },

(3)复杂搜索 Elasticsearch支持丰富而灵活的查询语言——Query DSL。 在学习之前,我们可以先熟悉一下Lucene查询语法(其实和使用google搜索引擎区别不大)

支持AND,OR,NOT 查询语句"apache AND lucene"的意思是匹配含apache且含lucene的文档。 查询表达式"apache OR lucene"能够匹配包含“apache”的文档,也能匹配包含"lucene"的文档,还能匹配同时包含这两个Term的文档。 查询表达式“lucene NOT elasticsearch”就只能匹配包含lucene但是不含elasticsearch的文档

支持+, -符号 例如:希望搜索到包含关键词lucene,但是不含关键词elasticsearch的文档,可以用如下的查询表达式:"+lucene -elasticsearch"。

支持指定字段名进行搜索(类似RDBS按列名搜索) 例如:查询title域中包含关键词elasticsearch的文档,查询表达式如下:title:elasticsearch

支持通配符 ? (匹配单个字符)

  • (匹配多个字符) 注意默认的通配符不能是关键词的首字母

支持~整数符号 一个~符号,后面紧跟一个整数,~后面的整数表示短语中可接收的最大的词编辑距离(短语中替换一个词,添加一个词,删除一个词) "writer~2"能够搜索到含writer和writers的文档。 title:"mastering elasticsearch"~2能够搜匹配title域中含"mastering elasticsearch"的文档与包含"mastering book elasticsearch"的文档

支持^符号进行加权boost设置 一个^符号后面接一个浮点数表示权重。如果权重小于1,就会降低关键词的重要程度。同理,如果权重大于1就会增加关键词的重要程度。默认的加权值为1

支持区间搜索 price:[10.00 TO 15.00查询price域的值在10.00到15.00之间的所有文档。 price:[10.00 TO 15.00}查询price域中价格在10.00(10.00要能够被搜索到)到15.00(15.00不能被搜索到)之间的文档

特殊字符需转义 +, -, &&, || , ! , (,) , { } , [ ] , ^, " , ~, *, ?, : , , /

更多,Lucene原理 (打分算法,TF-IDF算法一定会在搜索中出境)

我们可以看到elasticsearch支持丰富的数据查询方式,结果展示方式(按什么方式来排序结果,使用什么图形来展示统计结果) (1)关键词查询term (2)短语查询phrase (3)区间range (4)布尔Boolean (5)模糊fuzzy (6)跨度span (7)通配符wildcard (8)地理位置spatial (9) 统计aggregation ——这个功能非常非常赞,比如说生成各种统计图表 (10)prospective search

搜索语句支持通过URI提交(上面的例子演示的_search?q= 注意,使用这种方式的要遵循url编码,官方参考) ,也支持通过request body提交,简直就是HTTP RESTFULL最佳实践,官方参考

我们用熟悉的SQL语句来对比 实例1: curl -XPOST 'http://localhost:8108/shakespeare/line/_search?pretty' -d ' { "query":{ "match_all": {} }, "sort": {"line_id": {"order": "desc" }}, "size": 1, "from": 10 }' 等同于 use shakespeare; select * from line order by line_id desc limit 10,1 实例2: curl -XPOST 'http://localhost:8108/shakespeare/line/_search?pretty' -d ' { "query":{ "bool":{ "must":[ {"match_phrase": {"text_entry":"question"}},

{"match_phrase": {"text_entry":"not to be"}} ] } } }' 结果

"took" : 253, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "failed" : 0 }, "hits" : { "total" : 2, "max_score" : 4.0433946, "hits" : [ { "_index" : "shakespeare", "_type" : "line", "_id" : "34229", "_score" : 4.0433946, "_source":{"line_id":34230,"play_name":"Hamlet","speech_number":19,"line_number":"3.1.64","speaker":"HAMLET","text_entry":"To be, or not to be: that is the question:"} }, { "_index" : "shakespeare", "_type" : "line", "_id" : "1397", "_score" : 4.0004296, "_source":{"line_id":1398,"play_name":"Henry IV","speech_number":152,"line_number":"2.4.392","speaker":"FALSTAFF","text_entry":"blackberries? a question not to be asked. Shall"} } ] } } 等同于 use shakespeare; select * from line where text_entry like "%question%" and text_entry like "%not to be%"

Search APIs Match Query APIs

展开阅读全文
打赏
0
5 收藏
分享
加载中
更多评论
打赏
0 评论
5 收藏
0
分享
返回顶部
顶部